Add a test for this case to verify the messages. This test will also be valuable to cover this code path in which we've had several regressions recently.
