I'm unhappy with a straight change in behaviour because it will break code that is currently catching AttributeError.

A slightly less invasive change would be to raise an AttributeError if the module doesn't exist, otherwise letting the original error propagate.

That means distinguishing between a module not existing and an ImportError raised whilst importing the module. Example code that does this by walking the stack: http://twistedmatrix.com/trac/browser/trunk/twisted/python/reflect.py#L382

In addition we could add a new method that loads a test from name, returning an 'ErrorHolder' if loading the test fails. (A TestCase that reraises the original error when run - test discovery already does this in fact so that a test module failing to load doesn't halt discovery.)