You would properly have to read through the source code of the anova function to find out exactly what it expects. Alternatively, since these are all independent tests, you could probably speed it up quite a bit with some simple parallisation using e.g. foreach.

The lm function calls the lm.fit function internally, so there's no reason that lm.fit should be significantly faster, since they're both calling the same code. Check that you are using consistent settings for both.