Do you know at which point you're having memory issues? You can always subset them and run them in multiple groups- that will work for getting a sense of the un-normalized data at least. For a normalization method like RMA which works across all of the samples, you'll need to be able to read in all the files. Also try closing everything you have running the background, and make sure you're running a 64-bit version of R. If you still have issues, you might need to track down a computer with 6 or 8GB RAM.