At first, you need to ask yourself a question. What is the main purpose of use? Since the GPU itself does not guarantee better performance. You need to have specialized tools which can utilize the GPU and can get the most of it. Which software are you planning to use?

I assume that you would like to build a powerful computational cluster. There are additional aspects you have to bear in mind, but they are rather technical. Such as sufficient power source - these GPUs are really greedy, reasonable CPU (not all the calculations are done with GPU, cooling... If you are unexperienced with these topics I would strongly encourage you to ask someone who is, perhaps someone from you IT department or IT faculty? Before spending your own budget, are you aware of NVIDIA academic program? They are donating really powerful pieces of hardware free of charge for academic, non-profit purposes.

Thanks for all the comments. As a follow up question something that I cannot get clear: Memory and communication speed. So the third generation Tesla GPUs (Kepler) are equipped with on board memory. The K40 has like 12Gb, the K20 has 5Gb. In my experience, memory is always limiting but I have no clue how the GPUs use memory. Is the on board memory like cache memory? If I put like 32 gb additional memory on the mother, would that be the point where the speed of data transfer via the PCIe becomes important? Or is that just to communicate with the CPU?

Important question since the K20 appears NOT to function with PCI express gen3 and I likely will buy a K40 or a K20 (depending on....)