In general, beta denotes the resulting coefficient from a fit and SE would be its standard error. Assuming that's about as clear as mud to you, let's restate that using statistics you're probably more familiar with...a T-test.

Suppose you have two experimental groups (we'll use human males and females) and perform a measurement on them (in this case, we'll just measure their height). If you were to graph the results you'd probably see that the males tend to be a bit taller than the females. If you calculated the mean of each group and subtracted them, then the result would be the expected difference in height due to gender. The is a simple example of a beta value. But of course unless you measured the height from ALL of the males and females in the world, then this isn't an exact value (even ignoring measurement error). Rather, since we only measured a subset of all people there's some error associated due to our sampling. This ends up becoming the standard error of the measurement. In the case of a T-Test, you can divide the beta value by the standard error and you have your T-statistic, which you would then use to find a p-value.

The methods used to get the beta and SE values are rather more complicated for GWAS, of course, but the underlying principles are the same. So as with the height example, the beta value and its error give you an idea of the effect size. A p-value is nice, but you also want to know if it's associated with a small but very consistent (and if it's really really small, do you even care about it?) or large but highly variable effect.