Heritability is the proportion if the variance explained by the genetic polymorphisms. There are severalheritabilities and I guess here it is more the additive heritability (also names narrow sense I think, but definitions vary somehow). For a quantitative trait in an unselected population, it can ne related to R2 when you know all the truly associated SNPs (you need to account for the SNPs'variance). Otherwise you need some methods of correlation between genetic identitiy by descent and phenotypic ressemblance for pairs of individuals.

See below a modern version - and in general you may follow the work of Visscher's team (in human genetics, don't know for others).

http://www.sciencedirect.com.gate2.inist.fr/science/article/pii/S0002929711000206

Thanks Genotypes group for really making it clear in a simple way. It was also interesting. I also want to ask one more question, how to calculate the variation explained by all the significant SNPs in combined, because tool I am using only calculating it for only individual SNP basis for a particular trait but some paper mentioning that all the significant SNPs contributed this % of variation for this trait. Can you please explain. Thanks,

Vinod