Update: The site is down even though it was published in Nature Communications only 2 years ago. http://www.nature.com/articles/ncomms7275
I have just discovered the following tool: The GCAT benchmarking tool is a web-application with which one can compare software and pipelines for read alignment and variant calling. It provides a couple of challenge datasets and precomputed results for different variant calling pipelines, which can be compared based on plots of different statistics, eg. Venn-diagrams to depict overlap of variant calls. It also allows to test your own software and pipelines by uploading the results and doing a comparison with pre-computed pipelines.
From the web-site:
Have you ever wanted to benchmark genome analysis tools? With GCAT, it is now incredibly simple. Genome Comparison & Analytic Testing (GCAT) was developed as a platform to facilitate easy testing, comparison, and benchmarking of tools. Want to know how BWA + GATK Unified Genotyper fares against other analysis pipelines? Check out the report.
It appears to me that this kind of benchmarking application can be highly useful for evaluation of variant calling pipelines, and in order to establish a rational approach to choice of software and parameter settings.
Certainly, http://www.bioplanet.com/forum/categories/gcat-general currently is a dead link, which is unfortunate.
I'm the primary developer behind GCAT, and it's great to see other folks as excited about it as we are!
We're hoping that GCAT can help to standardize metrics for comparing the accuracy of NGS software, and to that end we encourage everyone to provide feedback and suggestions on the GCAT Forum.
Hi Dean,
please let us know where this site went, the forum is down as well. Or share the data and code under an open-source license!
Hi Dean, I did not use the tool for a while and am sad to see it down. I second the others, is there anyway you could get the server back online.
Log in to answer this question.
GCAT is impressive; It was presented in just concluded BIOIT conference in Boston.
hi folks, http://www.bioplanet.com/GCAT is down, it would be helpful if this site is restored as it enables us to perform exome validation on NA12878 sample.
it's probably just down for maintenance, as it was running yesterday.
ok, good to know, thanks, hoping to see the GCAT up and running soon.
It looks like the http://bioplanet.com/GCAT is completely moved to somewhere else, and does anyone know where I can access that nice web-application tool "GCAT benchmarking tool" ?
We believe that this site is not available anymore.
Site is down, maybe to lack of funding, interest, etc. This is unfortunate but underscores the need for sustainable bioinformatics resources. If you are the owner of this site or if anyone knows a comparable service, or has access to or archived the underlying code and data, please share it with us.
For this reason we have closed this question, too bad!
If you disagree please tell us why in a reply below, we'll be happy to talk about it.
Cheers!