Hi Elisa,

At the most simple level, you could use different somatic variant callers on the same dataset, and then derive test statistics from that. For example, use 3 somatic variant callers, create a 'gold standard' dataset that has variants called in all 3 variant callers, and then derive test statistics for each caller (sensitivity, specificity, precision, accuracy) by comparing these to the gold standard..

Kevin

I would suggest using germline sequencing data from two individuals, and mixing a low percentage of reads of individual B with individual A. Variant private to individual B should show up as somatic variants when analyzing the combined data.

You can use hap.py for benchmarking somatic calls as well as germline. We are developing a benchmarking framework at Truwl.com to automate this process. There have been a number of somatic benchmarking papers, some of which have included gold standard calls. If you can get your pipeline into WDL we'd be happy to work with you to evaluate it. Just drop us a line.