For all 3 samples the coverage for target bases at 20X - 92%, at 30X - 86%, 40X - 80%, 50X - 72% (Is this coverage good enough for exome sequencing?)

Those numbers look normal, but since those are averages it is still possible that there are regions with low coverage (unavoidable)

Variant calling using Unified Genotyper - GATK

Why not HaplotypeCaller?

What are the other general ways to reduce my variants?

We don't know which disease you are working on, but ExAC is probably a good resource to eliminate frequent variants. If you are looking for a highly penetrant mutation for a rare disease you can exclude all variants that are frequent (well above disease prevalence).

Sorry, I meant intersection, not union. "Haplotypecaller" is recommended by GATK team. Last week, I ran my 3 samples individually using "Haplotypecaller" with GVCF option. It failed 6 times due to memory issue. So that's the reason I first tried with "Unified Genotyper". But now I have used other high computing machine to generate calls using "Haplotypecaller-GVCF". I got calls following calls from "UnifiedGenotypeCaller - 930,000 vcf records" and "HaplotypeCaller - 1.4 million vcf records".

1) Why am I getting more number of calls in Haplotypecaller-GVCF? or That's pretty normal to expect from it.

2) Also, I noticed that 200,000 vcf records fall under the below contigs (chr1_gl000191_random,...chr4_ctg9_hap1,chrUn_gl0000210..etc). What about the variants in these unknown contigs in hg19 reference genome?

3) As per GATK best practices, I cannot go for the VQSR approach for filtering, because I have only 3 samples, but they say at least 25 exome-sequenced samples needed for VQSR. They suggest either hard filtering or doing variant calling by adding some exome bam files from 1000 Genomes project. But, I did the hard filtering for filtering the raw VCF. Which one is the best option for TRIO analysis?

Hypercholesterolemia is a familial inherited disease.

Currently, I am working on annotating my calls with ExAC (used release 0.3.1 from FTP broad website), Clinvar (used vcf_GRCh37 from FTP NCBI clinvar site), DBNSFP using SnpEff.

Is this for research purpose, clinical or something else?

You can also remove variants that are synonymous and add variants that are in splice sites (I prefer RefSeq transcrpts and ACMG recommends). You may find more recommendations in ACMG guidelines. Maybe in your case, the list of genes you are focusing on should be extended with the help of OMIM and HPO.

At ALAPY we developed ALAPY Genome Explorer to analyze vcf files, trios, filter variants based on different data sources like ExAC. This was done to test our database design, but you definitely can use it already for your study and it is free at the moment http://alapy.com/services/alapy-genome-explorer/ Register now to get free access. Also, you generated fastq files and we finished the first version of fastq compression program called ALAPY Compressor http://alapy.com/services/alapy-compressor/ It is also free and will reduce your fastq.gz (gzipped fastq) file sizes by 1.5 to 3 times. It is lossless compression.

We would love to hear back from you about ALAPY Compressor andALAPY Genome Explorer.

Thank you