This is a test version of Biostars. For the public version, visit https://www.biostars.org.
how to remove duplicate SNP rows in vcf using bcftools norm

Hello,

I am trying to remove duplicate SNP rows from a multiple sample vcf file. SNPs have different positions but multiple duplicate rows. I tried using 

bcftools norm -d in.vcf -o out.vcf

but it does not work. Is there any other way to remove duplicates from vcf file that does not change the file format. Thank you!

snp
• 11,452 views • link
updated by brianaloredana • brianaloredana • written by evelyn evelyn

2 answers

That's also pretty strange for me! Neither bcftools norm nor bcftools concat did not remove the duplicates from my vcf file.

That's why I applied to another solution.

grep "#" myfile.vcf > header                   ## here you separate the header of
                                               ## your vcf file
grep -v "#" myfile.vcf | sort | uniq >> header ## here firstly you separate the vcf file 
                                               ## apart the header part, then sort it
                                               ## and remove the duplicates by using
                                               ## uniq command. Lastly you pass the
                                               ## output to the header.

I checked the file if it is still compatible to work with bcftools. Yes! It is!

• 0 views • link
updated by Ram Ram • written by brianaloredana • brianaloredana

Shouldn't it be:

bcftools norm -D in.vcf -o out.vcf

(uppercase D) ?

• 0 views • link
written by finswimmer finswimmer

@finswimmer, thank you! I have tried D as well but it just results in the same as input file without removing duplicates.

• 0 views • link
written by evelyn evelyn

Hm, do you have an example of your vcf file?

This works for me:

##fileformat=VCFv4.2
##contig=<ID=chr1,length=249250621>
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  1
chr1    977330  rs2799066   T   C   225 PASS    .   GT  0/1
chr1    977330  rs2799066   T   C   225 PASS    .   GT  0/1

$ bcftools norm -D in.vcf
##fileformat=VCFv4.2
##FILTER=<ID=PASS,Description="All filters passed">
##contig=<ID=chr1,length=249250621>
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##bcftools_normVersion=1.10.1+htslib-1.10.2
##bcftools_normCommand=norm -D 1.vcf; Date=Fri Feb  7 21:22:54 2020
#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  1
chr1    977330  rs2799066   T   C   225 PASS    .   GT  0/1
Lines   total/split/realigned/skipped:  2/0/0/0
• 0 views • link
written by finswimmer finswimmer

Thank you! I will check again.

• 0 views • link
written by evelyn evelyn

Log in to answer this question.

More posts like this