Picard tools duplicate removal
Hi, I want to use PICARD tools markduplicates option, but after reading the manual I am still not sure I understand the method used. http://broadinstitute.github.io/picard/command-line-overview.html#MarkDuplicates It reads: "The MarkDuplicates tool works by comparing sequences in the 5 prime positions of both reads and read-pairs in a SAM/BAM file"
Does this mean duplicates are marked based on their chr+start position and the 5'-sequence? or does the tool take the full sequence into account by using the CIGAR data?
Thanks in advance.
0 answers
No answers yet.
Log in to answer this question.
More posts like this
-
Can MarkDuplicates of Picard be used for RNA reads?
written by Nemo •
-
Picard MarkDuplicates from bam files
written by bandanaschapagain •
-
MarkDuplicates of a queryname sorted bam more expensive than a coordinate sorted bam
written by drkennetz
-
Samtools rmdup and Piccard Markduplicates
written by Prakash
-
How is 'ESTIMATED_LIBRARY_SIZE' computed in picard tools?
written by abhishekniroula7
-
Mark duplicates for single end reads: Why only 5'end?
written by dariober
-
picard Hs metrics and MarkDuplicates
written by User6891
-
To remove or to keep duplicates in alignment of NGS paired reads to a draft genome (set of assemble…
written by misaghb •
-
Why Do We Need Markduplicates For Variants Detection In Gatk Processing Pipeline?
written by Lds •
-
About Paired-End Sequencing
written by Pierre Lindenbaum
Will the answer to this question influence your decision to use it or not in any way?
Yes. Duplicate removal had influenced my results dramatically in the past.
Hope you do not want to remove duplicates from RNA-seq data, as the tags of your post suggest?
That is exactly why this operation is so dangerous. You better be sure that the removed duplicates are all artificial and not a natural effect of the high coverage.
There is a common myth floating around that "duplicates" are a synonym of "error". That is a remnant of the past when coverages were typically low.
I don't doubt you, but do you have a source for this? I am new to RNA seq and what I have read is inline with the "myth" you're referring to. I would like to know more about whether or not I should be removing duplicates.
You can google for papers (mostly newer ones) which used Unique Molecular Identifiers (UMIs) to investigate how many of the observed duplicates are actually based on PCR redundancy and which are based on coverage. The current consensus, from what I know, is that in targeted assays one generally does not remove duplicates as it would remove too many non-technical duplicates. I in fact know of no pipeline that would remove RNA-seq duplicates, here it is generally well accepted to go with the reads/counts as-observed rather than deduplicating the experiment.
See https://dnatech.genomecenter.ucdavis.edu/faqs/should-i-remove-pcr-duplicates-from-my-rna-seq-data/ for references
Keeping in mind @ATPoint's note, if you do want to remove PCR/optical duplicates for other reasons then use Clumpify (A: Introducing Clumpify: Create 30% Smaller, Faster Gzipped Fastq Files ) It does not need the data to be aligned and works from sequences.