This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Tools to remove duplicate or substring reads

Is there any efficient tool to remove substring reads or duplicate reads from NGS data set? I know that readjoiner could remove the duplicated reads, but seems not work on substring reads. Thanks.

Example: Duplicates: read1: AGTCAT read2: AGTCAT In this case, only one read will be kept.

Substring: read1: GTCA read2: AGTCAT In this case, read1 will be removed.

next-gen sequencing alignment

2 answers

The most efficient tool for this purpose is Dedupe from the BBMap package. However, it requires all reads to be stored in memory, so it needs a lot of memory. Can you explain in more detail what you are trying to do?

Thanks for your help. I have added examples in the question.

Just noticed that SGA tool is able to do this. Collapse Reads That Are Substrings Of Other Reads In Same Library

Log in to answer this question.