I am trying to reduce the size of a FASTA file that I got from the BLAST database archive. Some of the FASTA files they post already have identical sequences removed, but that still leaves a lot of very similar sequences. For example, I am working with "nt" and there are a lot of sequences in there that are very minor variations of each other or are overlapping. Is there a good way to combine those and eliminate "duplicate" entries?
1 answer
Try cd hit-est
They use cd hit to cluster protein sequences to make the UniRef databases. I presume you could do something similar for nucleotide sequences. Maybe someone already has.
•
written
by
Sean R Johnson
Maybe you could do some kind of iterative strategy to make it more manageable? Like breaking the database up into 100 parts, removing the redundancies in the sub-parts and then combining them, and removing the redundancies in the combinations?
It really is a huge database, though, maybe there isn't a computationally feasible way to do this. Or maybe you just need to use a cluster computer. I'm not sure.
Log in to answer this question.
-
Remove duplicates in multifasta, where entries are paired
written by SaltedPork
-
Merging all sequences with identical ID's
written by sativus •
-
Get all bacterial protein sequences from Genbank but non redundant
written by mschmid
-
Assembling long similar contigs
written by dcdanko •
-
BLASTing long sequence
written by igor
-
Duplicate_SeqIds Blast database
written by sukesh1411 •
-
Length of FASTA lines
written by modernsynthesis •
-
please help with blast database creation error
written by sohra •
-
Faster BLAST alternative
written by igor
-
Questions : Data management of whole exome sequencing
written by mangfu100
