How to remove redundant sequences from fasta file ?

I've fasta file containing nucleotide sequences. How can I remove the redundant sequences?I'm trying to access cd-hit but web server is not available. Is there any other tool available for removing redundancy? I really appreciate any help or suggestion!

fasta cd-hit

• 1,591 views • link

updated 3.4 years ago by Ram

• written 4.9 years ago by ANAM •

seqkit rmdup can remove duplicated sequences in a fasta file.

printf '> A1\nATTG\n> A2\nTTTA\n> A3\nATTG' | seqkit rmdup -sP

> A1
ATTG
> A2
TTTA
[INFO] 1 duplicated records removed

• 0 views • link

written 4.9 years ago by rpolicastro

$ printf '>A1\nATTG\n>A2\nTTTA\n>A3\nATTG\n' | awk '/^>/ NR > 1 {getline seq; print $0,seq}' | sort -uk2,2 | tr -s " " "\n"

works if sequence is in a single line.

• 0 views • link

written 4.9 years ago by cpad0112

0 answers

No answers yet.

More posts like this

Cut down CD-HIT-EST time
written 3.8 years ago by Neeraja •

Hello, I am trying to run my fasta file with ~1.98M sequences with 80% threshold on CD-HIT-EST and it seems to take a really long …
How to separate sub-families from transposons sequence based fasta files?
written 5.0 years ago by ANAM •

I'm working on the classification of transposable elements. I want to retrieve sequences of their sub-classes in separate files. Is there any code or tool …
Copy number alteration prediction from DNA-seq
written 5.1 years ago by ANAM •

I want to do project on copy number alteration prediction from DNA-seq. I have selected cancer type 'Breast cancer'. Dataset requirement is genes, their CNA …
cd-hit log file
written 7.0 years ago by saber mohammadi •

Hey everybody. I'm trying to cluster set of sequences by cd-hit. I'm using cd-hit on linux by the way. I'm wondering how can I know …
how to speed up cdhit clustering?
written 7.8 years ago by bitpir

I'm trying to run CDHIT to cluster ~250M of cds at nucleotide/protein levels. These are mostly NR-like sequences from NCBI. According to the paper it …
CD-HIT can not complete remove redundant transcript
written 8.9 years ago by xioli2013 •

I tried to use CD-HIT-EST to remove redundancy in trinity de novo transcripts however I can still see redundant annotations, cd-hit-est -i trinity.fasta -o clstr_out …
How to cluster transcripts assembled using TRINITY into unigene after blast?
written 9.3 years ago by sbchua.1990 •

I have assembled a transcriptome using TRINITY with 100,000 transcripts. I have perform blastx against NR database (-outfmt 5). I want to remove redundancy in …
Remove redundancy from GenBank plasmid database using cd-hit-est
written 10.2 years ago by wanderingstefan •

Hey all, I have the following problem. I have a plasmid sequence database (ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/plasmid/) that is heavily redundant. I have been trying to remove redundancy …
DNA sequence clustering tools
written 10.2 years ago by ngs101.com

Hi guys, I have about 300,000 sequences stored in a fasta file. I am trying to reduce the redundancy of these sequences. I used CD-HIT-EST …
Go Terms Redundancy Remover
written 14.1 years ago by Woa

<p>Can anybody recommend a GO term redundancy removing stand alone software like the REVIGO Web server? Is there any R, Perl or Python package available …