This is a test version of Biostars. For the public version, visit https://www.biostars.org.
fasta redundant sequence

Hello all, I have downloaded GENCODE and RefSeq transcripts and I want to combine these and filter for redundancy. Please suggest me how to proceed. Thank you

redundant sequence removal
• 2,445 views • link
updated by cpad0112 cpad0112 • written by qudrat qudrat

Please post example input (one or two eg from gencode and refseq) and expected output.

• 0 views • link
written by cpad0112 cpad0112 
>uc001aal.1 CDS=1-916
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATTCTATGGAGGAATCGTGTTTGGAAACCTTCTTATTGTCATAACAGTGGTATCTGACTCCCAC
>uc001aak.4
CACACAACGGGGTTTCGGGGCTGTGGACCCTGTGCCAGGAAAGGAAGGGCGCAGCTCCTGCAATGCGGAGCAGCCAGGGCAGTGGGCACCAGGCTTTAGCCTCCCTTTCTCACCCTACAGAGGGCAG
>NM_001005484.1 CDS=1-916
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATTCTATGGAGGAATCGTGTTTGGAAACCTTCTTATTGTCATAACAGTGGTATCTGACTCCCAC
>XM_017003010.1 CDS=512-893
AAATATGGGATTCCTGGGTTTAAAAGTATAAAATAAATATGTTTAATTTGTTAACTGATTACTATCAGAATTGTACTGTTCTGTATCCCACCAGCAATGTCTAGGAATGCCTGTTTCTCCACAAAGTGTTT

The above two sequences are from GENCODE nad the last two sequences are from RefSeq so there are total four sequences, first sequence and third sequence are redundant but they have different id. I want one of these two sequence to be removed while merging.The result should be like this

>uc001aak.4
CACACAACGGGGTTTCGGGGCTGTGGACCCTGTGCCAGGAAAGGAAGGGCGCAGCTCCTGCAATGCGGAGCAGCCAGGGCAGTGGGCACCAGGCTTTAGCCTCCCTTTCTCACCCTACAGAGGGCAG
>NM_001005484.1 CDS=1-916
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATTCTATGGAGGAATCGTGTTTGGAAACCTTCTTA
>XM_017003010.1 CDS=512-893
AAATATGGGATTCCTGGGTTTAAAAGTATAAAATAAATATGTTTAATTTGTTAACTGATTACTATCAGAATTGTACTGTTCTGTATCCCACCAGCAATGTCTAGGAATGCCTGTTTCTCCACAAAGTGTTT
• 0 views • link
updated by Pierre Lindenbaum Pierre Lindenbaum • written by qudrat qudrat

Cluster the sequences with CD-HIT

• 0 views • link
written by Joe Joe

1 answer

with seqkit:

 $ seqkit rmdup -w 0 -s -i  test.fa --quiet

output:

>uc001aal.1 CDS=1-916
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATTCTATGGAGGAATCGTGTTTGGAAACCTTCTTATTGTCATAACAGTGGTATCTGACTCCCAC
>uc001aak.4
CACACAACGGGGTTTCGGGGCTGTGGACCCTGTGCCAGGAAAGGAAGGGCGCAGCTCCTGCAATGCGGAGCAGCCAGGGCAGTGGGCACCAGGCTTTAGCCTCCCTTTCTCACCCTACAGAGGGCAG
>XM_017003010.1 CDS=512-893
AAATATGGGATTCCTGGGTTTAAAAGTATAAAATAAATATGTTTAATTTGTTAACTGATTACTATCAGAATTGTACTGTTCTGTATCCCACCAGCAATGTCTAGGAATGCCTGTTTCTCCACAAAGTGTTT

Input:

$ cat test.fa 
>uc001aal.1 CDS=1-916
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATTCTATGGAGGAATCGTGTTTGGAAACCTTCTTATTGTCATAACAGTGGTATCTGACTCCCAC
>uc001aak.4
CACACAACGGGGTTTCGGGGCTGTGGACCCTGTGCCAGGAAAGGAAGGGCGCAGCTCCTGCAATGCGGAGCAGCCAGGGCAGTGGGCACCAGGCTTTAGCCTCCCTTTCTCACCCTACAGAGGGCAG
>NM_001005484.1 CDS=1-916
ATGGTGACTGAATTCATTTTTCTGGGTCTCTCTGATTCTCAGGAACTCCAGACCTTCCTATTTATGTTGTTTTTTGTATTCTATGGAGGAATCGTGTTTGGAAACCTTCTTATTGTCATAACAGTGGTATCTGACTCCCAC
>XM_017003010.1 CDS=512-893
AAATATGGGATTCCTGGGTTTAAAAGTATAAAATAAATATGTTTAATTTGTTAACTGATTACTATCAGAATTGTACTGTTCTGTATCCCACCAGCAATGTCTAGGAATGCCTGTTTCTCCACAAAGTGTTT
• 0 views • link
written by cpad0112 cpad0112

Log in to answer this question.

More posts like this
  • cut off value of fpkm
    written by mdfardin374 • mdfardin374
    Hello all, I have gtf files from Hisat2/Stringtie and I am interested in identification of novel genes so I want to filter genes based on …
  • sratoolkit and download
    written by qudrat qudrat
    Hello all, I have been trying to download a .sra file of 20GB using "wget" and "curl" command but due to bad connectivity of internet …
  • stringtie and fpkm and coverage
    written by qudrat qudrat
    Hello all, I have assembled transcripts from a tissue using StringTie and I want to filter transcripts so that I can get rid of false …
  • Cufflinks and fpkm
    written by qudrat qudrat
    Hello guys, Can somebody suggest me how to choose cut off fpkm value for a list transcripts generated from Cufflinks.
  • gtf file and rnaseq
    written by qudrat qudrat
    Hello all I was wondering to ask whether there are any method to find common transcripts between two gtf file? I have produced gtf files …
  • gene expression and RNA-Seq
    written by qudrat qudrat
    Hello All, I want to know through RNA-Seq analysis whether a particular transcript is getting expressed in a tissue or not. Please suggest me how …
  • transcript and fasta sequence
    written by qudrat qudrat
    Hello all! I have two file containing ten thousands of transcripts fasta sequences each with different ids and I am interested in finding common sequences …
  • FPKM threshold and filter
    written by qudrat qudrat
    Can somebody suggest me how to choose the minimum FPKM value as a threshold to filter low quality transcripts?
  • How to cluster transcripts assembled using TRINITY into unigene after blast?
    written by sbchua.1990 • sbchua.1990
    I have assembled a transcriptome using TRINITY with 100,000 transcripts. I have perform blastx against NR database (-outfmt 5). I want to remove redundancy in …
  • Human gut metagenome data
    written by glady glady
    Hello friends, I want to download all the human gut meta genome sequences available of the continent - ASIA. Can someone suggest me how should …