This is a test version of Biostars. For the public version, visit https://www.biostars.org.
removing fasta sequences that have Ns in it in a fasta file

Hi everyone,

i have a giant fasta file, but some of the sequences have got Ns in them

GeneID:107003026

AAATTTACTTGTCCTTGTGAT

GeneID:107005138

TATGCACNNNGGTTGC

GeneID:107004481

GATTTTATGTTGCTGAA

so the second one has got Ns in them, what can i do to get rid of the whole sequence so that the outcome would look like this? thank you very much

GeneID:107003026

AAATTTACTTGTCCTTGTGAT

GeneID:107004481

GATTTTATGTTGCTGAA

fasta
• 9,012 views • link
updated by Brian Bushnell Brian Bushnell • written by kws15 • kws15

did you search for similar posts on biostars.org ? what did you find ? what have you tried ?

• 1 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

That doesn't look like a FASTA file (no ">").

• 1 views • link
written by igor igor

yeah, there are some '>' s in my file, they are just gone when i posted them here for some reason

• 1 views • link
written by kws15 • kws15

If you want to use only the default unix tools, you can use grep to filter out Ns (assuming your sequence names do not have Ns):

grep -v "N" in.fa

Then filter out empty records (where sequence was removed by grep):

awk '$2{print RS}$2' FS='\n' RS=> ORS= in.fasta

• 1 views • link
written by igor igor

I do not recommend this, as it is unsafe. A good solution should handle all possible Fasta variants, whether they are multi-line, contain 'N' in headers, etc.

• 1 views • link
written by Brian Bushnell Brian Bushnell

It didn't look like any of those sequences were in danger of being multi-line. To be safe, you can convert multi-line fasta to single-line fasta:

awk '/^>/ {printf("\n%s\n",$0);next; } { printf("%s",$0);} END {printf("\n");}'
• 1 views • link
written by igor igor

There are many ways to do this correctly, but are you sure you want to? What is your rationale?

• 1 views • link
written by Brian Bushnell Brian Bushnell

i am doing analysis on promoter sequences for two close species in which i would need to align them together and see the similarity , so i would need to use may be blast, but it just gives me error when i tried to do that in R when sequences contain Ns, so i guess i would just have to ignore the sequences that have Ns.

• 1 views • link
written by kws15 • kws15

2 answers

Python, using Biopython

import sys
from Bio import SeqIO
handle = open(sys.argv[1], "rU")
filtered = [record for record in SeqIO.parse(handle, "fasta") if record.seq.count('N') == 0]
output_handle = open("N_removed.fasta", "w")
SeqIO.write(filtered, output_handle, "fasta")
output_handle.close()
handle.close()

Save script (e.g. removeNfromfas.py) and execute as python removeNfromfas.py <yourfile.fasta>

Updated version 19 months later:

import sys
from Bio import SeqIO
for record in SeqIO.parse(sys.argv[1], "fasta"):
    if record.seq.count('N') == 0:
        print(record.format("fasta")

Save script (e.g. removeNfromfas.py) and execute as `python removeNfromfas.py yourfile.fasta > newfile.fasta More flexible and can handle enormous files, if necessary. Lower memory requirements.

• 1 views • link
written by WouterDeCoster WouterDeCoster

thank you very much , that works!

• 1 views • link
written by kws15 • kws15

Using pyfaidx and awk:

• 1 views • link
written by Matt Shirley Matt Shirley

Log in to answer this question.

More posts like this