This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Extract fasta files that are not empty from a directory and subdirectories

Dear biostars,

I have a directory containing a few hundred subdirectories, each containing 7 fasta files. A significant fraction of these appear to be empty fasta files. Does anyone have an idea how to extract only those fasta sequences that are not empty?

Cheers,

Sam

fasta recursively
• 2,294 views • link
updated by ATpoint ATpoint • written by samlambrechts299 samlambrechts299

Will they be truly empty, or will they have no sequence, but a header? Wouter's approach may not work if thats the case.

• 0 views • link
written by Joe Joe

can you try this? samlambrechts299

 find . -type f -name "*.fa" -exec awk 'NR % 2 == 0 {if (length >=1) print FILENAME}' {} \;
• 0 views • link
written by cpad0112 cpad0112

1 answer

A naive solution would be to use find to list all fasta files in the current and subdirectories and then simply check if beyond the headers they have any content:

function FindNonEmpty {

  if [[ $(grep -v '^>' $1 | head | awk NF | wc -l) > 0 ]]
    then
      $(realpath $1 >> not_empty.txt)
      else $(realpath $1 >> empty.txt)
    fi
}; export -f FindNonEmpty

find ./ -maxdepth 1000 -name "*.fa" | parallel FindNonEmpty {}

not_empty.txt will list all non-empty and empty.txt all empty files.

• 0 views • link
written by ATpoint ATpoint

Log in to answer this question.

More posts like this