This is a test version of Biostars. For the public version, visit https://www.biostars.org.
how to grep list of words from a file to another file

I have a list of words in a file (file 1) and large genomic gff information in another file (file 2). I want to grep information from file 2 of matching words from file 1 in output file (file 3) and also want to generate a file of words which are not matching in file 2. Please see the example below:

File 1:
NODE_55_length_30858_cov_27.421
NODE_54_length_29424_cov_167.508
NODE_22_length_84792_cov_25.8257
NODE_38_length_25225_cov_29.0986

File 2: (a large gff file, not showing )

##gff-version 3
##sequence-region NODE_1_length_232048_cov_20.4417 1 232048
...................................


OUTPUT 1:

##sequence-region NODE_54_length_29424_cov_167.508 1 29424
NODE_54_length_29424_cov_167.508    Prodigal:2.6    CDS 454 975 .   -   0   ID=LGE3207_04049;Name=insJ;gene=insJ;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:G7776-MONOMER;locus_tag=LGE3207_04049;product=IS150 protein InsA
NODE_54_length_29424_cov_167.508    Prodigal:2.6    CDS 1026    1745    .   -   0   ID=LGE3207_04050;inference=ab initio prediction:Prodigal:2.6;locus_tag=LGE3207_04050;product=hypothetical protein

OUTPUT 2: (not found words)

NODE_55_length_30858_cov_27.421
NODE_22_length_84792_cov_25.8257
NODE_38_length_25225_cov_29.0986
sequencing alignment genome
• 1,344 views • link
written by Kumar Kumar

Hello Manoj!

We believe that this post does not fit the main topic of this site.

Please restructure your "answers". Once that's done, I'll re-open the post.

For this reason we have closed your question.

If you disagree please tell us why in a reply below, we'll be happy to talk about it.

Cheers!

• 0 views • link
written by Ram Ram

4 answers

grep -f my_regexes my_text

Example:

$ cat my_regexes 
word
anotherone
somethingelse

$ cat my_text 
The is a word
Another sentence
and somethingelse
bubblegum

$ grep -f my_regexes my_text 
The is a word
and somethingelse

and -v looks for lines that do not match.

$ grep -v -f my_regexes my_text 
Another sentence
bubblegum

So to get your second case, add the -o switch to return just the matched part, save that to a file, and then grep your input regex file with your -o matches.

$ grep -o -f my_regexes my_text  | sort | uniq > matched_regexes
$ grep -v -f matched_regexes my_regexes 
anotherone
• 0 views • link
written by mbelmadani mbelmadani

This command taking a long time without any results...

grep -f words_list.fasta marge.gff >out_file.txt

• 0 views • link
written by Kumar Kumar

How many expressions are you searching, and how many lines in your gff?

It looks like you're trying to match a FASTA file to a GFF file, is that correct? You example seemed different. I just tested with the exact input your provided and it works here. If your inputs are different than what was in your original question, could you show an example?

$ cat example.gff 
NODE_54_length_29424_cov_167.508    Prodigal:2.6    CDS 454 975 .   -   0   ID=LGE3207_04049;Name=insJ;gene=insJ;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:G7776-MONOMER;locus_tag=LGE3207_04049;product=IS150 protein InsA
NODE_54_length_29424_cov_167.508    Prodigal:2.6    CDS 1026    1745    .   -   0   ID=LGE3207_04050;inference=ab initio prediction:Prodigal:2.6;locus_tag=LGE3207_04050;product=hypothetical protein

$ cat inputs.txt 
NODE_55_length_30858_cov_27.421
NODE_54_length_29424_cov_167.508
NODE_22_length_84792_cov_25.8257
NODE_38_length_25225_cov_29.0986

$ grep -f inputs.txt example.gff 
NODE_54_length_29424_cov_167.508    Prodigal:2.6    CDS 454 975 .   -   0   ID=LGE3207_04049;Name=insJ;gene=insJ;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:G7776-MONOMER;locus_tag=LGE3207_04049;product=IS150 protein InsA
NODE_54_length_29424_cov_167.508    Prodigal:2.6    CDS 1026    1745    .   -   0   ID=LGE3207_04050;inference=ab initio prediction:Prodigal:2.6;locus_tag=LGE3207_04050;product=hypothetical protein

Also, is this linux or a mac? Can you do grep --version and show the output?

• 33 views • link
written by mbelmadani mbelmadani

There are total 36430 number of expressions are in file 1 and gff file contains nodes list, descriptions and seqs. Here is a part of gff file.

Version is :: grep (GNU grep) 3.1

##gff-version 3
##sequence-region NODE_1_length_232048_cov_20.4417 1 232048
##sequence-region NODE_2_length_219754_cov_21.8907 1 219754

NODE_1_length_232048_cov_20.4417    Prodigal:2.6    CDS 173 892 .   +   0   
ID=LGE3336_00001;Name=yedW_1;gene=yedW_1;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:G7057-MONOMER;locus_tag=LGE3336_00001;product=putative DNA-binding response regulator in two-component system with YedV
NODE_1_length_232048_cov_20.4417    Prodigal:2.6    CDS 896 2197    .   +   0   ID=LGE3336_00002;Name=envZ_1;gene=envZ_1;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:ENVZ-MONOMER;locus_tag=LGE3336_00002;product=EnvZ sensory histidine kinase

NODE_2_length_219754_cov_21.8907    Prodigal:2.6    CDS 1776    2348    .   +   0   ID=LGE3336_00225;inference=ab initio prediction:Prodigal:2.6;locus_tag=LGE3336_00225;product=hypothetical protein
NODE_2_length_219754_cov_21.8907    Prodigal:2.6    CDS 2383    3294    .   -   0   ID=LGE3336_00226;inference=ab initio prediction:Prodigal:2.6;locus_tag=LGE3336_00226;product=hypothetical protein


>NODE_1_length_232048_cov_20.4417
GAGTGTTTATTTTTGAGCTTTATTTAAAACTTTTTGTAATTCAACATAGTTGCTTAACAC
..........................
• 0 views • link
updated by Ram Ram • written by Kumar Kumar

This should be a comment-reply to manuel's comment. Please make the necessary changes:

  1. Copy the contents of your reply from this answer (you can edit this answer (Ctrl/Cmd + click the link to open it in a new tab) and do a Select All -> Copy there).
  2. Click on Add Reply on manuel 's comment here: C: how to grep list of words from a file to another file
  3. Paste the copied text
  4. Click on the green Add Comment button
  5. Click on moderate back in your answer here: A: how to grep list of words from a file to another file
  6. Choose Delete Post
  7. Click on the blue Submit button.
• 0 views • link
written by Ram Ram

If I understand correctly, you would be grepping >NODE_1_length_232048_cov_20.4417, not NODE_1_length_232048_cov_20.4417, so that wouldn't work. I'm assuming the sequence is also not of interest since you only care about the header?

If that's the case, preprocess your fasta file into expressions by keeping only the header lines and removing the leading > before matching. i.e. grep ">" my.fasta | sed 's|^>||g' > my_headers_as_patterns

• 0 views • link
written by mbelmadani mbelmadani

joining:

    join -t $'\t' -1 1 -2 1 <(sort -t $'\t' -k1,1 file1) <(sort -t $'\t' -k1,1 file2)

not found

    join -t $'\t' -v 1 -1 1 -2 1 <(sort -t $'\t' -k1,1 file1) <(sort -t $'\t' -k1,1 file2)
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

Hi, My interest is in grepping annotation (an example below) based on the list in file 1 not header. For the single node here is command that I tried, but I have a list of words,

grep -w 'NODE_1_length_232048_cov_20.4417' marge.gff >out.txt

NODE_1_length_232048_cov_20.4417 Prodigal:2.6 CDS 173 892 . + 0 ID=LGE3336_00001;Name=yedW_1;gene=yedW_1;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:G7057-MONOMER;locus_tag=LGE3336_00001;product=putative DNA-binding response regulator in two-component system with YedV

NODE_1_length_232048_cov_20.4417 Prodigal:2.6 CDS 896 2197 . + 0 ID=LGE3336_00002;Name=envZ_1;gene=envZ_1;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:RefSeq:ENVZ-MONOMER;locus_tag=LGE3336_00002;product=EnvZ sensory histidine kinase NODE_1_length_232048_cov_20.4417

• 0 views • link
written by Kumar Kumar

Hi,

Please add your replies to the appropriate comments/answers. Until then, I'm closing your post to prevent further answers being added..

• 0 views • link
written by Ram Ram

Log in to answer this question.

More posts like this