Dear all can anyone tell me how to parse this type of file using either perl or awk
AUO97_RS0005
AUO97_RS0005 alpha hydrolase wp_567465 GI:54365463
AUO97_RS0007
AUO97_RS0007 beta hydrolase wp_567465 GI:65456475
AUO97_RS0020
AUO97_RS0020 gamma hydrolase wp_567465 GI:4536473
I want to retrieve only only those values having data in next columns and remove duplicates. Output file:
AUO97_RS0005 alpha hydrolase wp_567465 GI:54365463
AUO97_RS0007 beta hydrolase wp_567465 GI:65456475
AUO97_RS0020 gamma hydrolase wp_567465 GI:4536473
•
written
by
sharmatina189059 •
How is the second line from above example separated ( space or TAB ) ?
If TAB delimited, you can extract the lines containing TAB delimited entries.
grep -P "\t" YOUR_FILE.txt > DESIRED_OUTPUT.txt
If the single column entries in the above example also has TAB in the end,
grep -P "\t.*\t" YOUR_FILE.txt > DESIRED_OUTPUT.txt
It is tab separated but i have done it using awk. awk -F '\t' '$2!=""' file2. but thank you so much for your reply.
I suspect this will generally be fastest on the larger inputs, by avoiding cat and sort and uniq operations:
$ awk 'NF > 1 && !a[$0]++' in.txt > answer.txt
It strips duplicate lines without needing to sort the input. It can be memory hungry, but memory is cheap and fast, these days. If you can do without sorting, go for it, I say.
Full bash solution, just for the sake of it!
#!/bin/bash
# Usage: bash scriptname.sh textfile.tsv
while read line ; do
read -a fields <<< "$line"
[ ! -z "${fields[1]}" ] && echo "$line"
done < $1
Or as a one-liner:
while read line ; do read -a fields <<< "$line" ; [ ! -z "${fields[1]}" ] && echo "$line" ; done < textfile.tsv
AUO97_RS0005 AUO97_RS0005 alpha hydrolase wp_567465 GI:54365463 AUO97_RS0007 AUO97_RS0007 beta hydrolase wp_567465 GI:65456475 AUO97_RS0020 AUO97_RS0020 AUO97_RS0020 gamma hydrolase wp_567465 GI:4536473
Sorry done by mistake.
$ cat test.txt
AUO97_RS0005
AUO97_RS0005 alpha hydrolase wp_567465 GI:54365463
AUO97_RS0007
AUO97_RS0007 beta hydrolase wp_567465 GI:65456475
AUO97_RS0020
AUO97_RS0020 gamma hydrolase wp_567465 GI:4536473
ouput: awk solution
$ awk '$2 == "" {next} {print}' test.txt
AUO97_RS0005 alpha hydrolase wp_567465 GI:54365463
AUO97_RS0007 beta hydrolase wp_567465 GI:65456475
AUO97_RS0020 gamma hydrolase wp_567465 GI:4536473
non-awk solution:
$ grep -i GI test.txt
AUO97_RS0005 alpha hydrolase wp_567465 GI:54365463
AUO97_RS0007 beta hydrolase wp_567465 GI:65456475
AUO97_RS0020 gamma hydrolase wp_567465 GI:4536473
A similar solution has been used/posted by OP already.
Log in to answer this question.
Your formatting makes any effort on our side impossible. Please put the example into code blocks to preserve new lines and other formatting.
Your question is unanswerable as written.
file is like this: