This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Extract Common Line Out Of Multiple Files

Need to extract common list of Organisms from different files based on the second tab separated column

Example :

Below is the Query: File A:

  • YP_001173296 Pseudomonas stutzeri A1501
  • ZP_10847128 Pseudomonas fragi A22
  • YP_006325640 Pseudomonas fluorescens A506
  • ZP_08518919 Aeromonas caviae Ae398
  • ZP_10474222 Rickettsiella grylli
  • EKB21198 Aeromonas veronii AMC34

File B:

  • P_02062827 Rickettsiella grylli
  • YP_004473601 Pseudomonas fulva 12-X
  • ZP_10438680 Pseudomonas extremaustralis 14-3 substr. 14-3b
  • YP_528004 Aeromonas caviae Ae398
  • ZP_11138829 Gallaecimonas xiamenensis 3-C-1
  • YP_52800475 Pseudomonas stutzeri A1501

File C:

  • P_02062827 Pseudomonas extremaustralis
  • YP_004473601 Pseudomonas fulva 12-X
  • ZP_10438680 Pseudomonas extremaustralis 14-3 substr. 14-3b
  • YP_528004 Aeromonas caviae Ae398
  • ZP_11138829 Rickettsiella grylli
  • YP_52800475 Pseudomonas stutzeri A1501

Expected Result file : Common organism

  • Aeromonas caviae Ae398
  • Pseudomonas stutzeri A1501
  • Rickettsiella grylli
awk comparison

3 answers

cut the 2nd column, sort and get the lines having a count=3.

   cut -d '  ' -f 2 file*.tsv | sort | uniq -c |  grep -E '^      3 ' | cut -c 9-

actual i have 1000 files and i want to get common set of organisms using tab separated 2nd column I checked this command. Could you explain if I am wrong, the 3 stands for three files? In each file duplicates are present of the same organism

yes, "3" is for "3" files. Try : (for F in *.tsv; do cut -d ' ' -f2 $F | sort | uniq: done )| sort | uniq -c | grep -E '^ 1000 ' | cut -c 9-

I received syntax error near unexpected token `)'

I let you solve that syntax error as an exercise. :-)

hehe :) thanks dude

fileA = open("A.txt",'r')
fileB = open("B.txt",'r')
fileC = open("C.txt",'r')

listA1 = []
for line1 in fileA:
    listA = line1.split('\t')
    listA1.append(listA)


listB1 = []
for line1 in fileB:
    listB = line1.split('\t')
    listB1.append(listB)


listC1 = []
for line1 in fileC:
    listC = line1.split('\t')
    listC1.append(listC)

for key1 in listA1:
    for key2 in listB1:
        for key3 in listC1:
            if key1[1] == key2[1] and key2[1] == key3[1] and key3[1] == key1[1]:
                print key1[1]

This code also helps you. The three files should be in same format i.e the ID separated by organism name by 'tab' and no empty lines at the end of the file.

Thank you very much :)

Log in to answer this question.