Dataset A is annotated, Dataset B is diseased & phenotype info.
I have two datasets (A & B) which contain snp id's. I have to match snp ID's in both the datasets, where common SNP ID's found, I have to extract other col values (from dataset B) and merge with dataset A using shell scripting or R.
To resolve this issue, I wrote a basic python script. Size of abc is 18MB (1lakh rows>), size of xyz is 120MB (>40lakh rows) I am getting desired output but it is taking long computing hours. Can we modify this script or write using pandas?
F1 = open('xyz.csv','r').read()
F2 = open('abc.tsv','r').read()
z = 0
for line in F1.strip().split('\n'):
l = line.strip().split(',')
for row in F2.strip().split('\n'):
r = row.strip().split('\t')
if "CHROM" in line:
print(line+'\t'+row)
continue
if l[20] == r[0]:
print(line+'\t'+row)
z=1
break
else:
z=0
check here : https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html and here : https://jakevdp.github.io/PythonDataScienceHandbook/03.07-merge-and-join.html . In a nutshell : use pd.merge()
Yes, it is only showing merging cols across datasets. I have to match all rows of one col to another col dataset.
Log in to answer this question.
Please give some example of your datasets and the expected result. Also, is this a school assignment ?
Thanks all for your response. I am new to Bioinformatics. enter image description here
In Dataset A, I have the snpIDs and I have to match them with datasetB. Where-ever it matched, it will fetch other cols from DatasetA and join in DatasetB corresponding to that matched snpID.
Pls check image for ref. https://ibb.co/3vXBv75
in R check dplyr's full_join (or left_join) using SNP ID as key . Then use select() to extract the columns of interest. Basic R stuff.
Thanks all for your response. I am new to Bioinformatics.
In Dataset A, I have the snpIDs and I have to match them with datasetB. Where-ever it matched, it will fetch other cols from DatasetA and join in DatasetB corresponding to that matched snpID.
Pls check image for ref. https://ibb.co/3vXBv75
https://ibb.co/3vXBv75
Ref image
as I said in my previous comment : start with R dplyr : https://rpubs.com/williamsurles/293454
Am very thankful for your suggestion. I am learning R too. Tomorrow I have to submit my thesis and I need some string calculations so asked or requested for the algo.
This is a basic merge operation in R, see this post at SO: