find matching rows based on snp ids
I have 2 files, one is about 7 MB and other is 1 GB. The first file seems:
rs58108140 A/G chr1 10582
rs10218492 A/G chr1 10827
rs10218493 A/G chr1 10903
and the second one is:
rs58108140
rs10218493
rs11240777
I need to search second files snps in the first file and return the matching rows. It should be like that:
rs58108140 A/G chr1 10582
rs10218493 A/G chr1 10903
I tried to put all them in a database and take their inner join based on snpids but it is still working. Do you know a fast solution?
2 answers
Yup, like this, but make sure LC_ALL=C is set when you're using large files, or you'll be there all day! Either add export LC_ALL=C to your ~/.bashrc and re-source the file (source ~/.bashrc) or put in manually in the terminal.
LC_ALL=C grep -wFf file1 file2 > file3
Log in to answer this question.
More posts like this
-
Create a new bed file with all pairwise combinations between two other bed files, based on bp dista…
written by J •
-
Distance index computation during `vg --autoindex` takes too long with one GFA file but not with th…
written by Harsh •
-
Compare genotype genome sequences at basepair level
written by kashiff007
-
Merging two files based on matched column and print only the matched lines
written by munaj86 •
-
How to compare data in two files and print matching data of file1 and file2
written by Kumar
-
bwa aln writing speed became very slow after being very fast. What is the pattern of output size?
written by Jerry •
-
Adding the csv file names as a separate column to the csv file
written by paramount.amin •
-
Merging plink files
written by jramo033 •
-
Combine several VCF files
written by Max Ivon
-
Go Terms From Ensembl
written by Mdeng
I strongly recommend alessia's solution! But just for the record: have you build an index over the columns that you join on (i.e., the columns containing the IDs)? Otherwise, it is not surprising that the query takes forever ;-)