How to parallel task in bash elegantly?
Are there any elegant way to parallel processing in bash? Something like R packages "parallel" or Python module "multiprocessing"! Thanks!
• 1,358 views
•
link
updated
by
WouterDeCoster
•
written
by
wang.yiguan •
•
written
by
wang.yiguan •
1 answer
GNU parallel is a very flexible tool for this, e.g.
ls *.fastq | parallel -j 8 'bwa mem ref.fa {} > {.}.sam'
Use 8 jobs, do alignment for each file in *.fastq, output is input without extension (+ .sam).
• 1 views
•
link
written
by
WouterDeCoster
Log in to answer this question.
More posts like this
-
Conversion of GFF3 file to ptt format
written by qstefano •
-
Pipelines in bash vs other languages
written by jaynishshah •
-
Running BLAST tool using python's multiprocessing package
written by m_asi •
-
Gather sorting by parameter in the metadata of input file
written by grange •
-
RefSeq ID to TaxID?
written by bird77 •
-
What is a rational way to filter VCF and other genome files using BED file with selected genes?
written by matveyspr •
-
Fast datastructure to get counts of overlapping reads per nucleotide (!) in Python?
written by endrebak
-
Identify Population Specific SNPs from List of SNPs
written by gaiusjaugustus
-
python module or package to work with agp files ?
written by sid5427 •
-
Any Cog Enrichment Tools For Non-Model Organisms?
written by biotech
Hello wang.yiguan!
Questions similar to yours can already be found at:
We have closed your question to allow us to keep similar content in the same thread.
If you disagree with this please tell us why in a reply below. We'll be happy to talk about it.
Cheers!
PS: many duplicate on biostars.search for gnu parallel, Makefile , snakemake, cwl, xargs ...