Scripting a NGS analysis pipeline, in Shell or Python ?

Question

[vin.darb]

Hello,

For my internship, I write some bash scripts in order to carry out a pipeline that I could reuse for future RNA seq analyzes.

I thought that in case the pieline would really elaborate, I had to go on python

So I started to "translate" my scripts into python but I realized that I mainly called bash commands via python (Subprocess.run Subprocess.run Subprocess.run ...) , so I gave up

My question is, when does it make sense to switch from bash to python to program NGS analysis scripts?

If you have examples of scripts for example that you have used / written for the NGS I would be interested to compare with mine

thank you in advance for your help

[WouterDeCoster]

Take a look at workflow managers such as snakemake, allowing you to mix shell commands with python, and a lot of parts of your workflow that you no longer have to reinvent.

[colindaven]

I'm a big fan of the nf-core pipelines and use them productively for RNA-seq and bisulfite analysis.

You need to invest a bit configuring nextflow for your environment, but it's plain sailing after that. Certainly way better than another crappy bash script.

https://github.com/nf-core/rnaseq

For custom analyses I'd recommend SnakeMake or NextFlow, though we have written quite involved Python pipelines which are used almost every day in production for the many intricate projects we get on a daily basis in the genomics core.

https://github.com/MHH-RCUG/Wochenende

[vin.darb]

Ok thank's for your answers, I will have a look on SnakeMake

Answer 1

[Pierre Lindenbaum]

If you have examples of scripts for example that you have used / written for the NGS I would be interested to compare with mine

using a simple Makefile: see for example:

see also:

A: Parallel for bwa mem - problem with -R argument for ID and SM

https://gist.github.com/lindenb/832a8da14217238a019d

[cschu181]

Like the part where you put the words "simple" and "Makefile" into the same context XD