This is a test version of Biostars. For the public version, visit https://www.biostars.org.
any idea why hisat2 is usurping all my RAM and threads

Hi, I am trying to run hisat2 with 400 samples in a loop. for some reason when i specify -p it simply takes over all the resources in my computer and exits with error message "(ERR): hisat2-align exited with value 137". It works perfectly ok when don't specify any -p. commands i used

#!/bin/bash
cat /mnt/samplesNewList.txt  | while read line
do
sudo hisat2 -p 2  -x /mnt/genomeindex-hisat/genome  -U /mnt/$line.clean.fq.gz  -S $line.sam  --summary-file mnt/summary.txt &
done

any one encountered same issue?

rna-seq
• 3,898 views • link
updated by ATpoint ATpoint • written by shripathiacademics • shripathiacademics

You should not use sudo for things like this, only for sysadmin stuff or installing new software globally.

• 1 views • link
written by WouterDeCoster WouterDeCoster

2 answers

Adding on h.mon, you also did not feed the stream from stdin to the loop. It should be

cat (...) | (...)
done < /dev/stdin

Still, I suggest using GNU parallel here. This allows to efficiently use your ressources while aligning several samples at once. Say you have 16 cores and want to run 4 samples in parallel:

cat /mnt/samplesNewList.txt | \
parallel -j 4 "hisat2 -p 4  -x $GENOME  -U /mnt/{}.clean.fq.gz  -S {}.sam  --summary-file mnt/{}.summary.txt"

Edit: To save disk space, consider to stream the output directly into samtools view to get a BAM file. Within the parallel command:

alignment | samtools view -bo {}.bam
• 1 views • link
written by ATpoint ATpoint

Because you are sending the hisat runs to the background (the & at the end, just before done), so the loops continues immediately to next iteration. This means you are aligning 400 samples simultaneously.

• 1 views • link
written by h.mon h.mon

To just run the alignments one after each other, remove the &.

To do some parallel work with more control, look at gnu parallel.

• 1 views • link
written by WouterDeCoster WouterDeCoster

Log in to answer this question.

More posts like this