This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Alignment loop using STAR

I'm running scRNA-seq and am trying to create a loop for all samples to run alignment. However, the loop stops about 20 or so samples later. Am I missing something from code? The alignment works fine without the loop.

for i in *.fastq.gz
do
    STAR --genomeDir ./human_index \
              --runMode alignReads \
              --readFilesIn $i \
              --runThreadN 10 \
              --readFilesCommand zcat \
              --outSAMtype BAM SortedByCoordinate \
              --chimSegmentMin 20 \
              --quantMode GeneCounts \
              --outReadsUnmapped Fastx \
              --outFileNamePrefix ./star_out/$i
done

EDIT by @RamRS: I formatted the code to be more readable. Original code copy-pasted below:

for i in *.fastq.gz; do STAR --genomeDir ./human_index --runMode alignReads --readFilesIn $i --runThreadN 10 --readFilesCommand zcat --outSAMtype BAM SortedByCoordinate --chimSegmentMin 20 --quantMode GeneCounts --outReadsUnmapped Fastx --outFileNamePrefix ./star_out/$i; done
rna-seq rna-seq alignment star

there is a possibility that you're running out of storage space to store the results?

Did this not produce any errors then?

I had a slightly different code where the --runThreadN was set to 20 and it was aborting but now with it set to 10 its coming up with error message about not enough space for bam - however this time its continuing onto the next sample despite the error message

Do you really need --outReadsUnmapped Fastx? Since you are running out of disk space (could you copy the exact error message?), you could save space by not outputting unmapped reads..

Jan 10 19:59:01 ..... started sorting BAM

EXITING because of FATAL ERROR: number of bytes expected from the BAM bin does not agree with the actual size on disk: 4944559   4928845   15

    Jan 10 19:59:03 ...... FATAL ERROR, exiting

    EXITING because of FATAL ERROR: number of bytes expected from the BAM bin does not agree with the actual size on disk: 4653064   4637278   13

    Jan 10 19:59:03 ...... FATAL ERROR, exiting
    *** Error in `STAR': double free or corruption (!prev): 0x00000000026d89a0 ***
    ======= Backtrace: =========
    /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f08711c27e5]
    /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f08711cb37a]

    7ffdd9196000-7ffdd9199000 r--p 00000000 00:00 0                          [vvar]
    7ffdd9199000-7ffdd919b000 r-xp 00000000 00:00 0                          [vdso]
    ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
    *** Error in `STAR': corrupted double-linked list: 0x00000000026d8990 ***
    Aborted (core dumped)
   Jan 10 19:59:07 ..... started STAR run

The error comes after sorting out the bam files. I previously used --outReadsUnmapped Fastx? and thought to keep it incase I wanted to know the reads that were unmapped. That is a good idea to save space, thank you.

Look at the log files of the iteration at which the loop breaks. I suspect that you run out of memory when sorting the bam file.

1 answer

It would help to know what the error is. If the issue isn't storage space, it might be RAM used while sorting. You might try adding this

--limitBAMsortRAM 31000000000

To your STAR command line.

I added this code and it worked overnight, thank you! Just to confirm, is this limiting the size of the BAM files? or is it doing something else?

It is limiting amount of RAM used for sorting to 31GB.

Log in to answer this question.