This is a test version of Biostars. For the public version, visit https://www.biostars.org.
bam_sort_core problem when bam files be processed by samtools

I sorted my RNA-Seq bam files after mapping by star, and here is my key command line in a for cycle bash file: 

samtools sort $file -o ${filename}.sorted.bam

At last, the one error file shows like: 

[bam_sort_core] merging from 6 files and 1 in-memory blocks...
[bam_sort_core] merging from 4 files and 1 in-memory blocks...
[bam_sort_core] merging from 8 files and 1 in-memory blocks...
[bam_sort_core] merging from 9 files and 1 in-memory blocks...
[bam_sort_core] merging from 7 files and 1 in-memory blocks...
[bam_sort_core] merging from 4 files and 1 in-memory blocks...
[bam_sort_core] merging from 5 files and 1 in-memory blocks...
[bam_sort_core] merging from 3 files and 1 in-memory blocks...
[bam_sort_core] merging from 6 files and 1 in-memory blocks...
[bam_sort_core] merging from 13 files and 1 in-memory blocks...
[bam_sort_core] merging from 8 files and 1 in-memory blocks...
[bam_sort_core] merging from 2 files and 1 in-memory blocks...
...

Something wrong here? I searched but still don't know why, so I'm not sure these sorted files be used for next step?
What's more, I have M(like 63) files, and there M-N(like 6, yes, few) lines, it means not all file will meet this problem? It makes me more confused.

Any ideas will be appreciated!!!

rna-seq samtools
• 6,485 views • link
updated by linouhao • linouhao • written by 1106518271 1106518271

If you have the memory, you can reduce the number of temporary files by increasing the default memory usage from 768Mb to, say, 2G using the -m option, e.g. samtools sort -m 2G -o out.bam in.bam. Be sure to never use something like -m 2 rather than -m 2G as this would set the memory limit to 2 bytes resulting in thousands of tmp files, eventually crashing the system.

• 0 views • link
written by ATpoint ATpoint

Got it, many thanks!

• 0 views • link
written by 1106518271 1106518271

so is it really an error, and anyone knows to set -@ and -m, which is more useful when dealing with hundreds of bams at the same time in cluster, thanks a lot

• 1 views • link
written by linouhao • linouhao

Please use Add Comment for comments. As Istvan explained, these are just status messages, neither errors nor warnings. Set -@ and -m as you like, but these are options that still deal with one file at a time. If you want things parallelized, have a look at GNU parallel, like:

find ./ -maxdepth 1 -name "*.bam" | parallel -j 8 "samtools sort -@ 2 -m 2G -o {.}_sorted.bam {}

This command will sort all BAM files in your current directories, 8 at a time with 2 cores and 2GB of memory per core each.

• 1 views • link
written by ATpoint ATpoint

By the way, will this "error" here lead to give wrong result(${filename}.sorted.bam)?

• 1 views • link
written by 1106518271 1106518271

1 answer

These are not error messages, just debugging notes.

Large files cannot be sorted in memory thus get saved into temporary files. Once the sort completes the temporary files are removed.

There is nothing to be concerned about

• 0 views • link
written by Istvan Albert Istvan Albert

Log in to answer this question.

More posts like this