This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Samtools Sort Set Max Memory , Value In Bytes, Kb, Mb Or Gb ?

Does the samtools sort maximum memory parameter (m- ) require / accept values given in bytes, kB, MB of GB?

I would like to set the maximum memory parameter to 32 GB, so all the sorting of a bam chunk is done in memory. The default value is 500000000 something.

samtools

2 answers

If you use the most recent samtools version from the github repository you'll be able to set unit of memory and use threads, makes a massive difference!

https://github.com/samtools/samtools

$ samtools sort
Usage:   samtools sort [options] <in.bam> <out.prefix>
Options: -n        sort by read name
     -f        use <out.prefix> as full file name instead of prefix
     -o        final output to stdout
     -l INT    compression level, from 0 to 9 [-1]
     -@ INT    number of sorting and compression threads [1]
     -m INT    max memory per thread; suffix K/M/G recognized [768M]

ah you're right! The version I tested at home was the 'old' 1.18

Nice! I'll certainly try this. What effect does the compression level have? Sam format is not compressed? Does is mean compression to keep the memory use down?

the records stored in memory are uncompressed.

What is the option -l for then?

What is the advantage / disadvantage of using it? If the data is not compressed in memory and also the output data is not compressed?

non compressed in memory: the sorting algorithm goes faster because it doesn't need to compress/uncompress each item. output data compressed=less storage needed. output data uncompressed=faster if a downstream program reads the data in a pipeline.

Does the samtools sort maximum memory parameter (m- ) require / accept values given in bytes, kB, MB of GB?

sort answer: no

$grep  "max_mem" bam_sort.c 

size_t max_mem = 500000000;
    case 'm': max_mem = atol(optarg); break;

but you can always use a bash inline multiplication:

samtools sort -m $((10*1024)) ....

Ehm. So a number of bytes is expected? Or is it a number of records?

it's a number of bytes:

  max_mem  approxiate maximum memory (very inaccurate)
  (...)
  buf = (bam1_t**)calloc(max_mem / BAM_CORE_SIZE, sizeof(bam1_t*));

Log in to answer this question.