toil stats under-reporting memory usage?
The output from toil stats <jobstore> seems to be reporting values ~5000K (5Mb), but I believe it should be more like 5G
file:///home/johnsoni/pipeline_0.0.39/ACCESS-Pipeline/cwl_tools/bwa-mem/bwa-mem.cwl
Memory
min med ave max total
5659K 5950K 5854K 6128K 128807K
Is there something peculiar about this report? Here is how I'm specifying the resource requirements:
requirements:
- class: InlineJavascriptRequirement
- class: ResourceRequirement
ramMin: 30000
coresMin: 4
outdirMax: 20000
I would expect it to be using at least 1G on reasonably-sized fastqs
1 answer
I haven't tracked this down fully, but it seems like this line:
tag_str += reportMemory(t, options, field=width, isBytes=True)
means that reportMemory will divide by 1024:
if isBytes:
k /= 1024.
If the memory values comes from resource.getrusage then I believe the memory should already in KB:
me = resource.getrusage(resource.RUSAGE_SELF)
childs = resource.getrusage(resource.RUSAGE_CHILDREN)
See https://stackoverflow.com/questions/12050913/whats-the-unit-of-ru-maxrss-on-linux
Therefore dividing by 1024 on this line would result in MB, not KB:
https://github.com/DataBiosphere/toil/blob/master/src/toil/utils/toilStats.py#L211
Log in to answer this question.
More posts like this
For bwa mem, memory usage would depend mainly on genome index size, not fastq size. Is your reference genome very small?
It's actually the full sized hg19 reference, would you happen to be able to confirm the units for this command perchance?
I don't use CWL or Toil, so I really don't know what they report. But bwa mem with the human genome should use about 8Gb memory, so the numbers seem off.