This is a test version of Biostars. For the public version, visit https://www.biostars.org.
temp directory as output in snakemake

Hi, does anyone know of the effects of using both temp() and directory() in snakemake's output?

e.g.

rule:
  output: temp(directory("bowtie/{samp}/global_alignment"))
directory temp snakemake
• 1,720 views • link
updated by Jesse • Jesse • written by hamarillo • hamarillo

2 answers

if you use temp(), the folder will be deleted after snakemake finishes the whole run

• 0 views • link
written by Ming Tommy Tang Ming Tommy Tang

After all rules that take in the output as input are ran then it get deleted, not at the end of the whole run. So if you only use "bowtie/{samp}/global_alignment" as input in rule A, then after rule A finishes the files will be deleted You can read more here: https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#protected-and-temporary-files

• 0 views • link
written by Mauricio • Mauricio

Yep, this behavior is easy to confirm with a couple of test rules and something like shell: "sleep 60" in the last one, so you can see the temp directory outputs get removed before the whole workflow is done running. That distinction (versus removal at the very end) can matter if you have big workflows with multiple batches of temporary files.

I'm pretty sure that Snakemake implicitly sets rule priorities to try to get temp outputs cleared out sooner rather than later, too, but I can't find this in the docs. (Maybe implied by the help text for --scheduler where it "aims to reduce runtime and hdd usage by best possible use of resources"? I could have sworn I saw it described somewhere but can't find it now.)

• 0 views • link
written by Jesse • Jesse

Log in to answer this question.

More posts like this