The errors reported when running snakemake file with log function

Question

[wangdp123]

Hi there,

I am using snakemake to compile a pipeline and encounter an issue as follows:

The error reported is:

RuleException in line 284 of pipeline_1.snakemake: Wildcards in input, params, log or benchmark file of rule fastx_collapser cannot be determined from output files: 'PIPELINE_NAME'

The scripts in snakemake file is:

PIPELINE_NAME = "pipeline_1"

FOLDER = "/data/analysis/"

    rule test1:

        input:
            "{FOLDER}/raw/{smp}.txt"
        output:
            "{FOLDER}/processed/{smp}.result.txt"
        log:
            out="{FOLDER}/{PIPELINE_NAME}/logs/{smp}.log.stdout",
            err="{FOLDER}/{PIPELINE_NAME}/logs/{smp}.log.stderr"
        shell: """
               program -i {input} -o {output} 2> {log.err} 1> {log.out}
               """

BTW, when the log function is not used, there is no problem. Therefore, I am just thinking about something wrong with the log file function.

Would you like to help me out about this problem?

Thank you very much,

Regards,

Tom

Answer 1

[Devon Ryan]

The contents of wildcards is derived from what you have in the input: and output: directives. Since you've defined only FOLDER and smp there, only those wildcards exist. What you want to do is the following:

log:
    out ="{{FOLDER}}/{}/logs/{{smp}}.log.stdout".format(PIPELINE_NAME),
    err ="{{FOLDER}}/{}/logs/{{smp}}.log.stderr".format(PIPELINE_NAME)

The {{...}} will keep things from getting filled in by format() right away.

[Medhat]

Isn't it enough the double quotes?! as in documentation:

rule all:
  input: ["{dataset}/file.A.txt".format(dataset=dataset) for dataset in DATASETS]

[Devon Ryan]

There are two levels of .format() going on, only one of which is explicit and available when the Snakefile is first run. The second level is implicit and done for each sample that's processed through each rule.

[GenoMax]

Going to high-jack this thread for some time. @Medhat you are now a moderator see: C: Inviting Biostars moderators to join Biostars slack channel