This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Running python script in Snakemake without input/output

Hi I have a very simple question. Is it possible to run a python script in a snakefile, without specifying an output/input? I have a python script that renames some of my generated output. However, in the script is already specified input files. Is there a way to just run the script? My thought we be:

rule pythonscript:
    shell:
        "script.py"

Is this possible?

snakemake

Thanks for your answer. The problem in giving input and output, is that I don't know the name of the output files. The renaming is based on some other files, which vary according to my input file. So I can't tell snakemake the name of the output.

maybe this can also help, don't know the details myself. https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#dynamic-files I understand these things can be difficult in snakemake.

EDIT: You could also build in some kind of checks in the python script and if there are no errors the script creates a "successful_complete" file or something.

EDIT2: Another option is to create a wildcard before the rule.

Can you provide some examples of these input/output files?

1 answer

Not knowing the exact input or output at code time is a relatively common thing in bioinformatics, and I think snakemake handles it rather well.

Let's assume taxonomy.txt is formatted as below.

[~/Data/scratch/tmp/biostar/checkpoint]$ cat taxonomy.txt 
bin unique  multi   tax
90-20-09-2018.001   25  15  Lactobacillus
90-20-09-2018.003   24  0   Streptococcus
90-20-09-2018.002   15  0   Lactobacillus_2

There are many ways to accomplish it, and below is an example usage of checkpoint (https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#data-dependent-conditional-execution ).

[~/Data/scratch/tmp/biostar/checkpoint]$ cat example.py
import os, csv

def aggregate(wildcards):
    """aggregate paths from `read_input_txt` and return as list""" 
    with open(checkpoints.read_input_txt.get(**wildcards).output[0], 'r') as fin:
        return [loc.rstrip() for loc in fin]

rule:
    input: aggregate

checkpoint read_input_txt:
    """implement logic to turn `taxonomy.txt` into a list of desired files to create"""
    input: 'taxonomy.txt'
    output: 'files_to_be_created.txt'
    run:
        with open(input[0], 'r') as fin, open(output[0], 'w') as fout:
            reader = csv.DictReader(fin, delimiter='\t')
            writer = csv.writer(fout)
            for data in reader:
                writer.writerow([os.path.join('binned', data['bin'].split('.')[0], data['tax'])])

rule create_file:
    """implement logic to create the actual file"""
    output: touch('{prefix}/{file}')

After snakemake -s example.py,

[~/Data/scratch/tmp/biostar/checkpoint]$ tree binned/
binned/
└── 90-20-09-2018
    ├── Lactobacillus
    ├── Lactobacillus_2
    └── Streptococcus

1 directory, 3 files

Hope you'll find this helpful and able to expand the example into a working solution.

Hi Eric After some tweaking, I think I have a sensible solution! Thanks for your input!

Log in to answer this question.