i am running fastqc for 10 paired end fastq files through snakemake, can you guys help me whats wrong with the code i wrote.

Hi guys, can anyone help me with the error i am getting when i am running fastqc through snakemake my code runs for a while and suddenly it gives an error. my code looks like this.

SAMPLES, = glob_wildcards("/scratch/mk9uc/RNA_Seq/{sample}_1.fastq.gz")
READS=["1", "2"]

rule fastqc_raw:
  input:  expand("/scratch/mk9uc/RNA_Seq/{sample}_{read}.fastq.gz", sample=SAMPLES, read=READS)
  output: expand("qc_output/{sample}_{read}_fastqc.html", sample=SAMPLES, read=READS)
  log:    expand("logs/fastqc/{sample}_{read}_raw.fastqc.err", sample=SAMPLES, read=READS)
  shell: """
      module load fastqc
      mkdir -p  log/fastqc
      mkdir -p  qc_output
      fastqc --outdir qc_output --thread 8 --nogroup {input} \
             >> /scratch/mk9uc/RNA_Seq/fastqc.log 2>>{log}
        """

error which i am getting looks like this.

Error in rule fastqc_raw: jobid: 0 output: qc_output/SRR7058334_1_fastqc.html, qc_output/SRR7058334_2_fastqc.html, qc_output/SRR7058336_1_fastqc.html, qc_output/SRR7058336_2_fastqc.html, , qc_output/SRR7058333_1_fastqc.html, qc_output/SRR7058333_2_fastqc.html log: logs/fastqc/SRR7058334_1_raw.fastqc.err, logs/fastqc/SRR7058334_2_raw.fastqc.err, logs/fastqc/SRR7058339_1_raw.fastqc.err, logs/fastqc/SRR7058339_2_raw.fastqc.err, logs/fastqc/SRR7058333_1_raw.fastqc.err, logs/fastqc/SRR7058333_2_raw.fastqc.err

RuleException: CalledProcessError in line 11 of /sfs/lustre/scratch/mk9uc/RNA_Seq/Snakefile: Command ' set -euo pipefail; module load fastqc mkdir -p log/fastqc mkdir -p qc_output fastqc --outdir qc_output --thread 8 --nogroup /scratch/mk9uc/RNA_Seq/SRR7058334_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058334_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058336_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058336_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058335_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058335_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058338_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058338_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058332_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058332_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058331_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058331_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7064063_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7064063_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058339_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058339_2.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058333_1.fastq.gz /scratch/mk9uc/RNA_Seq/SRR7058333_2.fastq.gz >> /scratch/mk9uc/RNA_Seq/fastqc.log 2>>logs/fastqc/SRR7058334_1_raw.fastqc.err logs/fastqc/SRR7058334_2_raw.fastqc.err logs/fastqc/SRR7058336_1_raw.fastqc.err logs/fastqc/SRR7058336_2_raw.fastqc.err logs/fastqc/SRR7058335_1_raw.fastqc.err logs/fastqc/SRR7058335_2_raw.fastqc.err logs/fastqc/SRR7058338_1_raw.fastqc.err logs/fastqc/SRR7058338_2_raw.fastqc.err logs/fastqc/SRR7058332_1_raw.fastqc.err logs/fastqc/SRR7058332_2_raw.fastqc.err logs/fastqc/SRR7058331_1_raw.fastqc.err logs/fastqc/SRR7058331_2_raw.fastqc.err logs/fastqc/SRR7064063_1_raw.fastqc.err logs/fastqc/SRR7064063_2_raw.fastqc.err logs/fastqc/SRR7058339_1_raw.fastqc.err logs/fastqc/SRR7058339_2_raw.fastqc.err logs/fastqc/SRR7058333_1_raw.fastqc.err logs/fastqc/SRR7058333_2_raw.fastqc.err ' returned non-zero exit status 137. File "/sfs/lustre/scratch/mk9uc/RNA_Seq/Snakefile", line 11, in __rule_fastqc_raw File "/apps/software/standard/core/anaconda/5.2.0-py3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run Removing output files of failed job fastqc_raw since they might be corrupted:

rna-seq snakemake fastqc

• 598 views • link

updated 7.2 years ago by Jeremy Leipzig

• written 7.2 years ago by arshil •

1 answer

You are kind of abusing expand. Why not just set up a wildcard rule and ask for outputs as a target?

rule fastqc_raw:
input:  "/scratch/mk9uc/RNA_Seq/{sample}_{read}.fastq.gz"
output: "qc_output/{sample}_{read}_fastqc.html"
...

rule mytarget:
input: expand("qc_output/{sample}_{read}_fastqc.html", sample=SAMPLES, read=READS)

• 0 views • link

written 7.2 years ago by Jeremy Leipzig

Thanks Jeremy. I did the way you suggested it. it runs and get completed within 10 sec. but there aint any output coming. my new code as per your suggestion looks like this.

SAMPLES, = glob_wildcards("/scratch/mk9uc/RNA_Seq/{sample}_1.fastq.gz")
READS=["1", "2"]
rule fastqc_raw:
  input:  "/scratch/mk9uc/RNA_Seq/{sample}_{read}.fastq.gz"
  output: "qc_output/{sample}_{read}_fastqc.html"
  shell: """
      module load fastqc
       mkdir -p  qc_output
       fastqc --outdir qc_output --thread 8 --nogroup {input} \
             >> /scratch/mk9uc/RNA_Seq/fastqc.log 2>>{log}
        """

rule mytarget:
  input: expand("qc_output/{sample}_{read}_fastqc.html", sample=SAMPLES, read=READS)

• 0 views • link

updated 7.2 years ago by Ram

• written 7.2 years ago by arshil •

two thoughts: 1. make sure SAMPLES is actually a variable with something in it (make a rule to print its contents) 2. make sure you are running Snakemake mytarget

• 0 views • link

written 7.2 years ago by Jeremy Leipzig