This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Nextflow runs Successfully but output files are empty. How to debug?

Hi Everyone!!

I am trying to alter the reference fasta files with the vcfs that I have. For that I wrote the following nextflow script. It runs successfully but the output fasta files are empty

nextflow.enable.dsl=2
params.raw = "results/06_gatkresults/vcfs/*.vcf.gz"
params.ref = "$baseDir/results/00_indexes/bwaidx/sequence.fasta"
params.outdir="results/07_assembly"
params.bed="$baseDir/results/05_lowcoverageBed"


process COV{
    publishDir "$params.outdir"
    input:
    tuple val(sid), path(vcf)
    output:
    file "*.fa"
        file "*.fasta"
    script:
    """ 
    echo ${sid}
    bcftools consensus -f ${params.ref} ${vcf} | sed 's/NC_045512.2/${sid}/' > ${sid}.fa 
    bedtools maskfasta -fi ${sid}.fa -bed ${params.bed}/${sid}.bed -fo ${sid}.fasta
    """
}

reads_ch = Channel.fromPath(params.raw).map { file -> tuple(file.simpleName, file) }

workflow {
  COV(reads_ch)
}

All the files are of zero bytes. How to find out what's wrong?

N E X T F L O W  ~  version 21.10.6
Launching `07_assembly.nf` [curious_swartz] - revision: 684fccf78d
executor >  local (12)
[8f/141386] process > COV (4) [100%] 12 of 12 ✔
nextflow bcftools
• 2,714 views • link
written by rohitsatyam102 rohitsatyam102

and what's the content of

cat work/8f/141386*/.command.sh
cat work/8f/141386*/.command.out
cat work/8f/141386*/.command.err

also, retry with

set -x
set -o pipefail
echo ${sid}
bcftools version
bcftools consensus -f "${params.ref}" "${vcf}" | sed 's/NC_045512.2/${sid}/' > "${sid}.fa"
test -s "${sid}.fa"
bedtools maskfasta -fi ${sid}.fa -bed "${params.bed}/${sid}.bed" -fo "${sid}.fasta"
test -s "${sid}.fasta"
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

.command.sh can be found in the process folder in the work directory. You will need to navigate to the right folder. In the initial post this would be work/8f/141386.... It's a hidden file so you will need something like ls -la to see it.

• 0 views • link
written by Christopher • Christopher

2 answers

  1. Shouldn't your output files actualy be a path if you want to do a glob like "*.fasta" ? Can't you define as ${sid}.fasta like in the script section ?
  2. Check your work directories and the logs (errors ?)
  3. from the work directories use the various hidden files prepended by a dot eg .command.sh or whatever. One of these can be run with the exact input you have linked in. You can test, modify and improve this command directly, applying it in this work directory, before you know what's going wrong and repair your full script
• 0 views • link
written by colindaven colindaven
  1. In the previous six scripts based on same template file worked just fine without any problem. But yes path I tried with output: path "*.fa" but the files are still empty.
  2. Here is the log file: https://gist.github.com/Rohit.
  3. I am not sure where to find .command.sh. Can you help?
• 0 views • link
written by rohitsatyam102 rohitsatyam102

Tried your suggestion 1 and still the files are empty

nextflow.enable.dsl=2
params.raw = "results/06_gatkresults/vcfs/*.vcf.gz"
params.ref = "$baseDir/results/00_indexes/bwaidx/sequence.fasta"
params.outdir="results/07_assembly"
params.bed="$baseDir/results/05_lowcoverageBed"


process COV{
        publishDir "$params.outdir"
        input:
        tuple val(sid), path(vcf)
        output:
        path "*.fa"
        script:
        file1= sid + '.fa'
        file2= sid + '.fasta'
        """
        echo ${sid}
        bcftools consensus -f ${params.ref} ${vcf} | sed 's/NC_045512.2/${sid}/' > $file1
        bedtools maskfasta -fi ${sid}.fa -bed ${params.bed}/${sid}.bed -fo $file2
        """
}

reads_ch = Channel.fromPath(params.raw).map { file -> tuple(file.simpleName, file) }

workflow {
  COV(reads_ch)
}
• 0 views • link
written by rohitsatyam102 rohitsatyam102

Okay!! So I tried just running the bcftools and I realized that it wasn't able to locate the index file .csi unlike other tools like bwa and gatk. So I have to use ${vcf.toRealPath()} instead of just ${vcf} and that solved the issue. Thanks anyway for helping me colindaven

• 0 views • link
written by rohitsatyam102 rohitsatyam102

Log in to answer this question.

More posts like this