This is a test version of Biostars. For the public version, visit https://www.biostars.org.
CWL: Does cwltool support to download both input file and its secondaryFiles remotely?

As mentioned in issue: https://github.com/common-workflow-language/cwltool/pull/507

cwltool allows http/https as input, that means it will automatically download and stage the remote input files for both tools and workflows.

One question here is: does cwltool also support the secondaryFiles which are also remote?

For example:

If define workflow.cwl file as:

Inputs:
     ref_file:
     type: File?
     secondaryFiles: ['.fai?']

Can I define the input in the job.yaml file as following:

ref_file:
    path: http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa
    class: File
    secondaryFiles:
    - class: File
       path: http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa.fai
cwl workflow
• 1,699 views • link
updated by Michael R. Crusoe Michael R. Crusoe • written by Linda • Linda

1 answer

Hello Linda, and thank you for your posting,

Yes, this should work, but use location instead of path when specifying using URLs

ref_file:
    location: http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa
    class: File
    secondaryFiles:
    - class: File
       location: http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa.fai
• 0 views • link
written by Michael R. Crusoe Michael R. Crusoe

Log in to answer this question.

More posts like this