This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Using the Directory type in Common Workflow Language

In trying to solve a problem posed by Stuti Agrawal, I've been trying to make a working example using the Directory type of CWL v1.0 as an input and not getting very far. I tried writing a tool that pics up the files from a Directory and adds them to a zip file. Note, this doesn't add the directory, but rather its contents. So if you have:

/home/foo/one.txt
/home/foo/two.txt

it should add one.txt and two.txt to the command line, i.e:

zip data.zip one.txt two.txt

I'm not sure if this is the intended use case for Directory? Alternatively, I'd like to able to add /home/foo to the command line, yielding:

zip -r data.zip /home/foo

In any event here is a non-working experiment:

#!/usr/bin/env cwl-runner

cwlVersion: v1.0

requirements:
    - class: InlineJavascriptRequirement

class: CommandLineTool

baseCommand: ["zip"]

inputs:
  testdir:
    type: Directory
    doc: |
        string(s): list files in a directory.
    inputBinding:
        position: 1
outputs:
  zipped_file:
    type: File
    outputBinding:
      glob: data.zip

arguments:
  - valueFrom: data.zip

What should be changed to get either my first or my second example command line?

Thanks

cwl
• 7,436 views • link
updated by Michael R. Crusoe Michael R. Crusoe • written by Peter vH Peter vH

By the way I used this input to the above CWL:

testdir:
  class: Directory
  location: file:///home/pvh/Documents/code/cwltool/foo
• 1 views • link
written by Peter vH Peter vH

Hello Peter vH!

We believe that this post does not fit the main topic of this site.

Not a bio-informatics question, have a look at stackoverflow.com

For this reason we have closed your question. This allows us to keep the site focused on the topics that the community can help with.

If you disagree please tell us why in a reply below, we'll be happy to talk about it.

Cheers!

• 1 views • link
written by WouterDeCoster WouterDeCoster

I don't know, the common workflow language is sort of an up-and-coming important aspect of reproducible bioinformatics.

• 1 views • link
written by Devon Ryan Devon Ryan

Well in that case, forgive my ignorance :) (reopened)

• 1 views • link
written by WouterDeCoster WouterDeCoster

Some of the CWL folks have directed "support" questions to Biostars recently on the understanding that when they get answered here the answers are easy for people to find in the future. I'll alert the community to this question though.

• 1 views • link
written by Peter vH Peter vH

Ah, didn't know that but using Biostars makes sense. I just haven't seen many CWL-related queries here yet.

• 1 views • link
written by Chris Fields Chris Fields

Hi Peter,

I think this is a bug. It's supposed to add the directory path to the command line, but that doesn't seem to be happening here. I'll look into it.

• 1 views • link
written by peter.amstutz peter.amstutz

1 answer

With thanks to Peter's patch to cwltool, here is a working variation:

#!/usr/bin/env cwl-runner

class: CommandLineTool
cwlVersion: v1.0

requirements:
 - class: InitialWorkDirRequirement
   listing: $(inputs.directory_to_zip.listing)     

inputs:
  directory_to_zip:
    type: Directory

baseCommand: [zip, "--recurse-paths", "-", "."]

outputs:
  zipped_file:
    type: stdout
    format: application/zip

Sample usage with command line input:

$ ls t
bam  bam.bai  three
$ cwltool zipper.cwl --directory t
/home/michael/cwltool/env/bin/cwltool 1.0.20160820220956
[job zipper.cwl] /tmp/tmpppA9iT$ zip \
    --recurse-paths \
    - \
    . > /tmp/tmpppA9iT/8cccc91d-7bf6-4008-a727-952ad9752114
  adding: bam (stored 0%)
  adding: three (stored 0%)
  adding: bam.bai (stored 0%)
  adding: 8cccc91d-7bf6-4008-a727-952ad9752114 (deflated 53%)
Final process status is success
{
    "zipped_file": {
        "format": "file:///home/michael/cwltool/application/zip", 
        "checksum": "sha1$4e7aea5191e78a98a66b23cb69652df3a607fbde", 
        "basename": "8cccc91d-7bf6-4008-a727-952ad9752114", 
        "location": "file:///home/michael/cwltool/8cccc91d-7bf6-4008-a727-952ad9752114", 
        "path": "/home/michael/cwltool/8cccc91d-7bf6-4008-a727-952ad9752114", 
        "class": "File", 
        "size": 746
    }
}

Finally sample usage with an input document

$ cat zipper.in.yml 
directory_to_zip:
  class: Directory
  location: t
$ cwltool zipper.cwl zipper.in.yml 
/home/michael/cwltool/env/bin/cwltool 1.0.20160820220956
[job zipper.cwl] /tmp/tmpQqxPjt$ zip \
    --recurse-paths \
    - \
    . > /tmp/tmpQqxPjt/2d0ca78b-6248-46aa-8b1c-0708fd0008a1
  adding: bam (stored 0%)
  adding: 2d0ca78b-6248-46aa-8b1c-0708fd0008a1 (deflated 11%)
  adding: three (stored 0%)
  adding: bam.bai (stored 0%)
Final process status is success
{
    "zipped_file": {
        "format": "file:///home/michael/cwltool/application/zip", 
        "checksum": "sha1$7c896e9247972af38d3836e621cd9aa66c1b3499", 
        "basename": "2d0ca78b-6248-46aa-8b1c-0708fd0008a1", 
        "location": "file:///home/michael/cwltool/2d0ca78b-6248-46aa-8b1c-0708fd0008a1", 
        "path": "/home/michael/cwltool/2d0ca78b-6248-46aa-8b1c-0708fd0008a1", 
        "class": "File", 
        "size": 710
    }
}
• 1 views • link
written by Michael R. Crusoe Michael R. Crusoe

Log in to answer this question.

More posts like this