This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Missing CWL files in a Directory when passed as input

I wasn't sure if this was a question or a bug, so decided to first post a question about it. I can file an issue if it's confirmed to be a bug.

It looks like files are not mounted properly in directories when they're passed from one tool to another. Note that the files are tracked properly by CWL; they're just not mapped to the directory.

An example workflow is shown below. The output of ls is just an empty directory. The expected output is a full list of files and their subdirectories. I ran it using cwl-runner v1.0 and cwltool v1.0.20190831161204. Is there something that I'm missing when operating on directories in CWL?

class: Workflow
cwlVersion: v1.0
requirements:
  InlineJavascriptRequirement: {}
  SubworkflowFeatureRequirement: {}

inputs: {}
steps:
  gen_dir:
    run:
      class: CommandLineTool
      baseCommand: [bash, -c, 'mkdir -p test/subdir; echo hello > test/hello.txt; echo world > test/subdir/world.txt']
      inputs: {}
      outputs:
        dir:
          type: Directory
          outputBinding:
            glob: 'test'
    in: {}
    out: [dir]

  ls:
    run:
      class: CommandLineTool
      baseCommand: [ls, -laR]
      stdout: lsout.txt
      inputs:
        dir:
          type: Directory
          inputBinding:
            position: 1
      outputs:
        ls_out:
          type: stdout
    in:
      dir: gen_dir/dir
    out: [ls_out]

outputs:
  ls_out:
    type: File
    outputSource: ls/ls_out

The workaround is to explicitly loop through the listing inside the input directory and map them using InitialWorkDirRequirement.

cwl

1 answer

I'm not able to test this right now, but my guess would be: Everything works as intended CWL-wise, meaning a symlink to the directory is created in the temporary directory where step two is carried out. Calling ls with the -lflag causes it to not follow symlinks, so nothing is listed.

Best regards, Tom

Thank you so much, Tom! You are correct, the symlinks do indeed exist and show up with ls -LaR.

Log in to answer this question.