Thank you so much, Tom! You are correct, the symlinks do indeed exist and show up with ls -LaR.
I wasn't sure if this was a question or a bug, so decided to first post a question about it. I can file an issue if it's confirmed to be a bug.
It looks like files are not mounted properly in directories when they're passed from one tool to another. Note that the files are tracked properly by CWL; they're just not mapped to the directory.
An example workflow is shown below. The output of ls is just an empty directory. The expected output is a full list of files and their subdirectories. I ran it using cwl-runner v1.0 and cwltool v1.0.20190831161204. Is there something that I'm missing when operating on directories in CWL?
class: Workflow
cwlVersion: v1.0
requirements:
InlineJavascriptRequirement: {}
SubworkflowFeatureRequirement: {}
inputs: {}
steps:
gen_dir:
run:
class: CommandLineTool
baseCommand: [bash, -c, 'mkdir -p test/subdir; echo hello > test/hello.txt; echo world > test/subdir/world.txt']
inputs: {}
outputs:
dir:
type: Directory
outputBinding:
glob: 'test'
in: {}
out: [dir]
ls:
run:
class: CommandLineTool
baseCommand: [ls, -laR]
stdout: lsout.txt
inputs:
dir:
type: Directory
inputBinding:
position: 1
outputs:
ls_out:
type: stdout
in:
dir: gen_dir/dir
out: [ls_out]
outputs:
ls_out:
type: File
outputSource: ls/ls_out
The workaround is to explicitly loop through the listing inside the input directory and map them using InitialWorkDirRequirement.
1 answer
I'm not able to test this right now, but my guess would be: Everything works as intended CWL-wise, meaning a symlink to the directory is created in the temporary directory where step two is carried out. Calling ls with the -lflag causes it to not follow symlinks, so nothing is listed.
Best regards, Tom
Log in to answer this question.