Dear All,
I have small Shell script to write the lines in a file to another file
writer.sh
#!/usr/bin/env bash
infile=$1
outfile=$2
while read line
do
echo $line >> "$outfile"
done < "$infile"
I converted it for CWL as following, writer.cwl
cwlVersion: v1.0
class: CommandLineTool
baseCommand: [sh,writer.sh]
inputs:
infile:
type: File
inputBinding:
position: 1
outfile:
type: string
inputBinding:
position: 2
outputs:
out_message:
type: File
outputBinding:
glob: $(inputs.outfile)
And the YML writer.yml file looks as following,
infile:
class: File
path: /user/home/hello.txt
outfile: monday.txt
class: File
path: /user/home/tuesday.txt
When I run as following, cwltool writer.cwl writer.yml. it is throwing permanent fail error as,
[job writer.cwl] /tmp/tmpmht4svls$ sh \
search.sh \
/tmp/tmpv3evb_ic/stg1a74b4ca-15ba-4d28-9598-62b7604e8185/hello.txt \
monday.txt
sh: writer.sh: No such file or directory
[job writer.cwl] Job error:
Error collecting output for parameter 'out_message':
writer.cw.cwl:18:2: Traceback (most recent call last):
writer.cw.cwl:18:2:
writer.cw:18:2: File "/cluster/home/aalva/.local/venvs/cwltool/lib/python3.6/site-packages/cwltool/command_line_tool.py", line 724, in collect_output
writer.cwl: 18:2: raise WorkflowException("Did not find output file with glob pattern: '{}'".format(globpatterns))
writer.cwl:18:2:
writer.cwl:18:2: cwltool.errors.WorkflowException: Did not find output file with glob pattern: '['monday.txt']'
writer.cwl:18:2:
writer.cwl:21:4: cwltool.errors.WorkflowException: Did not find output file with glob pattern: '['monday.txt']'
[job writer.cwl] completed permanentFail
[job writer.cwll] {}
[job writer.cwll] Removing input staging directory /tmp/tmpv3evb_ic
[job writer.cwl] Removing temporary directory /tmp/tmp8v9yxvvo
{}
Final process status is permanentFail
I am not understanding what is wrong here.Any suggestions would be appreciated
Hi fellow pipeline enthusiast,
Is the script named "writer.sh" or "search.sh"?
In the error you posted, the shell seems to throw the error search.sh: No such file or directory. I guess it can't find the script, does nothing except throw the error message and CWL complains because no output file is being generated.
Is the script stored in a location that is referenced in your $PATH?
A dirty but quick fix would be to just pass your script as an argument in inputs.
Regards, Tom
Hey, Sorry, it is an error , It should be writer.sh. Is the script stored in a location that is referenced in your $PATH? Yes, it is store the place where it is being invoked. A dirty but quick fix would be to just pass your script as an argument in inputs. >> I did not quite understand what you mean?
You mean without outputs ? or by passing something like
arguments:
valueFrom: $(runtime.outdir)/$(inputs.outfile) ?
The place were you invoke is most likely not referenced in the $PATH environment variable. This means if you call sh writer.sh then writer.sh has to be in the directory you are calling sh from.
However, CWL creates a temporary directory (in your example it was /tmp/tmpmht4svls ) where it copies all your input data (or symlinks to the data) and then tries to run what you have specified in baseCommand. In this temporary directory, writer.sh is not present, so sh can't find it. If you pass the script as one of your input files, then CWL will know to copy it into the temporary directory during runtime and sh will find it.
.cwl-description:
cwlVersion: v1.0
class: CommandLineTool
baseCommand: sh
inputs:
script:
type: File
inputBinding:
position: 0
infile:
type: File
inputBinding:
position: 1
outfile:
type: string
inputBinding:
position: 2
outputs:
out_message:
type: File
outputBinding:
glob: $(inputs.outfile)
.yml-file:
script:
class: File
path: /WHEREVER/THE/SCRIPT/LIVES/writer.sh
infile:
class: File
path: /user/home/hello.txt
outfile: monday.txt
class: File
path: /user/home/tuesday.txt
Log in to answer this question.
