This is a test version of Biostars. For the public version, visit https://www.biostars.org.
CWL Error Empty Scatter

Whenever I try do scatter over an array of files I get the error message

emptyscatter = [shortname(s) for s in scatter if len(inputobj[s]) == 0]
TypeError: object of type 'NoneType' has no len()

This happens only, if I use a JavaScript expression to concatenate the arrays, which I have to do in this case. If I use only one of the arrays without JS expression the scatter works fine.

Does anyone have an idea how to fix this? Please see my code below:

The Workflow File

cwlVersion: v1.0
class: Workflow

requirements:
  - class: ScatterFeatureRequirement
  - class: SubworkflowFeatureRequirement
  - class: StepInputExpressionRequirement
  - class: InlineJavascriptRequirement

inputs:

  arrayC:
    type:
    - type: array
      items: File
    secondaryFiles:
      - ".bai"

  arrayD:
    type:
    - type: array
      items: File
    secondaryFiles:
      - ".bai"

  arrayE: string[]
  arrayF: string[]

outputs: []

steps:
  stepname:
    run: <path to CWL>
    scatter: [A, B]
    scatterMethod: dotproduct
    in:
      A:
        valueFrom: $( return arrayC.concat(arrayD))
      B:
        valueFrom: $( return arrayE.concat(arrayF).map(function(e) {return e + ".raw.bamcov"}))

      arrayC: arrayC
      arrayD: arrayD
      arrayE: arrayE
      arrayF: arrayF

    out:
      - outputFile

The .yml File

arrayC: [
  {class: File, path: testfiles/file_1.bam},
  {class: File, path: testfiles/file_2.bam}
]

arrayD: [
  {class: File, path: testfiles/file_3.bam},
  {class: File, path: testfiles/file_4.bam}
]

arrayE: ["file_name_1", "file_name_2"]

arrayF: ["file_name_3", "file_name_4"]
cwl workflow pipeline scatter
• 1,733 views • link
written by HL • HL

Hello HL, thank you for your question.

$( return arrayC.concat(arrayD)) is not a valid expression. Did you mean $( return inputs.arrayC.concat(inputs.arrayD))?

Can you provide a testable example so we can help you?

• 1 views • link
written by Michael R. Crusoe Michael R. Crusoe

For A you can accomplish this using linkMerge: merge_flattened:

steps:
  stepname:
    run: <path to CWL>
    scatter: [A, B]
    scatterMethod: dotproduct
    in:
      A:
        source: arrayC, arrayD
        linkMerge: merge_flattened
• 1 views • link
written by Michael R. Crusoe Michael R. Crusoe

0 answers

No answers yet.

Log in to answer this question.

More posts like this