Nextflow: how to filter a channel by a list? (DLS2)

Hello !

I'm trying to do something in Nextflow I thought would be simple but I can find how to do it: I need to filter the output of a channel by the output of another channel. An item in the second channel will be output only if the first channel contains this item.

I tried to convert the output of my first channel to a List and then to use the filter operator of Nextflow, but no item is emitted. Here is a toy example (I'm using Nextflow DLS2):

#!/usr/bin/env nextflow
nextflow.enable.dsl=2

workflow {

  Channel.of(["a","c","e","g"]) \
    | flatten \
    | toList \
    | set { to_keep }

  Channel.of([['a', "file_a.bam"],
              ['b', "file_b.bam"],
              ['c', "file_c.bam"],
              ['d', "file_d.bam"],
              ['e', "file_e.bam"],
              ['f', "file_f.bam"],
              ['g', "file_g.bam"],
              ['h', "file_h.bam"]]) \
    | filter { it -> to_keep.find { e -> it[0].toString().equalsIgnoreCase(e.toString()) } } \
    | view

}

dsl2 nextflow

• 2,428 views • link

updated 1.6 years ago by nyoungb2 •

• written 3.1 years ago by lacb •

1 answer

I managed to pull it off using join:

#!/usr/bin/env nextflow
nextflow.enable.dsl=2
workflow {

Channel.of(["a"],["c"],["e"],["g"]) \
 | set { to_keep }

Channel.of(['a', "file_a.bam"],
          ['b', "file_b.bam"],
          ['c', "file_c.bam"],
          ['d', "file_d.bam"],
          ['e', "file_e.bam"],
          ['f', "file_f.bam"],
          ['g', "file_g.bam"],
          ['h', "file_h.bam"]) \
| set{dict}
dict.join(to_keep).view() 

}

Which resulted with

[a, file_a.bam]
[c, file_c.bam]
[e, file_e.bam]
[g, file_g.bam]

You might want to change the two channels to lowercase if that's an issue.

• 0 views • link

written 3.1 years ago by Asaf

This does NOT work if there are duplicate values in the channel that you want to filter:

#!/usr/bin/env nextflow
nextflow.enable.dsl=2
workflow {

Channel.of(["b"]) \
 | set { to_keep }

Channel.of(
          ['a', 1],
          ['b', 2],
          ['b', 3],
          ['c', 4],
) \
| set{dict}
dict.join(to_keep).view() 

}

results in:

['b', 2]

instead of:

['b', 2], ['b', 3]

• 0 views • link

written 1.6 years ago by nyoungb2 •