many thanks Hervé !
• 0 views
•
link
written
by
Pierre Lindenbaum
Minimal CWL (Common Workflow Language) workflow from Makefile
Common Workflow Language (CWL) https://github.com/common-workflow-language/common-workflow-language / http://common-workflow-language.github.io/draft-3/ has been trending on my twitter timeline during the last weeks.
However the spec is quite large and I find it hard to get some simple examples.
Furthermore, I have the feeling that all engines require a lot of dependencies or docker. I'd like to test my makefile-based workflows using CWL, how should I write and test the following simple Makefile using CWL:
SHELL=/bin/bash
.PHONY: all
all : database.dna
database.dna : seq1.dna seq2.dna seq3.dna
cat seq1.dna seq2.dna seq3.dna > database.dna
seq3.dna : seq3.rna
tr "U" "T" < seq3.rna > seq3.dna
seq3.rna :
echo "AUGCGAUCGAUCG" > seq3.rna
seq2.dna : seq2.rna
tr "U" "T" < seq2.rna > seq2.dna
seq2.rna :
echo "AUGAAGACUGCGAUCGAUCG" > seq2.rna
seq1.dna : seq1.rna
tr "U" "T" < seq1.rna > seq1.dna
seq1.rna :
echo "AUGAAGACUGACUCGUCG" > seq1.rna
EDIT: feel free to add the file for your favorite workflow-engine as an answer.
@yokofakun oh, still no reply.. I can provide it for @nextflowio DSL, but I guess you are not interested to that ;) @smllmp
— paoloditommaso (@PaoloDiTommaso) July 28, 2015
@yokofakun Could have 2-3 levels of "advanceness" though :) Have used this from a tutorial before: https://t.co/GivahZipKW @PaoloDiTommaso
— Samuel Lampa (@smllmp) July 28, 2015
• 7,520 views
•
link
updated
by
Ram
•
written
by
Pierre Lindenbaum
•
written
by
Pierre Lindenbaum
5 answers
To describe this workflow using CWL you need to:
- describe the 3 different tools (tr, echo, cat)
- describe the workflow structure
- describe the input data for the workflow job, i.e. the 3 input strings (AUGAAGACUGACUCGAUCGAUCG. etc.)
As an example to see the implementation of "tr", you can see this gist: . The github repository for common-workflow-language also has a number of tool and workflow descriptions that might help (https://github.com/common-workflow-language/common-workflow-language/tree/master/conformance/draft-2).
The reference implementation should be fairly easy to set up (pip install cwl-runner) but it is minimal (no cluster integration, etc.). Docker is not and should never become a requirement of CWL, some tools have been described with docker "requirements" but this is absolutely not mandatory. You can also ask your questions on all the channels mentionned here: https://github.com/common-workflow-language/common-workflow-language#community-and-contributing
Hope this helps,
Hervé
• 0 views
•
link
written
by
herve.menager •
Responded with an example on this github issue: https://github.com/common-workflow-language/workflows/issues/1
• 0 views
•
link
updated
by
Ram
•
written
by
peter.amstutz
•
written
by
peter.amstutz
I do plan to create a CWL format importer for flowr in the future. Though, here is a example, using two simple tab-delim files. Feedback, really welcome!
Install flowr following these steps.
You may edit this file, to run the same flow in various clusters, or local envir.
vi simple_make.def
Additionally, for easy visualization, you may use this command:
flowr plot_flow x=simple_make.def pdffile=simple_make.pdf
When done, submit:
flowr to_flow x=simple_make.tsv def=simple_make.def execute=TRUE
Output: https://github.com/sahilseth/flowr/files/4729/simple_make.pdf
• 0 views
•
link
updated
by
Ram
•
written
by
sahiilseth •
•
written
by
sahiilseth •
I'm copying Paolo Di Tommaso's gist for http://www.nextflow.io/ below
@yokofakun OK, I've uploaded it to the gist below. Feel free to comment if you have any doubt. https://t.co/n9rk8asJhI
— paoloditommaso (@PaoloDiTommaso) July 28, 2015
• 0 views
•
link
updated
by
Ram
•
written
by
Pierre Lindenbaum
•
written
by
Pierre Lindenbaum
correct me if I'm wrong but this isn't CWL-compliant
• 0 views
•
link
written
by
Jeremy Leipzig
"EDIT: feel free to add the file for your favorite workflow-engine as an answer."
That's right, that's nextflow.io.
• 0 views
•
link
written
by
peter.amstutz
is CWL a workflow engine itself or a standard for describing the interface between components?
• 0 views
•
link
written
by
Jeremy Leipzig
The second one. It's a specification for a declarative document format for describing portable workflows. The goal is to have lots of interoperable implementations.
• 0 views
•
link
updated
by
Ram
•
written
by
peter.amstutz
•
written
by
peter.amstutz
Will try to contribute. This is not an actual answer, but how I try to keep my projects tidy using Makefile (aka "reproducible research").
Dependency for every script is a dummy file (shed for sh or Red for R)
analysis=\
download.shed \
clean.shed \
statistics.Red \
figures.Red
###########################################
# Analysis Workflow
###########################################
all: $(analysis)
# Download data
download.shed: download.sh
# Clean data
clean.shed: clean.sh download.shed
# Summary statistcs
statistics.Red: statistics.R clean.shed
# Plot data
figures.Red: figures.R statistics.Red
###########################################
# Execute
###########################################
%.shed: %.sh
./$< 2> $(basename $<).out && touch $@
%.Red: %.R
R CMD BATCH --no-save --no-restore $< && touch $@
clean:
rm -f *out *.shed *.Red
thanks, I know how to write a makefile ( e.g. https://github.com/lindenb/ngsxml ). However, what I need is a solution for the alternative solutions, especially CWL.
• 0 views
•
link
written
by
Pierre Lindenbaum
Log in to answer this question.
Responded with an example on this github issue: https://github.com/common-workflow-language/workflows/issues/1