This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Getting a bash pipeline to output non-zero exit code when a step fails

I have a pipeline which looks something like below, its written in bash

Trimmomatic --> Samtools --> Smalt --> Lastz --> custom perl programs.

If anyone of those processes fails they usually return a non-0 exit code, usually a 1. However, my pipeline program will continue to run, report the error and return an exit code of 0, indicating success.

How can I get the pipeline to quit/stop with a non-0 exit code when a part of the pipeline fails?

bash command-line sh
• 15,887 views • link
updated by bernatgel bernatgel • written by SaltedPork SaltedPork

4 answers

You're looking for set -eo pipefail. Add that after the first (#!/bin/bash) line and your script will fail if any part of your script fails.

See for more information: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/

• 0 views • link
written by Ram Ram

Much better solution than mine!

• 0 views • link
written by Joe Joe

Thanks, this is a really good solution. Do you have any advice for if I wanted to just have the pipeline fail at certain points in the pipeline. Perhaps put a command in a loop and check that echo $? returns non-zero or not?

• 0 views • link
written by SaltedPork SaltedPork

Yep. You'll need to check that. It will take a few trials though, as this is almost an implementation of try-catch blocks.

You could also place the sensitive parts in a separate script that has set -eo pipefail enabled and call it from your non-sensitive script.

• 0 views • link
written by Ram Ram

If you follow Ram's link, you can combine it with the logic of mine below but invert it. i,e. if you have pipefail set, you can just invert the non-zero exit codes by doing something like:

mycommand || true

This means even if the command fails, the parent shell process will interpret is as having worked correctly (0), so you can use this to selectively override certain sections.

• 0 views • link
written by Joe Joe

Exactly. This works if the minority of statements are non-sensitive where OP wants to ignore the exit code. If the majority are non-sensitive, we're better off isolating the sensitive to separate scripts.

• 0 views • link
written by Ram Ram

To make the whole pipeline fail when any of the parts fail set the pipefail option. In zsh that can be done via setopt pipefail (don't know the bash equivalent by heart)/

• 0 views • link
written by kloetzl kloetzl

bash equivalent would be set -o pipefail

• 0 views • link
written by Ram Ram

I like the explicit error management described here, with a try, yell and die functions. As a bonus, you can modify die to do some additional logging of errors before actually dying and returning. It might be useful in addition to pipefail

yell() { echo "$0: $*" >&2; }
die() { yell "$*"; exit 111; }
try() { "$@" || die "cannot $*"; }
• 0 views • link
written by bernatgel bernatgel

mycommand || exit 1

Should work I think. Basically saying, either mycommand runs successfully to completion or exit the script with a non-zero status.

• 0 views • link
written by Joe Joe

Log in to answer this question.

More posts like this