This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Simple Redirection, I/O Problem With Bedtools

Hi Guys, Just a quick question. Its more of a Bash question rather than Bioinformatics, with Bedtools in question.

I mostly pipe the bedtools I/O. Here's a general scenario :

sed 1d fileA.bed | intersectBed -a stdin -b peaks.bed | intersectBed -u -a stdin -b fileB.bed

Now, the problem is fileB is also having a head, which is reported as an error by intersectBed (makes sense, non-integer start).

How can I remove the first line or the head of the fileB on the fly in the pipe.

Thanks

bedtools bash pipeline linux intersect
• 5,726 views • link
updated by Ryan Dale Ryan Dale • written by Sukhi Singh Sukhi Singh

3 answers

As of 2.13.0 bedtools supports FIFOs, so you can do:

intersectBed -a <(sed 1d fileA.bed) -b <(sed 1d peaks.bed) | intersectBed -u -a stdin -b fileB.bed
• 1 views • link
written by Ryan Dale Ryan Dale

That's a great news and post!! Cheers

• 0 views • link
written by Sukhi Singh Sukhi Singh

If I am right, this is a bash trick, which can be used with any programs, irrelevant of bedtools and its versions. But as it is shell dependent, you cannot use it in a C-shell.

• 0 views • link
written by lh3 lh3

That's right, FIFOs are a shell technology, not a bedtools technology. What @Daler is describing is that, owing to a silly mistake, early versions of bedtools were incapable of using FIFOs as input.

• 0 views • link
written by Aaronquinlan Aaronquinlan

Very cool, I didn't know you could do this. I'm going to have to start using this method.

• 0 views • link
written by Fwip Fwip

Have you tried using named pipes? Your commands would basically be the same as those listed by Damian, except with an additional mkfifo command at the front.

mkfifo tmp_pipe ;
sed 1d fileB.bed > tmp_pipe & 
sed 1d fileA.bed | intersectBed -a stdin -b peaks.bed | intersectBed -u -a stdin -b tmp_pipe ; 
rm -f tmp_pipe ;

(I split this into separate lines for ease of reading).

Edit: lh3 is right, you need to write to the pipe in a background process, or your shell will stall while waiting for the program to close. The commands have been updated. Thanks for the correction!

• 0 views • link
written by Fwip Fwip

One small thing: you need to put the second line to the background. i.e. you need "&" at the end of the 2nd line.

• 0 views • link
written by lh3 lh3

Good one, but Daler's answer saves more typing as FIFO is inbuilt now. Thanks

• 0 views • link
written by Sukhi Singh Sukhi Singh

Might be easier to just use several commands separated by semi-colon:

sed 1d fileB.bed > fileB.temp.bed;sed 1d fileA.bed | intersectBed -a stdin -b peaks.bed | intersectBed -u -a stdin -b fileB.temp.bed;rm -rf fileB.temp.bed
• 0 views • link
written by Damian Kao Damian Kao

Yeah, but this is a workaround, need something better.

• 0 views • link
written by Sukhi Singh Sukhi Singh

Log in to answer this question.

More posts like this