This is a test version of Biostars. For the public version, visit https://www.biostars.org.
samtools: writing to standard output failed: Broken pipe

Hi everyone!

I'm getting this error from tophat:

samtools: writing to standard output failed: Broken pipe

Log file:

[2014-09-15 16:31:23] Beginning TopHat run (v2.0.12)
-----------------------------------------------
[2014-09-15 16:31:23] Checking for Bowtie
                  Bowtie version:        2.2.3.0
[2014-09-15 16:31:23] Checking for Samtools
[2014-09-15 16:31:23] Checking for Bowtie index files (genome)..
[2014-09-15 16:31:23] Checking for reference FASTA file
[2014-09-15 16:31:23] Generating SAM header for 
[2014-09-15 16:31:23] Reading known junctions from GTF file
[2014-09-15 16:31:27] Preparing reads
         left reads: min. length=101, max. length=101, 99994 kept reads (6 discarded)
        right reads: min. length=101, max. length=101, 99455 kept reads (545 discarded)
[2014-09-15 16:31:32] Building transcriptome data files test_LC1.THout/tmp/Lj2.5_gene_models
[2014-09-15 16:31:38] Building Bowtie index from Lj2.5_gene_models.fa
[2014-09-15 16:32:07] Mapping left_kept_reads to transcriptome Lj2.5_gene_models with Bowtie2 
[2014-09-15 16:32:13] Mapping right_kept_reads to transcriptome Lj2.5_gene_models with Bowtie2 
[2014-09-15 16:32:18] Resuming TopHat pipeline with unmapped reads
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:18] Mapping left_kept_reads.m2g_um to genome Bowtie2Index with Bowtie2 
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:28] Mapping left_kept_reads.m2g_um_seg1 to genome Bowtie2Index with Bowtie2 (1/4)
[2014-09-15 16:32:31] Mapping left_kept_reads.m2g_um_seg2 to genome Bowtie2Index with Bowtie2 (2/4)
[2014-09-15 16:32:33] Mapping left_kept_reads.m2g_um_seg3 to genome Bowtie2Index with Bowtie2 (3/4)
[2014-09-15 16:32:36] Mapping left_kept_reads.m2g_um_seg4 to genome Bowtie2Index with Bowtie2 (4/4)
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:41] Mapping right_kept_reads.m2g_um to genome Bowtie2Index with Bowtie2 
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:32:50] Mapping right_kept_reads.m2g_um_seg1 to genome Bowtie2Index with Bowtie2 (1/4)
[2014-09-15 16:32:53] Mapping right_kept_reads.m2g_um_seg2 to genome Bowtie2Index with Bowtie2 (2/4)
[2014-09-15 16:32:56] Mapping right_kept_reads.m2g_um_seg3 to genome Bowtie2Index with Bowtie2 (3/4)
[2014-09-15 16:32:59] Mapping right_kept_reads.m2g_um_seg4 to genome Bowtie2Index with Bowtie2 (4/4)
[2014-09-15 16:33:03] Searching for junctions via segment mapping
[2014-09-15 16:33:18] Retrieving sequences for splices
[2014-09-15 16:33:31] Indexing splices
Building a SMALL index
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
[2014-09-15 16:33:36] Mapping left_kept_reads.m2g_um_seg1 to genome segment_juncs with Bowtie2 (1/4)
[2014-09-15 16:33:38] Mapping left_kept_reads.m2g_um_seg2 to genome segment_juncs with Bowtie2 (2/4)
[2014-09-15 16:33:39] Mapping left_kept_reads.m2g_um_seg3 to genome segment_juncs with Bowtie2 (3/4)
[2014-09-15 16:33:41] Mapping left_kept_reads.m2g_um_seg4 to genome segment_juncs with Bowtie2 (4/4)
[2014-09-15 16:33:42] Joining segment hits
samtools: writing to standard output failed: Broken pipe
samtools: error closing standard output: -1
samtools: writing to standard output failed: Broken pipe

and so on..

I couldn't find any specific solution...

Thanks in advance!

MF

rna-seq
• 19,877 views • link
updated by Ram Ram • written by mcff23 • mcff23

Could you put your tophat command you used? Do you have the write permission in the tophat output directory? And also samtools version? Thanks

• 0 views • link
updated by Ram Ram • written by Nicolas Rosewick Nicolas Rosewick

HI NicoBxl!

command:

tophat -p 8 -G annotation -o sample.THout bowtie2_index R1.fastq.gz R2.fastq.gz

Yes I have write permissions in the output directory. The samtools version is the latest 1.0

Thanks!

MF

• 0 views • link
updated by Ram Ram • written by mcff23 • mcff23

How much memory do you have on the computer running this? Also, have you tried an older version of samtools? Tophat2 doesn't natively support the most recent version.

• 0 views • link
written by Devon Ryan Devon Ryan

With that much RAM you'd be better off using STAR. Tophat is painfully slow and mildly annoying to debug.

• 0 views • link
updated by Ram Ram • written by Devon Ryan Devon Ryan

I would not muck around with unofficial patches, so many things can also break, just use the supported samtools until the main trunk supports it natively

Invoke the new samtools separately in your own work

• 1 views • link
updated by Ram Ram • written by Istvan Albert Istvan Albert

Check that your disk-space isn't close to full

• 0 views • link
updated by Ram Ram • written by spiderdijon • spiderdijon

I think that I have tracked the issue down to the following code in tophat.py:

samtools_view_cmd = ["samtools", "view", filepath]
samtools_view = subprocess.Popen(samtools_view_cmd, stdout=subprocess.PIPE)
head_cmd = ["head", "-1"]
head = subprocess.Popen(head_cmd, stdin=samtools_view.stdout, stdout=subprocess.PIPE)
samtools_view.stdout.close() # as per http://bugs.python.org/issue7678
output = head.communicate()[0][:-1]

The issue appears to be that python's handling of SIGPIPEs in this kind of pipeline is nonstandard - see http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html - and that the newer version of samtools cares about this, while the old version didn't.

Installing the backport of python3k's subprocess at http://code.google.com/p/python-subprocess32/ and changing the above code to use this version of subprocess gets rid of the error messages.

• 0 views • link
updated by Ram Ram • written by Jeff Hussmann Jeff Hussmann

When I submit Bismark jobs in PBS system, I did get some similar error report. However, every result have been successfully obtained. Even that the error log file will show you: 

bismark_methylation_extractor --single-end --bedGraph --cutoff 5 --multicore 3 --ignore 3 --ignore_3prime 3 --gzip --buffer_size 4G --zero_based --comprehensive --output ../methyfreq/ CTR154_trimmed.fq.gz_bismark_bt2.bam
Finished BedGraph conversion ...
gzip: stdout: Broken pipe

Don't care about them. I think it is the problem of PBS system.

• 1 views • link
written by Shicheng Guo Shicheng Guo

the same thing happened to me as well. This is the end of fhe PBS output file. but I got the coverage file.

Finished generating genome-wide cytosine report

samtools view: writing to standard output failed: Broken pipe
samtools view: error closing standard output: -1

gzip: stdout: Broken pipe
• 0 views • link
updated by Ram Ram • written by cjgunase • cjgunase

1 answer

The solution for this error was using an older version of samtools like Devon Ryan said. I though the patch was working fine and it wasn't.

With Samtools 0.1.19.0 it worked just fine!!

Thanks a lot guys!

• 0 views • link
updated by Ram Ram • written by mcff23 • mcff23

Thanks for reporting back and also mentioning this on the tophat forum.

• 0 views • link
updated by Ram Ram • written by Devon Ryan Devon Ryan

This error may also happen when using bam2cfg.pl in BreakDancer.

The solution was the same: use an older version.

• 0 views • link
updated by Ram Ram • written by xieshaojun0621 xieshaojun0621

Log in to answer this question.

More posts like this