MetaVelvet gets stuck at "Starting pebble resolution..." step (individually and probably as a part of MetAMOS)

Hello

I am using Ubuntu 64-bit 14.04 LTS with 32 Gb of RAM and running Velvet v1.2.10 and MetaVelvet v1.2.02 with MAXKMERLENGTH=31. I have the following problem: MetaVelvet gets stuck at the "Pebble resolution" (scaffoliding) step when trying to assemble contigs from a test dataset of 100000 Illumina MiSeq 2x300 PE reads, which were not processed in any way (no trimming, adapter removal etc.). After reaching "Starting pebble resolution..." step meta-velvetg gets stuck for indeterminable amount of time with one core showing 100% load. The last time I tried to test it I waited for more than 12 hours before killing the process manually.

I also tried to use MetaVelvet as one of the assemblers in the MetAMOS pipeline and it froze with the same symptoms at the (probably) same step.

$ meta-velvetg Assem -ins_length 350

[meta-velvetg] Check command line options...
    OK. Your command line options seem to be good.

[meta-velvetg] Load meta-graph ...
[0.000001] Reading read set file Assem/Sequences;
[0.048786] 200000 sequences found
[0.222462] Done
[0.593372] Reading graph file Assem/Graph2
[0.593503] Graph has 148377 nodes and 200000 sequences
[meta-velvetg] Category = 'short1'    Ave. = 350, SD = 35
[meta-velvetg] Category = 'short2'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'long'    Ave. = -1, SD = -1
[meta-velveth] ...done (load meta-graph).

[meta-velvetg] Estimate coverage parameters...
[mate-velvetg] Estimate expected coverage ... yes.    Expected coverage = 1.87805
[mate-velvetg] Estimate expected coverages ... yes.
[1.431262] Writing into stats file Assem/meta-velvetg.Graph2-stats.txt...
[MetaHisto] First valley = 2
[MetaHisto] Largest peak coverage = 2 (frequency count = 6.35635e+06)
[MetaHisto] Noise cutoff coverage = 635635

[meta-velvetg] Warning: Can't find multiple coverage peaks.
[meta-velvetg] Trun on single coverage peak mode. 

[MetaGraph] 1-th coverage peak = 1.87805
[meta-velvetg] Estimate coverage cutoff ... yes.    Coverage cutoff = 0.939024
[meta-velvetg] ...done (estimate coverage parameters).

[meta-velvetg] Remove low & high coverage nodes ...
[meta-velvetg] Min. coverage cutoff for short reads        = 0.939024
[meta-velvetg] Min. coverage cutoff for long reads         = -1
[meta-velvetg] Max. coverage cutoff for short & long reads = -1
[meta-velvetg] Min. contig length                          = -1
[VelvetGraph]  === Remove low coverage nodes ===
[1.642372] Removing contigs with coverage < 0.939024...
[1.645049] Concatenation...
[1.655673] Renumbering nodes
[1.655685] Initial node count 148377
[1.656127] Removed 0 null nodes
[1.656145] Concatenation over!
[1.659085] Concatenation...
[1.669341] Renumbering nodes
[1.669351] Initial node count 148377
[1.669692] Removed 0 null nodes
[1.669698] Concatenation over!
[VelvetGraph]  === Remove high coverage nodes ===
[VelvetGraph]  === Clip tips hardly ===
[1.669714] Clipping short tips off graph, drastic
[1.675634] Concatenation...
[1.686590] Renumbering nodes
[1.686609] Initial node count 148377
[1.686959] Removed 0 null nodes
[1.686967] Concatenation over!
[1.686984] 148377 nodes left
[meta-velvetg] ...done (remove low & high coverage nodes).

[meta-velvetg] Scaffolding based on paired-end information ...
[MetaGraph]  === Scaffolding with single peak mode ===
[VelvetGraph]  === Rock Bank  ===
[1.687007] Read coherency...
[1.690673] Identifying unique nodes
[1.693808] Done, 82558 unique nodes counted
[1.693821] Trimming read tips
[1.701690] Renumbering nodes
[1.701704] Initial node count 148377
[1.702079] Removed 0 null nodes
[1.702087] Confronted to 0 multiple hits and 0 null over 0
[1.702092] Read coherency over!
[VelvetGraph]  === Create read paring array ===
[VelvetGraph]  === Detach dubious reads ===
[VelvetGraph]  === Activate gap markers ===
[VelvetGraph]  === Scaffolding ===
[1.703117] Starting pebble resolution...

Any help would be really appreciated.

metavelvet pebble-resolution metamos assembly

• 7,189 views • link

updated 4.5 years ago by Ram

• written 11.8 years ago by Tim

2 answers

Try Trimmomatic for adapter and quality trimming.
Recompile Velvet with MAXKMERLENGTH=191 (or even bigger kmer)
Rerun with k = 127 or 191 (ideal kmer should be 1/3nd to 2/3nd of read length)

Good luck:)

• 0 views • link

updated 4.6 years ago by Ram

• written 11.8 years ago by rtliu

Thank you for your suggestions, rtilu, but none of them worked, unfortunately. I tried (in different combinations):

Removing adapters and doing quality trimming of reads (I used BBTools instead of Trimmomatic, in my tests BBTools showed better performance than Trimmomatic);
Just trimming reads to a specified length (to 100 bp);
Running velvetg and meta-velvetg with --scaffolding no parameter;
Recompiling Velvet and MetaVelvet with MAXKMERLENGTH=303;
Re-running Velvet and MetaVelvet with a whole range of different kmer values;
Running MetaVelvet with different reference sets of reads.

Every time I ran MetaVelvet, the run ended up in the exactly same way - freezing at the "Starting pebble resolution..." step.

As many people previously used MetaVelvet with success, I started to suspect that something is wrong with my computer and/or Linux installation. Thus I installed Velvet and MetaVelvet under a virtual machine running Ubuntu 12.04.5 LTS and MetaVelvet worked perfectly that time. I also noticed that two different versions of g++ were used to compile Velvet and MetaVelvet: g++ 4.6.3 in Ubuntu 12.04 (the version that worked) and g++ 4.8.2 in Ubuntu 14.04. So I suspect that this behaviour is due to the differences in these two versions of g++ compiler.

TL;DR: MetaVelvet doesn't run properly under Ubuntu 14.04 when compiled with g++ 4.8.2, but works in Ubuntu 12.04.5 (g++ 4.6.3).

• 0 views • link

updated 4.5 years ago by Ram

• written 11.8 years ago by Tim

Omega was recently released as an overlap-graph de novo Assembler for Metagenomics

http://bioinformatics.oxfordjournals.org/content/30/19/2717

or try IDBA-UD, which was pretty good for metagenomic assembly.

http://i.cs.hku.hk/~alse/hkubrg/projects/idba_ud/index.html

• 0 views • link

updated 4.5 years ago by Ram

• written 11.8 years ago by rtliu

Thank you for recommendations, rtliu, these two programs already were my top two alternative choices. I am also pondering about trying SPAdes as well. But I really needed MetaVelvet output for comparison, as it is (probably) the most popular metagenomic assembler nowadays.

• 0 views • link

updated 4.5 years ago by Ram

• written 11.8 years ago by Tim

Have you used parameter "exp_cov auto"? As the last resort write to MetaVelvet author. By the way, I have tried SPAdes on one lane HiSeq metagenomic data but it ran of memory on my server.

• 0 views • link

updated 4.5 years ago by Ram

• written 11.8 years ago by rtliu

Yes, I used this parameter every time I ran the assembler, it didn't help. I am almost sure that this behaviour is related to the version of g++ compiler, so I will let the developers of MetaVelvet know about this issue soon.

As I am working with MiSeq dataset, I am inclined to try SPAdes, read many good things about it.

• 0 views • link

updated 4.5 years ago by Ram

• written 11.7 years ago by Tim

Did you try by downgrading g++ to 4.6.3?

Thanks!

• 0 views • link

updated 4.5 years ago by Ram

• written 10.7 years ago by armando.torre •

Hello,

I have the same problem : it blocks at "Starting pebble resolution..." and I use a Mac.

Do someone find a way to fix this ?

anais$ meta-velvetg Velvet_out

[meta-velvetg] Check command line options...
    OK. Your command line options seem to be good.

[meta-velvetg] Load meta-graph ...
[0.000000] Reading read set file Velvet_out/Sequences;
[0.575730] 2500000 sequences found
[2.926542] Done
[5.324953] Reading graph file Velvet_out/Graph2
[5.325003] Graph has 116327 nodes and 2500000 sequences
[meta-velvetg] Category = 'short1'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'short2'    Ave. = -1, SD = -1
[meta-velvetg] Category = 'long'    Ave. = -1, SD = -1
[meta-velveth] ...done (load meta-graph).

[meta-velvetg] Estimate coverage parameters...
[mate-velvetg] Estimate expected coverage ... yes.    Expected coverage = 1.64596
[mate-velvetg] Estimate expected coverages ... yes.
[9.092905] Writing into stats file Velvet_out/meta-velvetg.Graph2-stats.txt...
[MetaHisto] First valley = 2
[MetaHisto] Largest peak coverage = 2 (frequency count = 1.85845e+06)
[MetaHisto] Noise cutoff coverage = 185845

[meta-velvetg] Warning: Can't find multiple coverage peaks.
[meta-velvetg] Trun on single coverage peak mode.

[MetaGraph] 1-th coverage peak = 1.64596
[meta-velvetg] Estimate coverage cutoff ... yes.    Coverage cutoff = 0.822981
[meta-velvetg] ...done (estimate coverage parameters).

[meta-velvetg] Remove low & high coverage nodes ...
[meta-velvetg] Min. coverage cutoff for short reads        = 0.822981
[meta-velvetg] Min. coverage cutoff for long reads         = -1
[meta-velvetg] Max. coverage cutoff for short & long reads = -1
[meta-velvetg] Min. contig length                          = -1
[VelvetGraph]  === Remove low coverage nodes ===
[9.373288] Removing contigs with coverage < 0.822981...
[9.375317] Concatenation...
[9.382159] Renumbering nodes
[9.382177] Initial node count 116327
[9.382566] Removed 0 null nodes
[9.382570] Concatenation over!
[9.384559] Concatenation...
[9.391212] Renumbering nodes
[9.391231] Initial node count 116327
[9.391626] Removed 0 null nodes
[9.391630] Concatenation over!
[VelvetGraph]  === Remove high coverage nodes ===
[VelvetGraph]  === Clip tips hardly ===
[9.391658] Clipping short tips off graph, drastic
[9.394651] Concatenation...
[9.401089] Renumbering nodes
[9.401101] Initial node count 116327
[9.401495] Removed 0 null nodes
[9.401500] Concatenation over!
[9.401502] 116327 nodes left
[meta-velvetg] ...done (remove low & high coverage nodes).

[meta-velvetg] Scaffolding based on paired-end information ...
[MetaGraph]  === Scaffolding with single peak mode ===
[VelvetGraph]  === Rock Bank  ===
[9.401533] Read coherency...
[9.402940] Identifying unique nodes
[9.404654] Done, 53448 unique nodes counted
[9.404666] Trimming read tips
[9.407842] Renumbering nodes
[9.407857] Initial node count 116327
[9.408259] Removed 0 null nodes
[9.408267] Confronted to 0 multiple hits and 0 null over 0
[9.408278] Read coherency over!
[VelvetGraph]  === Create read paring array ===
[VelvetGraph]  === Detach dubious reads ===
[VelvetGraph]  === Activate gap markers ===
[VelvetGraph]  === Scaffolding ===
[9.419840] Starting pebble resolution...

• 0 views • link

updated 4.5 years ago by Ram

• written 11.7 years ago by Anaïs Vittu •

Anaïs Vittu, could you please post the version of C++ compiler you used for installation of MetaVelvet?

• 0 views • link

updated 4.5 years ago by Ram

• written 11.7 years ago by Tim

Hi,

My version of c++ is 4.8.3 (MacPorts gcc48 4.8.3_1).

• 0 views • link

updated 4.5 years ago by Ram

• written 11.7 years ago by Anaïs Vittu •