Hi,
I just ran a minia assembly on a 3GB plant genome. Everything went smooth so far and I got contigs. However, in the log its says
[assembly] 70 % elapsed: 1042 min 31 sec estimated remaining: 446 min 47 sec minia
in the last line before final stats. Do the 70% mean that something went wrong and things terminated prematurely?
2 answers
Hello,
Indeed it seems a little bit strange. In order to try to understand, could you tell
- the version of minia you use
- the command line you used
- the full stats you got in the command line at the end of the execution
- you should have got a tool named "dbginfo" in the minia distribution; could you try "dbginfo -in yourfile.h5" and send the result ?
Actually, it doesn't look like a crash since it seemed you got final stats. The displayed percentage corresponds to the number of branching nodes (in the de Bruijn graph) that have been processed so far and there is no reason why it doesn't go to 100%.
1. minia-0.1.1 (It was already installed - I just saw, there is v1.0.3 available)
2.
minia \
-in il-01.fq,il-02.fq,il-03.fq,il-04.fq \
-kmer-size 31 \
-out minia-01 \
-out-dir minia-01 \
-max-memory 100000 \
-nb-cores 31
3.
input
-in : il-01.fq,il-02.fq,il-03.fq,il-04.fq
-traversal : monument
-starter : best
-contig-max-len : 10000000
-bfs-max-depth : 500
-bfs-max-breadth : 20
-kmer-size : 31
-abundance : 3
-bank-convert : tmp
-out : minia-01
-out-dir : minia-01
-verbose : 1
-max-memory : 100000
-max-disk : 0
-nb-cores : 31
-bloom : cache
-debloom : cascading
-branching-nodes : stored
-mphf : none
stats
traversal : monument
start_selector : best
nb_contigs : 5706141
nb_small_contigs_discarded : 16937817
nt_assembled : 640246631
max_length : 14688
max_length_left : 13719
max_length_right : 11291
debugging traversal stats
couldn't validate consensuses : 4596
large bubble breadth : 244
large bubble depth : 661
stopped at marked kmer : 4277466
no kmer extension : 73627
in-branchin large depth : 1286943
in-branching large breadth : 1693114
in-branching other : 4033741
time : 66263.691
assembly : 66263.691
4.
graph
gatb-core-library :
version : 1.0.5
build_date : 2014-12-07 04:45:49
build_system : Linux-3.16.0-4-amd64
build_compiler : /usr/bin/cc (4.9.1)
build_kmer_size : 32 64 96 128
dsk :
config :
kmer_size : 31
abundance : 3
available_space : 9304175
bank_size : 111738
sequence_number : 727274793
sequence_volume : 111738
kmers_number : 95347730379
kmers_volume : 727445
max_disk_space : 335214
max_memory : 100000
nb_passes : 1
nb_partitions : 155
nb_bits_per_kmer : 64
nb_cores : 31
partition_type : 0
nb_cores_per_partition : 1
nb_partitions_in_parallel : 31
stats :
kmers_nb_distinct : 3306584899
kmers_nb_solid : 1150271578
kmers_nb_weak : 2156313321
kmers_percent_weak : 65.2
fillsolid_time : 1021.531
1.read : 779.100
2.sort : 158.989
3.dump : 83.442
time : 3347.592
fill_partitions : 2109.983
fill_solid_kmers : 1237.609
bloom :
stats :
kind : cache
bitsize : 6941164544
nb_hash : 4
nbits_per_kmer : 6.034370
time : 93.784
build_from_kmers : 93.784
debloom :
stats :
kind : cascading
size : 9811137808
nbits_per_kmer : 8.529409
cfp : 2869973264
bloom2 : 2279565312
bloom3 : 387835008
bloom4 : 127369872
set : 75203072
time : 821.271
cascading : 127.392
fill_debloom_file : 348.423
finalize_debloom_file : 345.456
branching :
stats :
nb_branching : 90209237
percentage : 7.8
time :
build : 437.596
Ok, please let me know about your new run.
I don't see much wrong in the information you got. Getting all the stats at the end of minia means that it didn't crash, so I don't see what happened. The only possibility is that the loop over the branching nodes is stopped at 70% because reading them from the h5 file is wrongly considered as finished (something bad in the h5 file ?)
If you still have the h5 file, you could try the following to check how many branching nodes are actually in the h5 file (the h5dump command should also be in the minia distrib):
h5dump -d /branching/nodes yourfile.h5 | head -30
Then, the value at line starting by "DATASPACE" should be the number of branching nodes, 90209237 in your case (according to the dump of dbginfo)
I think I found the problem. I checked dmesg on the node that minia ran on. There are a couple of "NFS: state manager:" errors in the log right about the time the run finished. So you are probably right, minia lost connection to the h5 file during processing. Sorry for bugging you with this...
I installed minia-1.0.3 and will rerun the assembly. If I encounter any problems, I will get back to you
Log in to answer this question.
-
Minia tutorial in the Biostar Handbook
written by damonlbp •
-
Segmentation fault when running Minia v3 with some kmer values
written by matthew.nicotra •
-
minia seg fault (core dump) after inability to unlink mphf temp files?
written by mmelendrez •
-
Errors with larger kmers
written by hiimwalker •
-
Segmentation fault error while using GATB
written by AriyurD •
-
Minia crashes during kmer counting
written by shammond •
-
Internal memory allocator is full!
written by cmckay •
-
Scaffolding with SSPACE returned scaffold the same as initial contigs set
written by pbigbig
-
Minia exits with error in "Debloom: cascading" stage
written by Shellfishgene •
-
VelvetOptimiser: optimum cov_cutoff, PE insert stats, 2 Insert Size Libraries, N's without scaffold…
written by drtamermansour •
Is there a separate error log file? Maybe an error was displayed to stdout? This does look like premature termination.
No, no errors or something similar - that is what confuses me.
Hello thackl!
We believe that this post does not fit the main topic of this site.
The assembly terminated prematurely due to an NFS exception not connected to the minia assembler.
For this reason we have closed your question. This allows us to keep the site focused on the topics that the community can help with.
If you disagree please tell us why in a reply below, we'll be happy to talk about it.
Cheers!