I am running following command
vg construct -C -r chr1.fasta -v chr1.vcf.gz -t 1 -m 32 > human_v38.chr1.vg
with the following error
terminate called after throwing an instance of 'std::runtime_error'
what(): io::MessageEmitter::write: message too large
Crash report for vg v1.62.0 Ranzano
Stack trace (most recent call last):
#18 Object /usr/local/bin/vg at 0x62c1a4 in _start
#17 Object /usr/local/bin/vg at 0x20fff36 in __libc_start_main
#16 Object /usr/local/bin/vg at 0x20fe699 in __libc_start_call_main
#15 Object /usr/local/bin/vg at 0xe22bab in vg::subcommand::Subcommand::operator()(int char**) const
#14 Object /usr/local/bin/vg at 0xcc8aeb in main_construct(int char**)
#13 Object /usr/local/bin/vg at 0xfb5f5e in vg::Constructor::construct_graph(std::vector<std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > std::allocator<std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > > > const& std::vector<std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > std::allocator<std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > > > const& std::vector<std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > std::allocator<std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > > > const& std::function<void (vg::Graph&)> const&)
#12 Object /usr/local/bin/vg at 0xfb5124 in vg::Constructor::construct_graph(std::vector<FastaReference* std::allocator<FastaReference*> > const& std::vector<vcflib::VariantCallFile* std::allocator<vcflib::VariantCallFile*> > const& std::vector<FastaReference* std::allocator<FastaReference*> > const& std::function<void (vg::Graph&)> const&)
#11 Object /usr/local/bin/vg at 0xfb3cf3 in vg::Constructor::construct_graph(std::__cxx11::basic_string<char std::char_traits<char> std::allocator<char> > FastaReference& vg::VcfBuffer& std::vector<FastaReference* std::allocator<FastaReference*> > const& std::function<void (vg::Graph&)> const&)
#10 Object /usr/local/bin/vg at 0xcca190 in std::_Function_handler<void (vg::Graph&) main_construct(int char**)::{lambda(vg::Graph&)#1}>::_M_invoke(std::_Any_data const& vg::Graph&)
#9 Object /usr/local/bin/vg at 0xb545b0 in vg::io::ProtobufEmitter<vg::Graph>::write_copy(vg::Graph const&)
#8 Object /usr/local/bin/vg at 0x20fbd9d in _Unwind_Resume
#7 Object /usr/local/bin/vg at 0x20fb21b in _Unwind_RaiseException_Phase2
#6 Object /usr/local/bin/vg at 0x2037a39 in gxx_personality_v0
#5 Object /usr/local/bin/vg at 0x20d1ee8 in __cxa_call_terminate
#4 Object /usr/local/bin/vg at 0x20382eb in __cxxabiv1::__terminate(void (*)())
#3 Object /usr/local/bin/vg at 0x5f2bcb in __gnu_cxx::__verbose_terminate_handler() [clone .cold]
#2 Object /usr/local/bin/vg at 0x5f5313 in abort
#1 Object /usr/local/bin/vg at 0x2117115 in raise
#0 Object /usr/local/bin/vg at 0x2143c6c in __pthread_kill
What could be the reason and how to solve?
1 answer
vg construct makes the graph in pieces called "chunks", which by default include about 1024 variants, and saves each chunk to a Protobuf message. Protobuf messages can only be so large, and when writing them vg tries to limit each message to 1 GB. If it would write one that's larger, it fails with this message.
Try reducing the number of variants per chunk by setting -z/--region-size to a value less than 1024 on the command line, and see if that helps.
It could also be that you have a large number of overlapping variants; each chunk needs to include a complete set of overlapped variants, so chunks will grow until no more variants overlap those already in the chunk. If you have so many overlapping variants that vg can't split them into manageable-sized chunks, vg might need to be modified to work on your data.
Log in to answer this question.