This is a test version of Biostars. For the public version, visit https://www.biostars.org.
wf-trio (EPI2ME Desktop) – aggregate_pileup_gvcf failing with exit status 137 despite high-memory system

Hi everyone,

I’m running the EPI2ME Desktop wf-trio pipeline and consistently hitting an OOM-style failure:

###############################################
NOTE: Process `snp_trio:snp_mat:aggregate_pileup_gvcf (1)`
terminated with an error exit status (137) -- Execution is retried (1)
###############################################

System details
Execution: EPI2ME Desktop (GUI)
OS: Ubuntu
CPUs: 100
RAM: ~500 GB
Storage: local SSD
Data: Nanopore trio WGS (30X coverrage, WGS 90Gb)

So this doesn’t seem to be a true system-level memory limitation in my opinion

Based on the Nextflow troubleshooting guide for exit 137, I assumed this is either:

  1. an OOM kill due to per-process memory limits, or
  2. multiple heavy processes running concurrently.

I added the following Extra configuration in the EPI2ME Desktop launch window:

########################################

process {
        withName: 'snp_trio:.*aggregate_pileup_gvcf' {
        maxForks = 1

    }

    withName: 'snp_trio:callVarBam_trio' {
        maxForks = 1

    }

    maxForks = 4
}

########################################

Despite this, aggregate_pileup_gvcf is still being killed with exit code 137.

########################################

sequential errors in log file is:

1. [b6/76c151] Submitted process > snp_trio:snp_mat:aggregate_pileup_variants (1)
[2e/6c1f42] NOTE: Process `snp_trio:snp_mat:aggregate_pileup_gvcf (1)` terminated with an error exit status (137) -- Execution is retried (1)
[46/676e2b] Re-submitted process > snp_trio:snp_mat:aggregate_pileup_gvcf (1)
[63/532323] Submitted process > snp_trio:snp_mat:select_het_snps (19)

2.[df/9e5236] Submitted process > snp_trio:snp_pat:post_clair_contig_haplotag (1)
[8f/71545b] NOTE: Process `snp_trio:snp_pat:aggregate_pileup_gvcf (1)` terminated with an error exit status (137) -- Execution is retried (1)
[8d/dcb6f0] Re-submitted process > snp_trio:snp_pat:aggregate_pileup_gvcf (1)
[f1/bab217] Submitted process > snp_trio:snp_pat:post_clair_contig_haplotag (2)


3. [ea/d3a0b2] Submitted process > snp_trio:callVarBam_trio (93)
[42/223d0b] NOTE: Process `snp_trio:callVarBam_trio (23)` terminated with an error exit status (142) -- Execution is retried (1)
[3e/d8da25] Submitted process > snp_trio:callVarBam_trio (92)


4. [09/44028f] Submitted process > snp_trio:sortgvcf_Trio_contig (62)
ERROR ~ Error executing process > 'snp_trio:glnexus (18)'
Caused by:
  Process `snp_trio:glnexus (18)` terminated with an error exit status (9)
Command executed:
  glnexus_cli --threads 4 --dir glnexus_DB --config "gl_config.yml"     "proband.gvcf.gz" "paternal.gvcf.gz"  "maternal.gvcf.gz"      | bcftools view --threads 1 -o merged.vcf.gz -O z -
  bcftools index --threads 1 -t merged.vcf.gz

  echo -e A"\n"R"\n"S > samples.lst
  bgzip "proband.vcf"
  tabix -p vcf "proband.vcf.gz"

  bcftools view -S samples.lst -o "Family_RSA1.chr7.vcf.gz" merged.vcf.gz
  bcftools index --threads 1 -t "Family_RSA1.chr7.vcf.gz"
Command exit status:
  9
Command output:
  (empty)
Command error:
  unifier_config:
    drop_filtered: false
    min_allele_copy_number: 1
    min_AQ1: 0
    min_AQ2: 0
    min_GQ: 0
    max_alleles_per_site: 32
    monoallelic_sites_for_lost_alleles: true
    preference: common
  genotyper_config:
    revise_genotypes: true
    min_assumed_allele_frequency: 9.99999975e-05
    snv_prior_calibration: 1
    indel_prior_calibration: 1
    required_dp: 1
    allow_partial_data: true
    allele_dp_format: AD
    ref_dp_format: MIN_DP
    output_residuals: false
    more_PL: true
    squeeze: false
    trim_uncalled_alleles: true
    top_two_half_calls: false
    output_format: BCF
    liftover_fields:
      - {orig_names: [MIN_DP, DP], name: DP, description: "##FORMAT=<ID=DP,Number=1,Type=Integer,Description=\"Approximate read depth (reads with MQ=255 or with bad mates are filtered)\">", type: int, number: basic, default_type: missing, count: 1, combi_method: min, ignore_non_variants: true}
      - {orig_names: [AD], name: AD, description: "##FORMAT=<ID=AD,Number=R,Type=Integer,Description=\"Allelic depths for the ref and alt alleles in the order listed\">", type: int, number: alleles, default_type: zero, count: 0, combi_method: min, ignore_non_variants: false}
      - {orig_names: [GQ], name: GQ, description: "##FORMAT=<ID=GQ,Number=1,Type=Integer,Description=\"Genotype Quality\">", type: int, number: basic, default_type: missing, count: 1, combi_method: min, ignore_non_variants: true}
      - {orig_names: [PL], name: PL, description: "##FORMAT=<ID=PL,Number=G,Type=Integer,Description=\"Phred-scaled genotype Likelihoods\">", type: int, number: genotype, default_type: missing, count: 0, combi_method: missing, ignore_non_variants: true}
      - {orig_names: [DNP], name: DNP, description: "##FORMAT=<ID=DNP,Number=.,Type=Float,Description=\"de novo variant probability\">", type: float, number: basic, default_type: missing, count: 1, combi_method: missing, ignore_non_variants: true}
  [27] [2026-01-12 03:49:48.312] [GLnexus] [info] config CRC32C = 1351409500
  [27] [2026-01-12 03:49:48.312] [GLnexus] [info] init database, exemplar_vcf=proband.gvcf.gz
  [27] [2026-01-12 03:49:48.428] [GLnexus] [info] Initialized GLnexus database in glnexus_DB
  [27] [2026-01-12 03:49:48.428] [GLnexus] [info] bucket size: 30000
  [27] [2026-01-12 03:49:48.428] [GLnexus] [info] contigs: chr7
  [27] [2026-01-12 03:49:48.458] [GLnexus] [info] db_get_contigs glnexus_DB
  [27] [2026-01-12 03:49:48.489] [GLnexus] [info] Beginning bulk load with no range filter.
  [27] [2026-01-12 03:50:28.484] [GLnexus] [info] Loaded 3 datasets with 3 samples; 5044566832 bytes in 47530934 BCF records (14377 duplicate) in 15933 buckets. Bucket max 1477568 bytes, 13943 records. 1 BCF records skipped due to caller-specific exceptions
  [27] [2026-01-12 03:50:28.484] [GLnexus] [info] Created sample set *@3
  [27] [2026-01-12 03:50:28.484] [GLnexus] [info] Flushing database...
  [27] [2026-01-12 03:50:38.550] [GLnexus] [info] Bulk load complete!
  [27] [2026-01-12 03:50:38.804] [GLnexus] [warning] Processing full length of 1 contigs, as no --bed was provided. Providing a BED file with regions of interest, if applicable, can speed this up.
  [27] [2026-01-12 03:50:38.806] [GLnexus] [info] found sample set *@3
  [27] [2026-01-12 03:50:38.806] [GLnexus] [info] discovering alleles in 1 range(s) on 2 threads
  [27] [2026-01-12 03:50:52.968] [GLnexus] [info] discovered 847963 alleles
  [27] [2026-01-12 03:50:54.977] [GLnexus] [info] unified to 407920 sites cleanly with 427772 ALT alleles. 314 ALT alleles were additionally included in monoallelic sites and 0 were filtered out on quality thresholds.
  [27] [2026-01-12 03:50:54.977] [GLnexus] [info] Finishing database compaction...

[27] [2026-01-12 03:50:55.018] [GLnexus] [info] genotyping 407920 sites; sample set = *@3 mem_budget = 0 threads = 4
  [W::bgzf_read_block] EOF marker is absent. The input may be truncated
  Error: BCF read error

Work dir:
  /home/user/epi2melabs/instances/wf-trio_01KEQ7QEFVGWP2B7SDZY9QCBQR/work/b8/66ca4c29765eee361abf0c4e9fa3f5
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
 -- Check '/home/user/epi2melabs/instances/wf-trio_01KEQ7QEFVGWP2B7SDZY9QCBQR/nextflow.log' file for details
Execution cancelled -- Finishing pending tasks before exit
#####################################################

Any guidance from people who have successfully run wf-trio on large datasets would be really helpful.

Thanks in advance!

wf-trio

1 answer

[W::bgzf_read_block] EOF marker is absent. The input may be truncated

check the *.g.vcf.gz files under /home/user/epi2melabs/instances/wf-trio_01KEQ7QEFVGWP2B7SDZY9QCBQR/work/b8/66ca4c29765eee361abf0c4e9fa3f5 are not truncated.

something like:

find /home/user/epi2melabs/instances/wf-trio_01KEQ7QEFVGWP2B7SDZY9QCBQR/work/b8/66ca4c29765eee361abf0c4e9fa3f5 -name "*.vcf.gz" | while read F; do echo "$F" &&  gunzip -t "${F}" || echo "FAIL"  ; done

if any fail, check the upstream files.

Log in to answer this question.