Skip to content

Stuck forever on human chromosome read generation #241

@jdalino

Description

@jdalino

Describe the bug
Trying to simulate a trisomy event on human chr21, I used NEAT to generate reads with ploidy 3 on the chromosome, but the program end-up stuck in a infinite loop, without generating any results for hours.

To Reproduce
Use this config file:

reference: references/21.fa
read_len: 151
ploidy: 3
coverage: 3
paired_ended: true
fragment_mean: 300
fragment_st_dev: 30
produce_bam: true
produce_vcf: true
produce_fastq: false
threads: 64
parallel_mode: size
parallel_block_size: 10000000
rng_seed: 9999
overwrite_output: true
cleanup_splits: true

Launch with

neat read-simulator \
    -c test.yml" \
    -o test_chr21" \
    -p "test"

Expected behavior

Screenshots
Log:

2026-02-13 09:00:04,268:INFO:neat.common.logging:writing log to: /tools/NEAT/1770973203.5303376_NEAT.log
2026-02-13 09:00:04,268:INFO:neat.read_simulator.runner:Using configuration file /output/ploidy_sim/configs/test.yml
2026-02-13 09:00:04,269:INFO:neat.read_simulator.runner:Saving output files to /output/ploidy_sim/components/test_ch
r21
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Run Configuration...
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Input fasta: /references/21.fa
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Outputting files to /components/tes
t_chr21
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Producing the following files:
	- /components/test_chr21/test_golden.bam
	- /components/test_chr21/test_golden.vcf.gz

2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Running read simulator in parallel mode.
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Multithreading - 64 threads (or CPU Max)
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Splitting input by contig.
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Using pysam: 0.23.3
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Using a read length of 151
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Generating fragments based on mean=300, stand. dev=30
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Running in paired-ended mode.
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Average coverage: 3
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Using default error model.
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:Ploidy value: 3
2026-02-13 09:00:04,270:INFO:neat.read_simulator.utils.options:RNG seed value for run: 9999
2026-02-13 09:00:05,814:INFO:neat.read_simulator.runner:Splitting reference...
2026-02-13 09:00:21,627:INFO:neat.read_simulator.utils.split_inputs:Generated 94 FASTAs in /tmp/tmprhwojgy5/splits
2026-02-13 09:00:21,627:INFO:neat.read_simulator.runner:[parallel] Launching 94 NEAT job(s) (max 64 in parallel)...
2026-02-13 09:00:21,997:INFO:neat.read_simulator.runner:launching mutiprocess simulation, recording results.
2026-02-13 09:00:26,416:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 32 in 0.06 m
2026-02-13 09:00:26,435:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 48 in 0.06 m
2026-02-13 09:00:26,442:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 50 in 0.06 m
2026-02-13 09:00:26,444:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 64 in 0.06 m
2026-02-13 09:00:26,446:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 35 in 0.06 m
2026-02-13 09:00:26,447:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 40 in 0.06 m
2026-02-13 09:00:26,450:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 61 in 0.06 m
2026-02-13 09:00:26,456:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 39 in 0.06 m
2026-02-13 09:00:26,456:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 33 in 0.06 m
2026-02-13 09:00:26,458:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 54 in 0.06 m
2026-02-13 09:00:26,462:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 28 in 0.06 m
2026-02-13 09:00:26,462:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 27 in 0.06 m
2026-02-13 09:00:26,467:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 43 in 0.06 m
2026-02-13 09:00:26,468:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 36 in 0.06 m
2026-02-13 09:00:26,469:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 38 in 0.06 m
2026-02-13 09:00:26,471:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 57 in 0.06 m
2026-02-13 09:00:26,474:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 30 in 0.06 m
2026-02-13 09:00:26,476:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 47 in 0.06 m
2026-02-13 09:00:26,481:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 51 in 0.06 m
2026-02-13 09:00:26,481:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 55 in 0.06 m
2026-02-13 09:00:26,481:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 41 in 0.06 m
2026-02-13 09:00:26,484:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 60 in 0.06 m
2026-02-13 09:00:26,492:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 42 in 0.06 m
2026-02-13 09:00:26,495:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 31 in 0.06 m
2026-02-13 09:00:26,495:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 29 in 0.06 m
2026-02-13 09:00:26,497:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 34 in 0.06 m
2026-02-13 09:00:26,497:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 49 in 0.06 m
2026-02-13 09:00:26,500:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 44 in 0.06 m
2026-02-13 09:00:26,509:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 46 in 0.06 m
2026-02-13 09:00:26,512:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 56 in 0.06 m
2026-02-13 09:00:26,514:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 45 in 0.06 m
2026-02-13 09:00:26,517:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 63 in 0.06 m
2026-02-13 09:00:26,530:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 37 in 0.06 m
2026-02-13 09:00:26,537:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 53 in 0.06 m
2026-02-13 09:00:26,541:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 52 in 0.06 m
2026-02-13 09:00:26,545:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 62 in 0.06 m
2026-02-13 09:00:26,553:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 58 in 0.06 m
2026-02-13 09:00:26,564:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 59 in 0.06 m
2026-02-13 09:00:28,517:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 20 in 0.10 m
2026-02-13 09:00:28,653:INFO:neat.read_simulator.single_runner:bam for thread 32 written
2026-02-13 09:00:28,718:INFO:neat.read_simulator.single_runner:bam for thread 38 written
2026-02-13 09:00:28,722:INFO:neat.read_simulator.single_runner:bam for thread 40 written
2026-02-13 09:00:28,736:INFO:neat.read_simulator.single_runner:bam for thread 27 written
2026-02-13 09:00:28,750:INFO:neat.read_simulator.single_runner:bam for thread 28 written
2026-02-13 09:00:28,754:INFO:neat.read_simulator.single_runner:bam for thread 30 written
2026-02-13 09:00:28,755:INFO:neat.read_simulator.single_runner:bam for thread 47 written
2026-02-13 09:00:28,762:INFO:neat.read_simulator.single_runner:bam for thread 64 written
2026-02-13 09:00:28,764:INFO:neat.read_simulator.single_runner:bam for thread 36 written
2026-02-13 09:00:28,768:INFO:neat.read_simulator.single_runner:bam for thread 51 written
2026-02-13 09:00:28,768:INFO:neat.read_simulator.single_runner:bam for thread 42 written
2026-02-13 09:00:28,772:INFO:neat.read_simulator.single_runner:bam for thread 35 written
2026-02-13 09:00:28,773:INFO:neat.read_simulator.single_runner:bam for thread 41 written
2026-02-13 09:00:28,773:INFO:neat.read_simulator.single_runner:bam for thread 44 written
2026-02-13 09:00:28,775:INFO:neat.read_simulator.single_runner:bam for thread 43 written
2026-02-13 09:00:28,775:INFO:neat.read_simulator.single_runner:bam for thread 63 written
2026-02-13 09:00:28,775:INFO:neat.read_simulator.single_runner:bam for thread 50 written
2026-02-13 09:00:28,794:INFO:neat.read_simulator.single_runner:bam for thread 29 written
2026-02-13 09:00:28,794:INFO:neat.read_simulator.single_runner:bam for thread 33 written
2026-02-13 09:00:28,797:INFO:neat.read_simulator.single_runner:bam for thread 61 written
2026-02-13 09:00:28,797:INFO:neat.read_simulator.single_runner:bam for thread 48 written
2026-02-13 09:00:28,797:INFO:neat.read_simulator.single_runner:bam for thread 49 written
2026-02-13 09:00:28,797:INFO:neat.read_simulator.single_runner:bam for thread 58 written
2026-02-13 09:00:28,807:INFO:neat.read_simulator.single_runner:bam for thread 57 written
2026-02-13 09:00:28,807:INFO:neat.read_simulator.single_runner:bam for thread 31 written
2026-02-13 09:00:28,808:INFO:neat.read_simulator.single_runner:bam for thread 59 written
2026-02-13 09:00:28,813:INFO:neat.read_simulator.single_runner:bam for thread 55 written
2026-02-13 09:00:28,813:INFO:neat.read_simulator.single_runner:bam for thread 39 written
2026-02-13 09:00:28,820:INFO:neat.read_simulator.single_runner:bam for thread 37 written
2026-02-13 09:00:28,822:INFO:neat.read_simulator.single_runner:bam for thread 34 written
2026-02-13 09:00:28,825:INFO:neat.read_simulator.single_runner:bam for thread 54 written
2026-02-13 09:00:28,827:INFO:neat.read_simulator.single_runner:bam for thread 62 written
2026-02-13 09:00:28,827:INFO:neat.read_simulator.single_runner:bam for thread 52 written
2026-02-13 09:00:28,830:INFO:neat.read_simulator.single_runner:bam for thread 60 written
2026-02-13 09:00:28,830:INFO:neat.read_simulator.single_runner:bam for thread 46 written
2026-02-13 09:00:28,841:INFO:neat.read_simulator.single_runner:bam for thread 56 written
2026-02-13 09:00:28,859:INFO:neat.read_simulator.single_runner:bam for thread 45 written
2026-02-13 09:00:28,883:INFO:neat.read_simulator.single_runner:bam for thread 53 written
2026-02-13 09:00:30,333:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 15 in 0.13 m
2026-02-13 09:00:31,321:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 69 in 0.04 m
2026-02-13 09:00:31,656:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 84 in 0.04 m
2026-02-13 09:00:31,683:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 76 in 0.04 m
2026-02-13 09:00:31,748:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 94 in 0.04 m
2026-02-13 09:00:32,634:INFO:neat.read_simulator.single_runner:bam for thread 69 written
2026-02-13 09:00:32,781:INFO:neat.read_simulator.single_runner:bam for thread 20 written
2026-02-13 09:00:32,850:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 17 in 0.17 m
2026-02-13 09:00:32,903:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 73 in 0.06 m
2026-02-13 09:00:32,979:INFO:neat.read_simulator.single_runner:bam for thread 84 written
2026-02-13 09:00:33,005:INFO:neat.read_simulator.single_runner:bam for thread 76 written
2026-02-13 09:00:33,012:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 79 in 0.06 m
2026-02-13 09:00:33,043:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 66 in 0.06 m
2026-02-13 09:00:33,068:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 68 in 0.06 m
2026-02-13 09:00:33,071:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 74 in 0.06 m
2026-02-13 09:00:33,073:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 65 in 0.06 m
2026-02-13 09:00:33,085:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 86 in 0.06 m
2026-02-13 09:00:33,102:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 67 in 0.06 m
2026-02-13 09:00:33,112:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 71 in 0.06 m
2026-02-13 09:00:33,156:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 90 in 0.06 m
2026-02-13 09:00:33,159:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 72 in 0.06 m
2026-02-13 09:00:33,164:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 85 in 0.06 m
2026-02-13 09:00:33,168:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 75 in 0.06 m
2026-02-13 09:00:33,171:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 70 in 0.06 m
2026-02-13 09:00:33,179:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 81 in 0.06 m
2026-02-13 09:00:33,183:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 77 in 0.06 m
2026-02-13 09:00:33,187:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 80 in 0.06 m
2026-02-13 09:00:33,187:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 78 in 0.06 m
2026-02-13 09:00:33,188:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 91 in 0.06 m
2026-02-13 09:00:33,195:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 82 in 0.06 m
2026-02-13 09:00:33,209:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 89 in 0.06 m
2026-02-13 09:00:33,223:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 83 in 0.06 m
2026-02-13 09:00:33,248:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 18 in 0.18 m
2026-02-13 09:00:33,255:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 88 in 0.06 m
2026-02-13 09:00:33,320:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 92 in 0.06 m
2026-02-13 09:00:33,376:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 93 in 0.06 m
2026-02-13 09:00:33,554:INFO:neat.read_simulator.single_runner:bam for thread 94 written
2026-02-13 09:00:34,322:INFO:neat.read_simulator.single_runner:bam for thread 79 written
2026-02-13 09:00:34,384:INFO:neat.read_simulator.single_runner:bam for thread 86 written
2026-02-13 09:00:34,460:INFO:neat.read_simulator.single_runner:bam for thread 85 written
2026-02-13 09:00:34,809:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 19 in 0.20 m
2026-02-13 09:00:34,828:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 21 in 0.20 m
2026-02-13 09:00:34,916:INFO:neat.read_simulator.single_runner:bam for thread 73 written
2026-02-13 09:00:34,922:INFO:neat.read_simulator.single_runner:bam for thread 65 written
2026-02-13 09:00:34,952:INFO:neat.read_simulator.single_runner:bam for thread 72 written
2026-02-13 09:00:35,008:INFO:neat.read_simulator.single_runner:bam for thread 91 written
2026-02-13 09:00:35,037:INFO:neat.read_simulator.single_runner:bam for thread 66 written
2026-02-13 09:00:35,038:INFO:neat.read_simulator.single_runner:bam for thread 90 written
2026-02-13 09:00:35,077:INFO:neat.read_simulator.single_runner:bam for thread 77 written
2026-02-13 09:00:35,166:INFO:neat.read_simulator.single_runner:bam for thread 74 written
2026-02-13 09:00:35,176:INFO:neat.read_simulator.single_runner:bam for thread 68 written
2026-02-13 09:00:35,228:INFO:neat.read_simulator.single_runner:bam for thread 67 written
2026-02-13 09:00:35,238:INFO:neat.read_simulator.single_runner:bam for thread 71 written
2026-02-13 09:00:35,270:INFO:neat.read_simulator.single_runner:bam for thread 75 written
2026-02-13 09:00:35,285:INFO:neat.read_simulator.single_runner:bam for thread 82 written
2026-02-13 09:00:35,290:INFO:neat.read_simulator.single_runner:bam for thread 88 written
2026-02-13 09:00:35,309:INFO:neat.read_simulator.single_runner:bam for thread 81 written
2026-02-13 09:00:35,318:INFO:neat.read_simulator.single_runner:bam for thread 80 written
2026-02-13 09:00:35,338:INFO:neat.read_simulator.single_runner:bam for thread 92 written
2026-02-13 09:00:35,346:INFO:neat.read_simulator.single_runner:bam for thread 78 written
2026-02-13 09:00:35,358:INFO:neat.read_simulator.single_runner:bam for thread 89 written
2026-02-13 09:00:35,380:INFO:neat.read_simulator.single_runner:bam for thread 70 written
2026-02-13 09:00:35,410:INFO:neat.read_simulator.single_runner:bam for thread 83 written
2026-02-13 09:00:35,551:INFO:neat.read_simulator.single_runner:bam for thread 93 written
2026-02-13 09:00:35,630:INFO:neat.read_simulator.single_runner:bam for thread 15 written
2026-02-13 09:00:35,796:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 22 in 0.22 m
2026-02-13 09:00:35,864:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 87 in 0.11 m
2026-02-13 09:00:36,225:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 12 in 0.23 m
2026-02-13 09:00:39,101:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 16 in 0.27 m
2026-02-13 09:00:39,275:INFO:neat.read_simulator.single_runner:bam for thread 87 written
2026-02-13 09:00:39,518:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 14 in 0.28 m
2026-02-13 09:00:39,890:INFO:neat.read_simulator.single_runner:bam for thread 17 written
2026-02-13 09:00:39,905:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 11 in 0.29 m
2026-02-13 09:00:40,843:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 13 in 0.30 m
2026-02-13 09:00:42,410:INFO:neat.read_simulator.single_runner:bam for thread 18 written
2026-02-13 09:00:42,809:INFO:neat.read_simulator.single_runner:bam for thread 21 written
2026-02-13 09:00:43,571:INFO:neat.read_simulator.single_runner:bam for thread 19 written
2026-02-13 09:00:43,771:INFO:neat.read_simulator.utils.generate_reads:Finished sampling reads for thread 26 in 0.35 m
2026-02-13 09:00:44,737:INFO:neat.read_simulator.single_runner:bam for thread 22 written
2026-02-13 09:00:45,210:INFO:neat.read_simulator.single_runner:bam for thread 12 written
2026-02-13 09:00:52,089:INFO:neat.read_simulator.single_runner:bam for thread 16 written
2026-02-13 09:00:54,746:INFO:neat.read_simulator.single_runner:bam for thread 11 written
2026-02-13 09:00:56,049:INFO:neat.read_simulator.single_runner:bam for thread 13 written
2026-02-13 09:00:57,191:INFO:neat.read_simulator.single_runner:bam for thread 14 written
2026-02-13 09:01:05,316:INFO:neat.read_simulator.single_runner:bam for thread 26 written

Then it stay stuck, using ~10cpu at 100% but doing nothing clear. The outputs files are inexistent (only the vcf is created but empty).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions