asyncio profiling errors

[alexanderjaffe]

hi parsa,

one other question for you. while batch profiling i am getting a high proportion of batches failing with the following error, which seems to derive from a asyncio buffer issue:

2026-06-16T20:16:09Z BATCH batch_4 FAILED state=failed progress=5/10 success=5 failed=0 attempt=2 message=running->failed; runtime error: samtools: /opt/conda/bin/../lib/libtinfow.so
.6: no version information available (required by samtools)\nsamtools: /opt/conda/bin/../lib/libncursesw.so.6: no version information available (required by samtools)\nsamtools: /opt
/conda/bin/../lib/libncursesw.so.6: no version information available (required by samtools)\nsamtools: /opt/conda/bin/../lib/libncursesw.so.6: no version information available (requi
red by samtools)\nTraceback (most recent call last):\n  File "/opt/conda/lib/python3.12/asyncio/streams.py", line 568, in readline\n    line = await self.readuntil(sep)\n           ^
^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/lib/python3.12/asyncio/streams.py", line 646, in readuntil\n    raise exceptions.LimitOverrunError(\nasyncio.exceptions.LimitOverrunError
: Separator is not found, and chunk exceed the limit

Any suggestions for how to resolve? I'm giving the program plenty of RAM, profile runner is only using about 8-10% of it.

thanks,

alex

[ParsaGhadermazi]

Hi @alexanderjaffe

This is definitely not a RAM issue. I have not seen this before in my tests, so thanks for finding this issue! Are you expecting very high depth at some positions on the scaffold you are profiling? Some details about this sample would be super helpful. Looks like some of the samples have succeeded so maybe this sample has some characteristics breaking it...

Parsa

[alexanderjaffe]

yes, there could be - we performed targeted sequencing of select pathogens so there could be up to 100,000x coverage in rare cases. if you think it would be wise, i could downsample reads before profiling.

[ParsaGhadermazi]

Interesting! This should be it then. I have increased the byte limit for asyncio subprocess and just pushed it. I think a way to see if this will fix your problem is to install zipstrain using: pip install --upgrade "zipstrain @ git+https://github.com/OlmLab/ZipStrain.git#subdirectory=zipstrain" or clone the most recent version of the repo and install in editable mode in the directory that has ZipStrain: pip install -e . --upgrade. Please let me know how this goes!

[alexanderjaffe]

thank you! i have been using the apptainer image (very useful, by the way!) - will the change be reflected there as well?

[ParsaGhadermazi]

Thanks! Got it! Then this specific change won't be reflected there yet! Those are made automatically on a version change trigger. I will make a minor version change and once you see v0.10.1 give it a try! FYI: With this high coverage if you are doing a comparison maybe it is necessary to increase the default max coverage for building the null model. --max-total-reads argument controls that.

[alexanderjaffe]

i'll keep an eye out!

[ParsaGhadermazi]

I assume this problem is solved, but feel free to open this issue again if it is still there!