refactor(zip): make the archive's read callback re-entrant - #775
Merged
Conversation
`util::Archive` carried a `mutable std::mutex` that every operation took — `is_file`, `path`, `method`, `size`, opening a stream, and each 4 KiB `underflow`. It existed for one reason: the read callback drove a single shared `std::istream` with `seekg`/`read`, so two readers would fight over its position. miniz needs none of that. It keeps decompression state per iterator and addresses the input by absolute offset, so the callback only has to be re-entrant. `ReadSource` makes it so: a memory-backed file is read straight out of its buffer with no lock, and anything else takes a stream from a free list, reads, and puts it back — the lock covers the list, never the read. The archive-wide mutex is gone with it, so the metadata calls no longer serialise either. `mz_zip_archive::m_last_error` stays racy, as miniz documents; we never read it. `concurrent_entry_reads` reads eight 64 KiB entries from eight threads, over a disk file and a memory file. Against a shared unguarded stream it fails hard with `FileNotFound`, so it has teeth.
andiwand
force-pushed
the
refactor/zip-concurrent-reads
branch
from
August 29, 2026 20:48
74c9827 to
e4906f6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 Generated with Claude Code
Stacked on #774 — review that first; the base flips to
mainwhen it merges.util::Archivecarried amutable std::mutexthat every operation took:is_file,path,method,size, opening a stream, and each 4 KiBunderflow. It existed for one reason — the read callback drove a single sharedstd::istreamwithseekg/read, so two readers fought over its position.miniz needs none of that.
mz_zip_reader_extract_iter_newheap-allocates stateper iterator, and all I/O goes through a pread-shaped
m_pRead(opaque, absolute_offset, buf, n), so the callback only has to bere-entrant.
ReadSourcemakes it so:lock covers the list, never the read.
The archive-wide mutex goes with it, so the metadata calls stop serialising too.
mz_zip_archive::m_last_errorstays racy, which miniz documents; we never readit.
Verification
ZipArchive.concurrent_entry_readsreads eight 64 KiB entries from eightthreads, over a
DiskFileand aMemoryFile.FileNotFound: zip entry not found 6, not flakily.odr_testsegfaults at startup onmacOS arm64 with this toolchain, before reaching any test.
Context: #767, which concludes we stay on miniz. This is the concurrency win
that issue wanted; no dependency swap delivers it, and libzip would have made it
worse (its docs tell you to open the archive once per thread).