JITSU-163: klauspost/gzip on the ingest and Mixpanel read paths - #1503
Open
sahiltyagi-jitsu wants to merge 1 commit into
Open
JITSU-163: klauspost/gzip on the ingest and Mixpanel read paths#1503sahiltyagi-jitsu wants to merge 1 commit into
sahiltyagi-jitsu wants to merge 1 commit into
Conversation
Drop-in replacement for compress/gzip on the two decode paths named in JITSU-163 as out of scope for zstd: the s2s batch request body and the Mixpanel partial-failure response. Format-compatible and read-only, so nothing written changes and no consumer is affected. Decode is roughly 33% cheaper per the ticket's benchmarks. Verified identical accept/reject behaviour against compress/gzip across valid, truncated, empty, garbage, multi-member, bad-CRC and bogus-flag inputs — same decisions and same error strings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
absorbb
approved these changes
Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of the small PRs for JITSU-163, covering the half the ticket calls "independent of the zstd work and can ship first".
Swaps
compress/gzipforklauspost/compress/gzipon two decode paths: the s2s batch request body (router_batch_handler.go) and the Mixpanel partial-failure response (mixpanel.go). Both are named in the ticket's swap scope — "the HTTP wire formats below" and "Mixpanel".Readers only
Both sites are reads, so nothing written changes and no consumer is affected. Decode is ~33% cheaper per the ticket's benchmarks. Writers follow separately.
Verification
Behaviour is identical to
compress/gzipacross valid, truncated (half and single-byte), empty, garbage, multi-member, bad-CRC and bogus-flag inputs — same accept/reject decisions and the same error strings, so anything matching on error text is unaffected.Also checked against a live local stack: a batch compressed with GNU
gzip(1)decodes and itswriteKeyis parsed out of the body; corrupt bytes return a clean400 gzip: invalid header. Confirmed the running binary actually linked v1.19.1 rather than the previous v1.18.7, since the build container reports success even when it fails to copy source.gzip_compat_test.gois new and covers the round trip both directions. Not asked for in the ticket — happy to drop it.Dependency
klauspost/compresswas already indirect in both modules; promoted to direct and aligned on v1.19.1 (whatadminalready uses; the ticket benchmarked v1.19.2). Deliberately nogo mod tidy— it ignores the workspace and resolves the sibling modules from the proxy, which rewrites the whole graph.Risk
No format, config, schema or contract change. No deploy ordering; mixed old/new pods are fine. Revert is one import line per file. Not covered: very large bodies under memory pressure.
Blocking the next PR
"All five warehouse adapters" isn't actionable as written — those files contain no gzip calls. The compression lives in
bulkerlib/types/marshaller.goandsql/abstract_transactional.go. Which did you mean?🤖 Generated with Claude Code