Skip to content

JITSU-163: klauspost/gzip on the ingest and Mixpanel read paths - #1503

Open
sahiltyagi-jitsu wants to merge 1 commit into
newjitsufrom
feat/jitsu-163-klauspost-gzip-readers
Open

JITSU-163: klauspost/gzip on the ingest and Mixpanel read paths#1503
sahiltyagi-jitsu wants to merge 1 commit into
newjitsufrom
feat/jitsu-163-klauspost-gzip-readers

Conversation

@sahiltyagi-jitsu

Copy link
Copy Markdown

First of the small PRs for JITSU-163, covering the half the ticket calls "independent of the zstd work and can ship first".

Swaps compress/gzip for klauspost/compress/gzip on two decode paths: the s2s batch request body (router_batch_handler.go) and the Mixpanel partial-failure response (mixpanel.go). Both are named in the ticket's swap scope — "the HTTP wire formats below" and "Mixpanel".

Readers only

Both sites are reads, so nothing written changes and no consumer is affected. Decode is ~33% cheaper per the ticket's benchmarks. Writers follow separately.

Verification

Behaviour is identical to compress/gzip across valid, truncated (half and single-byte), empty, garbage, multi-member, bad-CRC and bogus-flag inputs — same accept/reject decisions and the same error strings, so anything matching on error text is unaffected.

Also checked against a live local stack: a batch compressed with GNU gzip(1) decodes and its writeKey is parsed out of the body; corrupt bytes return a clean 400 gzip: invalid header. Confirmed the running binary actually linked v1.19.1 rather than the previous v1.18.7, since the build container reports success even when it fails to copy source.

gzip_compat_test.go is new and covers the round trip both directions. Not asked for in the ticket — happy to drop it.

Dependency

klauspost/compress was already indirect in both modules; promoted to direct and aligned on v1.19.1 (what admin already uses; the ticket benchmarked v1.19.2). Deliberately no go mod tidy — it ignores the workspace and resolves the sibling modules from the proxy, which rewrites the whole graph.

Risk

No format, config, schema or contract change. No deploy ordering; mixed old/new pods are fine. Revert is one import line per file. Not covered: very large bodies under memory pressure.

Blocking the next PR

"All five warehouse adapters" isn't actionable as written — those files contain no gzip calls. The compression lives in bulkerlib/types/marshaller.go and sql/abstract_transactional.go. Which did you mean?

🤖 Generated with Claude Code

Drop-in replacement for compress/gzip on the two decode paths named in
JITSU-163 as out of scope for zstd: the s2s batch request body and the
Mixpanel partial-failure response.

Format-compatible and read-only, so nothing written changes and no
consumer is affected. Decode is roughly 33% cheaper per the ticket's
benchmarks.

Verified identical accept/reject behaviour against compress/gzip across
valid, truncated, empty, garbage, multi-member, bad-CRC and bogus-flag
inputs — same decisions and same error strings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@jitsu-code-review jitsu-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the gzip reader substitution in the ingest batch and Mixpanel error-reporting paths, along with the module updates and compatibility coverage.

No actionable correctness, security, or user-visible regression found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants