Skip to content

Normalize nested DST size formulas - #3659

Open
joshlf wants to merge 1 commit into
mainfrom
Ger2cetjqc3naucboffw4r4hal423gaa5
Open

Normalize nested DST size formulas#3659
joshlf wants to merge 1 commit into
mainfrom
Ger2cetjqc3naucboffw4r4hal423gaa5

Conversation

@joshlf

@joshlf joshlf commented Sep 7, 2026

Copy link
Copy Markdown
Member

DstLayout flattened a nested slice DST's trailing-padding operation into the alignment of its packed outer container. That could understate the Rust object size, causing safe parsers and allocators to construct references and boxes whose referents extended beyond their backing storage.

Represent slice-DST sizes as C + round_up(r + nE, A), encoding A and r in one nonzero word. Carry that four-word layout formula through composition, padding, metadata inference, casts, projections, allocation, and split overlap checks.

This is a smaller state-model alternative to #3630. Exhaustive reduced-width model checking matches #3630's core formula transforms and representability behavior, and the checked benchmark snapshots are byte-for-byte identical.

Add regression coverage for compiler-accepted generic packed/aligned nesting, parsing, allocation, zeroing, projection, metadata plateaus, and dynamic padding. Add modular Kani proofs for the normal form, transforms, inverse validator, affine cast arithmetic, alignment, containment, the conservative dynamic-padding predicate, and both modern and MSRV unchecked-arithmetic implementations. The arithmetic proofs are conditional on the documented compiler-layout premise; regression and Miri coverage exercise the actual compiler layout and pointer paths.

Closes #3617

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.


Latest Update: v3 — Compare vs v2

📚 Full Patch History

Links show the diff between the row version and the column version.

Version v2 v1 Base
v3 vs v2 vs v1 vs Base
v2 vs v1 vs Base
v1 vs Base
⬇️ Download this PR

Branch

git fetch origin refs/heads/Ger2cetjqc3naucboffw4r4hal423gaa5 && git checkout -b pr-Ger2cetjqc3naucboffw4r4hal423gaa5 FETCH_HEAD

Checkout

git fetch origin refs/heads/Ger2cetjqc3naucboffw4r4hal423gaa5 && git checkout FETCH_HEAD

Cherry Pick

git fetch origin refs/heads/Ger2cetjqc3naucboffw4r4hal423gaa5 && git cherry-pick FETCH_HEAD

Pull

git pull origin refs/heads/Ger2cetjqc3naucboffw4r4hal423gaa5

Stacked PRs enabled by GHerrit.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-07T17:05:48.151412Z d2049eb Manual request
🔒 Security Review Completed 2026-09-07T17:07:52.756068Z d2049eb Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@joshlf

joshlf commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

@codex review

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.

@codecov-commenter

codecov-commenter commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.42857% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 91.52%. Comparing base (52869eb) to head (d2049eb).

Files with missing lines Patch % Lines
zerocopy/src/util/mod.rs 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3659      +/-   ##
==========================================
- Coverage   91.86%   91.52%   -0.34%     
==========================================
  Files          20       20              
  Lines        6097     6597     +500     
==========================================
+ Hits         5601     6038     +437     
- Misses        496      559      +63     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joshlf

joshlf commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

Local validation on 2310d3b35559bd516c6cae7ed05383008f92c94c:

  • Kani 0.60: 39/39 harnesses verified, 0 failures, 956.864 seconds aggregate solver time. The run used function contracts, randomized layout, memory-safety, overflow, undefined-function, and unwind checks, with no per-harness timeout or reduced unwind bound.
  • Miri: the three nested packed-DST regressions passed under both Stacked Borrows and Tree Borrows with strict provenance and randomized layout.
  • Nightly all-feature tests, MSRV/stable/nightly checks, strict workspace and codegen Clippy, formatting, generated-code checks, and the repository pre-push hook all passed.

The Kani proofs establish the layout algebra under their stated valid-layout and representability assumptions. They do not prove rustc's documented compatibility premise for compiler-accepted transitive packed/aligned nesting or exercise pointer projection end-to-end; the compiler regressions and Miri runs cover those paths.

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2310d3b355

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread zerocopy/src/split_at.rs Outdated
pub fn via_unaligned(self) -> (&'a T, &'a [T::Elem])
where
T: Unaligned,
T::Elem: Unaligned,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the existing T: Unaligned caller contract

Downstream generic functions constrained by the previously documented T: SplitAt + Unaligned bound can no longer call via_unaligned, because Rust cannot infer T::Elem: Unaligned from T: Unaligned; the mirrored mutable overload has the same regression. This creates an unnecessary source-breaking API change even though via_no_dynamic_padding already provides the new bound-free path. Keep T: Unaligned on both existing via_unaligned methods and direct callers needing the broader layout-based behavior to the new method.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in d2049eb1263dc67f38d4b4d629f235ba8f9df314: restored the existing T: Unaligned bound on both public methods and the internal forwarding path, while retaining the monomorphization-time no-dynamic-padding assertion that supplies the soundness proof. via_no_dynamic_padding remains the broader API for types without that bound. I also added shared and mutable generic regression coverage using only T: SplitAt + Unaligned.

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 2310d3b355

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

DstLayout flattened a nested slice DST's trailing-padding operation into the alignment of its packed outer container. That could understate the Rust object size, causing safe parsers and allocators to construct references and boxes whose referents extended beyond their backing storage.

Represent slice-DST sizes as C + round_up(r + nE, A), encoding A and r in one nonzero word. Carry that four-word layout formula through composition, padding, metadata inference, casts, projections, allocation, and split overlap checks.

This is a smaller state-model alternative to #3630. Exhaustive reduced-width model checking matches #3630's core formula transforms and representability behavior, and the checked benchmark snapshots are byte-for-byte identical.

Add regression coverage for compiler-accepted generic packed/aligned nesting, parsing, allocation, zeroing, projection, metadata plateaus, and dynamic padding. Add modular Kani proofs for the normal form, transforms, inverse validator, affine cast arithmetic, alignment, containment, the conservative dynamic-padding predicate, and both modern and MSRV unchecked-arithmetic implementations. The arithmetic proofs are conditional on the documented compiler-layout premise; regression and Miri coverage exercise the actual compiler layout and pointer paths.

Closes #3617

*Authored by an AI agent acting on Josh Liebow-Feeser's behalf.*

gherrit-pr-id: Ger2cetjqc3naucboffw4r4hal423gaa5
@joshlf
joshlf force-pushed the Ger2cetjqc3naucboffw4r4hal423gaa5 branch from 2310d3b to d2049eb Compare September 7, 2026 16:57
@joshlf

joshlf commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

@codex review

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: d2049eb126

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: d2049eb126

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@joshlf

joshlf commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

Validation on d2049eb1263dc67f38d4b4d629f235ba8f9df314 after addressing the via_unaligned compatibility review:

  • Kani 0.60: all 39 harnesses passed (8,788 property checks; 0 unexpected failures) using function contracts, randomized layout, memory-safety, overflow, undefined-function, and unwind checks, with no reduced unwind bound.
  • The full nightly all-feature unit/UI/doctest suite, MSRV 1.56 and stable 1.93.1 checks, strict nightly Clippy, codegen snapshots, and formatting passed locally.
  • GitHub CI is terminal with 99 successful checks, 6 intentional skips, and no failures.
  • The commit-specific Codex code and security reviews found no further issues.

The Kani proofs establish the layout algebra under their documented valid-layout and representability assumptions. They do not prove rustc's compatibility premise for compiler-accepted transitive packed/aligned generic nesting; regression tests and Miri exercise the compiler layout and pointer paths.

Authored by an AI agent acting on Josh Liebow-Feeser's behalf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Security Advisory: Nested packed DST layouts underallocate memory and permit out-of-bounds access

2 participants