HDDS-16175. Do not charge usedNamespace on hsync re-commit - #11092
Open
chihsuan wants to merge 3 commits into
Open
HDDS-16175. Do not charge usedNamespace on hsync re-commit#11092chihsuan wants to merge 3 commits into
chihsuan wants to merge 3 commits into
Conversation
incrUsedNamespace was hoisted outside the if/else chain, so the isSameHsyncKey branch charged a namespace unit on every re-commit of a key the same client had already hsync'd. Move the increment back into the branches that actually add a key name.
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes OM bucket usedNamespace quota accounting so repeated same-client hsync re-commits for the same key do not consume additional namespace units, preventing namespace drift for multi-block files that hsync per block.
Changes:
- Move
incrUsedNamespace(1L)into the specific commit branches that actually add (or replace) a key entry, in both OBS and FSO commit request paths. - Update the stale comment above the commit quota/accounting branch chain to reflect same-client
hsyncre-commit behavior. - Add unit + integration coverage to assert
usedNamespaceincrements once per key creation and returns to baseline after delete/purge.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java | Ensures usedNamespace is charged only when the commit adds/replaces a key entry (not on same-client hsync re-commit). |
| hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequestWithFSO.java | Mirrors the same usedNamespace accounting fix for the FSO commit path. |
| hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCommitRequest.java | Adds unit test asserting repeated hsync commits + final close leave usedNamespace at 1. |
| hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestHSync.java | Adds cluster-level test asserting repeated hsync per block increments namespace once and returns to baseline after delete + purge. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
chihsuan
marked this pull request as ready for review
August 23, 2026 04:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
OMKeyCommitRequestcurrently incrementsusedNamespacefor every commit, including re-commits of a key that the same client already hsync'd. Because the client commits once per newly allocated block and again on close, a file spanning N blocks can leave N namespace units behind after deletion.This regressed in c351de9 (HDDS-13756), which moved the increment outside the branch where a key name is added and unintentionally reverted the namespace accounting from df4df20 (HDDS-7965).
This change moves
incrUsedNamespace(1L)back into the branches that add a key name inOMKeyCommitRequestandOMKeyCommitRequestWithFSO. The overwrite branch still increments the counter because it first refunds the previous key's namespace usage.Existing incorrect counters are not repaired by this patch and require
ozone repair om quota start. During a rolling upgrade, old and new OMs account for new commits differently, as they already do after HDDS-13756.Bucket versioning is out of scope, as discussed in HDDS-16127.
What is the link to the Apache JIRA?
https://issues.apache.org/jira/browse/HDDS-16175
How was this patch tested?
Added tests covering:
usedNamespaceremains 1. The test runs for both OBS and FSO commit paths.usedNamespacereturns to 0. This provides cluster-level FSO coverage.Ran:
Also ran
author.shsuccessfully.Generated-by: Claude Code (Opus 5)