Skip to content

HDDS-16175. Do not charge usedNamespace on hsync re-commit - #11092

Open
chihsuan wants to merge 3 commits into
apache:masterfrom
chihsuan:HDDS-16175
Open

HDDS-16175. Do not charge usedNamespace on hsync re-commit#11092
chihsuan wants to merge 3 commits into
apache:masterfrom
chihsuan:HDDS-16175

Conversation

@chihsuan

@chihsuan chihsuan commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

OMKeyCommitRequest currently increments usedNamespace for every commit, including re-commits of a key that the same client already hsync'd. Because the client commits once per newly allocated block and again on close, a file spanning N blocks can leave N namespace units behind after deletion.

This regressed in c351de9 (HDDS-13756), which moved the increment outside the branch where a key name is added and unintentionally reverted the namespace accounting from df4df20 (HDDS-7965).

This change moves incrUsedNamespace(1L) back into the branches that add a key name in OMKeyCommitRequest and OMKeyCommitRequestWithFSO. The overwrite branch still increments the counter because it first refunds the previous key's namespace usage.

Existing incorrect counters are not repaired by this patch and require ozone repair om quota start. During a rolling upgrade, old and new OMs account for new commits differently, as they already do after HDDS-13756.

Bucket versioning is out of scope, as discussed in HDDS-16127.

What is the link to the Apache JIRA?

https://issues.apache.org/jira/browse/HDDS-16175

How was this patch tested?

Added tests covering:

  • Three commits of the same key by one client (two hsync re-commits and close), verifying that usedNamespace remains 1. The test runs for both OBS and FSO commit paths.
  • A multi-block file written through ofs with hsync, then closed and deleted, verifying that usedNamespace returns to 0. This provides cluster-level FSO coverage.

Ran:

mvn -pl :ozone-manager test \
  -Dtest='TestOMKeyCommitRequest,TestOMKeyCommitRequestWithFSO' \
  -DskipShade -DskipRecon -DskipDocs

mvn -pl :ozone-integration-test test \
  -Dtest=TestHSync#testUsedNamespaceWithRepeatedHsync \
  -DskipShade -DskipRecon -DskipDocs

mvn -pl :ozone-manager,:ozone-integration-test checkstyle:check \
  -DskipShade -DskipRecon -DskipDocs

Also ran author.sh successfully.

Generated-by: Claude Code (Opus 5)

incrUsedNamespace was hoisted outside the if/else chain, so the
isSameHsyncKey branch charged a namespace unit on every re-commit of a
key the same client had already hsync'd. Move the increment back into
the branches that actually add a key name.
Copilot AI lite review requested due to automatic review settings August 23, 2026 04:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes OM bucket usedNamespace quota accounting so repeated same-client hsync re-commits for the same key do not consume additional namespace units, preventing namespace drift for multi-block files that hsync per block.

Changes:

  • Move incrUsedNamespace(1L) into the specific commit branches that actually add (or replace) a key entry, in both OBS and FSO commit request paths.
  • Update the stale comment above the commit quota/accounting branch chain to reflect same-client hsync re-commit behavior.
  • Add unit + integration coverage to assert usedNamespace increments once per key creation and returns to baseline after delete/purge.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java Ensures usedNamespace is charged only when the commit adds/replaces a key entry (not on same-client hsync re-commit).
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequestWithFSO.java Mirrors the same usedNamespace accounting fix for the FSO commit path.
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCommitRequest.java Adds unit test asserting repeated hsync commits + final close leave usedNamespace at 1.
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestHSync.java Adds cluster-level test asserting repeated hsync per block increments namespace once and returns to baseline after delete + purge.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@chihsuan
chihsuan marked this pull request as ready for review August 23, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants