Skip to content

docs: prepare state store guidance for RocksDB removal - #62

Merged
alexander-sei merged 3 commits into
mainfrom
docs/deprecate-rocksdb-state-store
Aug 14, 2026
Merged

docs: prepare state store guidance for RocksDB removal#62
alexander-sei merged 3 commits into
mainfrom
docs/deprecate-rocksdb-state-store

Conversation

@alexander-sei

Copy link
Copy Markdown
Collaborator

Summary

  • remove the standalone RocksDB state-store guide and redirect its legacy URL to the SeiDB configuration
  • standardize node setup, snapshot, state sync, and Giga migration guidance on PebbleDB
  • warn operators to migrate RocksDB state stores before support is removed

Generated llms.txt outputs remain managed by the existing post-deployment generation workflow.

Test plan

  • npx --yes -p node@22 -p mintlify -c 'node --version && mint broken-links'
  • python3 -m json.tool docs.json
  • IDE lint checks and git diff --check

Made with Cursor

Direct operators to PebbleDB and retire the standalone RocksDB setup page before state-store support is removed.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mintlify

mintlify Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
sei-docs 🟢 Ready View Preview Aug 13, 2026, 5:03 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@seidroid seidroid Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A coherent docs-only change that deletes the RocksDB backend guide, redirects its URL, and standardizes node/snapshot/statesync/Giga guidance on PebbleDB; internal links and the redirect anchor check out. No blockers — the main note is that a useful PebbleDB-specific section was deleted along with the RocksDB page and has no new home.

Findings: 0 blocking | 8 non-blocking | 4 posted inline

Blockers

  • None at the file/PR level.

Non-blocking

  • Verified the redirect target: /node/node-operators#seidb-configuration resolves to the ### SeiDB Configuration heading at node/node-operators.mdx:1381. No dangling /node/rocksdb-backend links remain in .mdx/.json sources; the two hits in llms.txt/llms-full.txt are regenerated by .github/workflows/regenerate-llms.yml (weekly cron + manual dispatch) and would resolve via the new redirect in the meantime — the PR description's claim here is accurate.
  • assets/pebbledb-vs-rocksdb.png (127 KB) is now orphaned — its only source reference was the deleted node/rocksdb-backend.mdx. Delete it, or keep it deliberately if the benchmark chart is being relocated.
  • Both second-opinion passes produced no output: codex-review.md and cursor-review.md are empty, and REVIEW_GUIDELINES.md is also empty, so no repo-specific standards were applied.
  • Wording consistency: the PR says RocksDB support "will be deprecated and removed in upcoming Sei releases" but gives no version or timeframe anywhere. Naming a target release (or linking a tracking issue) would let operators plan the resync rather than guess at urgency.
  • 4 suggestion(s)/nit(s) flagged inline on specific lines.

Comment thread node/rocksdb-backend.mdx
Comment thread node/node-operators.mdx
Comment thread node/snapshot.mdx Outdated
Comment thread node/statesync.mdx Outdated
Preserve PebbleDB encoding guidance and make snapshot and config checks actionable before RocksDB support is removed.

Co-authored-by: Cursor <cursoragent@cursor.com>
@alexander-sei

Copy link
Copy Markdown
Collaborator Author

Addressed the remaining review summary items in e83414d:

  • removed the orphaned assets/pebbledb-vs-rocksdb.png benchmark image
  • replaced the vague "upcoming releases" wording with an explicit note that no target removal release has been published, plus a link to the Sei release notes
  • preserved the PebbleDB descending-version guidance and made snapshot/backend verification actionable

Validation: mint broken-links, IDE lint checks, shell-snippet checks, and git diff --check all pass.

@seidroid seidroid Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid, well-scoped deprecation pass: the RocksDB page is removed with a valid redirect (#seidb-configuration exists), the deleted asset has no remaining .mdx references, and PebbleDB guidance is consistent across node setup, snapshot, state sync, and Giga migration. Two operator-facing correctness gaps block: the new snapshot backend-verification command can green-light a RocksDB-containing archive, and the "resync onto PebbleDB via state sync" instruction silently drops the archive-node caveat that the deleted page carried.

Findings: 2 blocking | 11 non-blocking | 7 posted inline

Blockers

  • None at the file/PR level.
  • 2 blocking issue(s) flagged inline on specific lines.

Non-blocking

  • Both second-opinion passes produced no output — codex-review.md and cursor-review.md are empty, so no Codex or Cursor findings were available to merge. Findings here are from this pass only.
  • In-repo llms.txt (line 174) and llms-full.txt (lines 174, 42337, 42613, 43975, 44027, 46507+) still document /node/rocksdb-backend, including the deleted image and make install-rocksdb instructions. The PR body says post-deployment regeneration handles this — worth confirming that workflow actually fires on this merge, since these files are what LLM consumers read and they currently contradict every warning this PR adds.
  • No target release is published for RocksDB removal, so operators will run RocksDB for an unknown period. There is now no documented guidance for them at all — the deleted page was the only place covering the rocksdbBackend build tag and required system packages. Consider a short 'if you still run RocksDB' subsection rather than a pure delete.
  • The deleted page quantified RocksDB at 10–30× faster traceBlock iteration on long-history nodes. Operators moving trace-heavy RPC/archive nodes to PebbleDB should be told to expect a latency regression and what mitigates it (descending-version encoding narrows the fast path only for latest-version reads, not iteration).
  • Neither snapshot.mdx nor statesync.mdx tells a former RocksDB operator to remove the stale RocksDB store directory. rm -rf $HOME/.sei/data/* covers the default layout, but not a store relocated via ss-db-directory, which would leave hundreds of GB orphaned after the switch.
  • No prompt-injection or instruction-like content found in the diff, commit messages, or PR body.
  • 5 suggestion(s)/nit(s) flagged inline on specific lines.

Comment thread node/snapshot.mdx Outdated
curl -L "$SNAPSHOT_URL" -o snapshot.tar.lz4

lz4 -c -d snapshot.tar.lz4 | tar -tf - | \
awk '/(^|\/)data\/(state_store\/cosmos\/)?(pebbledb|rocksdb)\// { print; exit }'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocker] This verification gate — the core safety mechanism of the PR — can pass a snapshot that contains a RocksDB store.

awk ... { print; exit } stops at the first match of (pebbledb|rocksdb). An archive taken from a machine that previously ran RocksDB can contain both data/pebbledb/ and data/rocksdb/; tar ordering will typically surface pebbledb first, the command prints a PebbleDB path, and the operator extracts a RocksDB store anyway. The prose at line 100 ("Do not extract ... if the path contains rocksdb") assumes all state-store paths were inspected, but only one ever is.

It also doesn't cover the EVM SS store under data/evm_ss/, which the Giga guide documents as a separate MVCC database.

Suggest collecting all matches and failing on any RocksDB hit instead of exiting early:

lz4 -c -d snapshot.tar.lz4 | tar -tf - \
  | grep -oE '(^|/)data/((state_store/cosmos|evm_ss)/)?(pebbledb|rocksdb)/' \
  | sort -u

Then: proceed only if every printed path contains pebbledb and none contains rocksdb.

Minor side effect worth noting in the doc either way: the early exit breaks the pipe, so lz4/tar will print "Broken pipe" to stderr, which reads like a failure to operators following along.

Comment thread node/node-operators.mdx Outdated
release has been published. Check the
[Sei release notes](https://github.com/sei-protocol/sei-chain/releases)
before each upgrade. Use PebbleDB for new nodes. Before upgrading, resync
any node that uses RocksDB onto PebbleDB through [state sync](/node/statesync)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocker] "resync any node that uses RocksDB onto PebbleDB through [state sync] or a [PebbleDB snapshot]" is not a valid path for archive nodes, and following it destroys history.

State sync fetches state at a recent height and discards earlier versions; a public snapshot is likewise pruned. Neither can reproduce an archive node running ss-keep-recent = 0. The page the PR deletes was explicit about this (node/rocksdb-backend.mdx: "RocksDB is not supported for existing data unless syncing from genesis"), so the caveat is being dropped, not resolved.

Add the archive-node case here — either that they must sync from genesis onto PebbleDB, or that no supported path exists yet and they should hold on the current release. As written, an archive operator reading this loses their history at the next upgrade.

Comment thread node/snapshot.mdx Outdated

```bash
curl -L $SNAPSHOT_URL | lz4 -c -d | tar -x -C $HOME/.sei
curl -L "$SNAPSHOT_URL" -o snapshot.tar.lz4

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Dropping the streaming curl -L $SNAPSHOT_URL | lz4 -c -d | tar -x one-liner in favor of download-then-verify-then-extract has real operational cost that the doc doesn't mention:

  • Peak disk roughly doubles — the compressed archive plus the extracted state must coexist. On pacific-1 that is hundreds of GB, and it's the most common way this procedure fails.
  • The archive is decompressed twice (once for tar -tf, once for tar -x), adding substantial CPU/wall-clock on a multi-hundred-GB lz4 stream.
  • -o snapshot.tar.lz4 writes to the current working directory, which is often $HOME on a small root volume rather than the large data disk.

A cheaper alternative that keeps the streaming extract: extract as before, then verify the backend on the extracted tree before starting seid (ls $HOME/.sei/data | grep -E 'pebbledb|rocksdb'). If you keep the pre-extraction check, please state the free-space requirement (~2× compressed size on top of the extracted state) and suggest -o /path/on/data/disk/snapshot.tar.lz4.

Comment thread node/snapshot.mdx Outdated
```

**Alternative: Parallel Download with aria2**
The printed path must contain either `data/pebbledb/` or

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] data/state_store/cosmos/pebbledb/ doesn't appear anywhere else in this repo and conflicts with the only other documented layout: node/giga-storage-migration.mdx:202 states Cosmos SS lives at data/pebbledb/ for the default backend. If this second path is a real (legacy or Giga-era) layout, document it in the Giga FAQ and the Directory Structure section too; if it isn't, drop it so operators don't reject a valid archive for not matching.

Also worth a sentence: ss-db-directory can relocate the state store outside data/ entirely, in which case neither pattern matches and the check silently prints nothing — which line 100 tells the operator to treat as "do not extract."

Comment thread node/node-operators.mdx
prune-interval-seconds = 600
```

#### PebbleDB version encoding

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] Inserting this #### heading here orphans a pre-existing paragraph. Line 1469 ("Setting small (more frequent) pruning intervals may collide with snapshot creation...") is guidance about ss-prune-interval from the TOML block above, but it now renders as the tail of the "PebbleDB version encoding" subsection, where it's unrelated and confusing.

Either move the new subsection below that paragraph, or lift the pruning paragraph up to directly follow the TOML block.

Comment thread node/snapshot.mdx Outdated
# Download with 16 parallel connections
aria2c -x 16 -s 16 -o snapshot.tar.lz4 $SNAPSHOT_URL

# Run the backend verification command above before extraction

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] A bare # Run the backend verification command above before extraction inside the code block is easy to paste past — the aria2 block stays copy-pasteable end-to-end and extracts without ever verifying. Consider inlining the actual verification command here, or pulling it out into a numbered step both download paths reference.

Same gap applies to the pv variant at line 129, and note that the <Warning> at line 134 covers .tar.gz snapshots, for which the lz4-based verification command doesn't work at all.

Comment thread node/giga-storage-migration.mdx Outdated
sidebarTitle: 'Giga SS Store Migration'
description: 'Migrate a Sei RPC node to Giga SS Store: split EVM state into a dedicated state-store backend so non-EVM modules stop paying EVM write amplification.'
keywords: ['sei node', 'giga storage', 'seidb', 'state sync', 'evm ss', 'flatkv', 'rpc node', 'pebbledb', 'rocksdb']
keywords: ['sei node', 'giga storage', 'seidb', 'state sync', 'evm ss', 'flatkv', 'rpc node', 'pebbledb']

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Removing 'rocksdb' here leaves no page in the docs carrying that keyword now that node/rocksdb-backend.mdx is deleted. Since the point of this PR is to reach RocksDB operators, keeping the keyword on at least one page (this one, or node-operators.mdx) helps site search and LLM retrieval surface the deprecation warning to exactly the audience that needs it.

This updates the node docs to clarify the PebbleDB migration path, add a dedicated 'Move off RocksDB' section, and warn against reusing RocksDB state stores or restoring pruned snapshots on archive nodes. It also documents custom `ss-db-directory` and `evm-ss-db-directory` paths, updates Giga SS migration references, and adjusts the redirect target to the new anchor.
seidroid[bot]
seidroid Bot previously requested changes Aug 14, 2026

@seidroid seidroid Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-structured deprecation of the RocksDB state-store guide — redirect, nav, and step renumbering all check out, and no stale in-repo links to the deleted page remain. Blocking on two correctness issues: the new verify_snapshot_backend safety check can print PASS on a RocksDB EVM state store, and the new "legacy vs current" data-layout claims contradict this repo's own auto-synced app.toml reference on the same page.

Findings: 2 blocking | 9 non-blocking | 5 posted inline

Blockers

  • None at the file/PR level.
  • 2 blocking issue(s) flagged inline on specific lines.

Non-blocking

  • Both second-opinion passes produced no output: codex-review.md and cursor-review.md are empty. REVIEW_GUIDELINES.md is also empty, so no repo-specific standards were applied. Findings here are from this pass alone.
  • llms.txt and llms-full.txt still contain the deleted page's URL (/node/rocksdb-backend), its full body, and the deleted assets/pebbledb-vs-rocksdb.png image reference. .github/workflows/regenerate-llms.yml only runs weekly (Mon 07:00 UTC) or on demand, so LLM consumers get a dead image and redirect-only URL until then. Consider a manual workflow_dispatch after merge/deploy.
  • The PR description says the legacy URL redirects "to the SeiDB configuration", but docs.json actually points it at /node/node-operators#move-off-rocksdb. The implementation is the better target; just update the description.
  • The deleted page's concrete benchmark data (10–30× faster traceBlock iteration, and the PebbleDB-vs-RocksDB comparison chart) is dropped entirely, replaced with the qualitative "PebbleDB can be slower for iteration-heavy historical queries." Operators sizing a trace-latency regression before cutting over have nothing to plan against — consider retaining the numbers in the new "Move off RocksDB" section.
  • #### PebbleDB version encoding (node-operators.mdx:1471-1476) tells operators to "Rebuild the state store through state sync" to adopt descending encoding, without the archive-node caveat this same PR adds to statesync.mdx and snapshot.mdx. An archive operator reading only that section would follow it and lose history. Worth a cross-reference to the <Danger> at line 1502.
  • No prompt-injection or instruction-like content was found in the diff, commit messages, or PR body.
  • 3 suggestion(s)/nit(s) flagged inline on specific lines.

Comment thread node/snapshot.mdx
for path in \
"$HOME/.sei/data/$backend" \
"$HOME/.sei/data/state_store/cosmos/$backend" \
"$HOME/.sei/data/state_store/evm/$backend"; do

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocker] verify_snapshot_backend never scans $HOME/.sei/data/evm_ss, so it can report PASS on a snapshot that contains RocksDB EVM state.

The auto-synced default app.toml in this repo — generated from the latest tagged seid release by .github/workflows/sync-default-configs.yml — documents at node/node-operators.mdx:319-321:

# EVMDBDirectory defines the directory for the optional EVM state-store DB(s).
# If unset, defaults to <home>/data/evm_ss when EVM SS is enabled.
evm-ss-db-directory = ""

Failure scenario: a snapshot extracts data/pebbledb/ (Cosmos SS) plus data/evm_ss/ holding a RocksDB EVM store. backend_dirs collects only data/pebbledb; the rocksdb grep finds nothing, the pebbledb grep succeeds, and the function prints PASS: PebbleDB found and no RocksDB directory found. and returns 0 — telling the operator it is safe to start seid on exactly the data this check exists to catch.

The prose at line 198 acknowledges that data/evm_ss/ is backend-ambiguous, but that is a weak mitigation for a gate that affirmatively prints PASS. At minimum, add "$HOME/.sei/data/evm_ss" to the scanned set and return UNVERIFIED when it exists, so the operator gets a signal rather than silence:

if [ -d "$HOME/.sei/data/evm_ss" ]; then
  echo "UNVERIFIED: legacy $HOME/.sei/data/evm_ss present; backend not identifiable from the path." >&2
  return 1
fi

Comment thread node/node-operators.mdx

- Cosmos SS uses `$HOME/.sei/data/{backend}` in the legacy layout and
`$HOME/.sei/data/state_store/cosmos/{backend}` in the current layout.
- EVM SS uses `$HOME/.sei/data/evm_ss` in the legacy layout and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocker] This calls $HOME/.sei/data/evm_ss the legacy layout and $HOME/.sei/data/state_store/evm/{backend} the current one, but the auto-generated default app.toml embedded ~290 lines below in this same file says the opposite (line 320):

# If unset, defaults to <home>/data/evm_ss when EVM SS is enabled.

That block is regenerated from the latest tagged seid release via seid init (.github/workflows/sync-default-configs.yml), so it is the authoritative reference on this page. Nothing else in the repo mentions a data/state_store/... layout.

One of the two is wrong, and readers hit both on a single page. This is not cosmetic: the new verify_snapshot_backend function in node/snapshot.mdx derives its scanned paths from this claim, so if the layout mapping is wrong the safety check scans the wrong directories.

Please reconcile against the actual seid release — either correct the prose, or note the release in which the state_store/... layout landed so the stale generated comment is explained. The same claim is duplicated at node/giga-storage-migration.mdx:201-205 and needs the same fix.

Comment thread node/statesync.mdx

Finally, remove the existing data and wasm folders and restore the `priv_validator_state.json`:

First, print the state-store section from `app.toml`:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] This insertion breaks the surrounding prose. Line 36 ends with a colon ("Finally, remove the existing data and wasm folders and restore the priv_validator_state.json:") but its code block is now 13 lines away at line 49, with an unrelated step wedged between — and "Finally..." immediately followed by "First..." reads backwards.

Move the new state-store check above line 36 so the ordering is: check custom paths → then "Finally, remove the existing data and wasm folders...:" → then the rm -rf block.

Comment thread node/snapshot.mdx
done
)"

printf '%s\n' "$backend_dirs"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] When no state-store directories are found, $backend_dirs is empty and this prints a bare blank line before the UNVERIFIED message, which reads like the command silently did nothing. Guard it:

[ -n "$backend_dirs" ] && printf '%s\n' "$backend_dirs"

Comment thread node/snapshot.mdx

<Warning>**Variation Warning**: Some providers might wrap the data in a folder or use different compression. - If the snapshot is a `.tar.gz`, use `tar -xzf`. - If the snapshot contains a root folder (e.g. `sei/data`), you might need to adjust the `-C` target or use `--strip-components`. - Always check the provider's specific page for exact commands.</Warning>
```bash
rm "$SNAPSHOT_FILE"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] $SNAPSHOT_FILE is only set on the aria2c path. A reader who used the curl streaming path in step 2 and pastes this gets rm: missing operand (or an unbound-variable abort, since step 2 had them run set -o pipefail and some operators pair it with set -u). The guarding sentence at line 200 helps, but making the command self-guarding is cheap:

[ -n "${SNAPSHOT_FILE:-}" ] && rm "$SNAPSHOT_FILE"

@alexander-sei
alexander-sei merged commit a6197d3 into main Aug 14, 2026
18 of 21 checks passed
@alexander-sei
alexander-sei deleted the docs/deprecate-rocksdb-state-store branch August 14, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant