Skip to content

feat(token): derive last-used per token for revocation triage (#622) - #633

Merged
padak merged 4 commits into
mainfrom
claude/issue-622-exploration-b15e87
Aug 21, 2026
Merged

feat(token): derive last-used per token for revocation triage (#622)#633
padak merged 4 commits into
mainfrom
claude/issue-622-exploration-b15e87

Conversation

@padak

@padak padak commented Aug 21, 2026

Copy link
Copy Markdown
Member

Closes #622.

Why

kbagent token list could say what tokens exist but not which are still in use. On a shared project with 25 tokens — per-user master tokens, MCP tokens, kbagent device tokens, [_internal] Token for triggering … orchestration tokens — nothing distinguished one used four minutes ago from one last used five months ago. So the questions you actually want answered before revoking anything ("which are dormant?", "which were minted and never used?", "who is still hitting this project?") required the web UI, one token at a time. In practice nobody does that, and the token set only ever grows.

The gap is not a CLI omission: Storage API token payloads genuinely carry no lastUsed field — only the Manage API's PAT response does.

What

token list --with-last-used derives it per token from GET /v2/storage/tokens/{id}/events, fanned out in parallel over the existing max_parallel_workers pool. Adds lastUsed / lastUsedEvent / lastUsedStatus per token plus a top-level errors, and sorts dormant-first so reading order is cleanup order.

Opt-in by design — it is one extra request per token, and a plain token list that only wants an id for token delete must stay cheap. Without the flag the response shape is byte-for-byte what it was before (asserted in both unit and E2E tests).

Also: --columns (repeatable, human output only) selects and orders the table, and Refreshed joins the default columns — --json already returned it but the fixed 6-column table had no way to show it. --json is deliberately not affected by --columns; the machine contract stays whole.

The correctness bit that matters

The issue proposed reading events[0] and filtering client-side on objectId. Reading Connection's source (Storage\Events\EventsSearchQueryGenerator::getTokenEventsSearchQuery) shows the endpoint ORs two groups:

(objectId == {id} AND objectType == 'token')   -- events ABOUT the token
OR token.id == {id}                            -- events BY the token

Only the second is evidence of use. This PR narrows to it server-side with q=token.id:{id} — the query form Connection's own E2E suite uses (EventTesterUtils). That avoids two distinct wrong answers:

  1. Naive events[0] — a freshly minted token's newest raw event is its own storage.tokenCreated, so it reports never-used as "used today", exactly backwards from what an audit wants.
  2. Client-side filtering — right after an admin rotates a token, the single event a limit=1 fetch returns is that rotation, leaving the filter with nothing and calling an actively-used token unused.

Verified live on connection.us-east4.gcp against a token created today and never used:

RAW feed (no q=):
   2026-08-21T23:45:50+0200  storage.tokenCreated  objectId=6135776 objectType=token performedBy=5815875
   -> naive events[0] would report lastUsed = 2026-08-21T23:45:50+0200   (WRONG)

NARROWED feed (q=token.id:6135776):
   []  -> never used                                                     (CORRECT)

Worth noting explicitly: the service honours q=. That is not a given — the Notification Service accepts its documented ?event= and silently ignores it (#600), which is why this was checked rather than assumed.

never vs unknown are not the same answer

Events are retained ~6 months, so an empty feed has two meanings. lastUsedStatus keeps them apart by comparing against the token's own creation date:

status meaning
used lastUsed is a real timestamp
never proven never used — minted inside the retention window, no activity
unknown older than retention, so the API genuinely cannot say
error that token's lookup failed; the row degrades, the audit still completes

Collapsing never/unknown would be a confident lie about exactly the population someone is about to revoke. Live example from the E2E Azure project: a colleague's token created 2023-03-09 (1261 days old) with an empty feed correctly reports unknown, not never.

Known limitation (documented, not fixable here)

GET /v2/storage/tokens/{id}/events is not branch-addressable (isAvailableInBranch: false) and always resolves to the default branch, narrowing to idBranch == <production> OR NOT EXISTS idBranch. Activity inside a development branch is therefore invisible, and a branch-only token reads as dormant. Called out in gotchas.md, CLAUDE.md, AGENT_CONTEXT and the command's own help so nobody revokes a live branch token on its say-so.

Upstream bug found while investigating (not fixed here)

sortOrder=garbage on this endpoint returns 500, not 400. Root cause is one line: in TokenEventsListAction::__invoke, EventsFilter::fromArray() is called outside the try block whose catch (EventsException) maps to HttpException(400) — and the action already declares OA\Response(400). Relatedly limit=abc casts to 0 and returns an empty feed (not the default), which a naive caller would read as "never used". Happy to file this with the Storage API team separately.

Scope

Layer File
Client client/tokens.pylist_token_events()
Derivation services/_token_last_used.py — shared by CLI and SDK
Service services/token_service.py — fan-out (now extends BaseService)
Command commands/token.py — flags, column registry
SDK lib.py, result_models.pyClient.list_tokens(with_last_used=True)
REST server/routers/token.py?with_last_used=

Version renumbered to 0.88.0; 0.87.0 went to the data-app workspace flag (#626) while this was in flight.

Testing

  • make check green: 5724 passed, 12 skipped.
  • Unit tests written first, per layer: never / unknown / used / error, no-extra-call-without-flag, dormant-first ordering, column selection, unknown column → exit 2, --json unaffected by --columns.
  • E2E: the token create → list → refresh → delete lifecycle gains a step asserting a just-minted token reads as never — the exact case a naive implementation gets backwards.
  • Live-verified on ~/kbagent/e2e across two stacks (GCP us-east4, Azure north-europe), including the raw-vs-narrowed comparison above.

plugins/kbagent/agents/keboola-expert.md deliberately untouched: it sits at 61999 B against its hard 62000 B cap, and per CONTRIBUTING adding to an existing command group needs no new tool-matrix row (same call as #598). The knowledge went to gotchas.md. The owed trim is tracked separately.


Open in Devin Review

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment thread src/keboola_agent_cli/commands/token.py

@padak padak left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of #633 — feat(token): derive last-used per token for revocation triage (#622)

Generated by kbagent-pr-reviewer subagent. Verdict and findings below
are advisory; the human author retains every veto. CI-coverable issues
(lint, format, tests) are confirmed via make check, not duplicated here.

Summary

This PR adds kbagent token list --with-last-used, deriving each Storage token's most recent activity from its per-token event feed (GET /v2/storage/tokens/{id}/events, narrowed server-side to q=token.id:{id}) so a shared project's token set can be triaged for revocation. It also adds --columns for the human table and threads the same capability through the SDK facade (Client.list_tokens(with_last_used=True)) and kbagent serve's /token/{project}/list route. The implementation is careful and well-reasoned: it correctly distinguishes "never used" from "unknown" (retention-window comparison against created), documents the dev-branch blind spot, degrades individual per-token failures without aborting the audit, and keeps the default response byte-for-byte unchanged (verified by tests). Documentation sync is thorough across CLAUDE.md, commands/context.py, commands-reference.md, and gotchas.md (correctly version-tagged); the PR explicitly and correctly justifies skipping a new keboola-expert.md tool-matrix row (existing group, file at 61960/62000 B). Verdict: COMMENT — no blocking findings; one non-blocking documentation gap and a minor test-coverage note.

Verdict

  • Verdict: COMMENT
  • Blocking findings: 0
  • Non-blocking findings: 2
  • Nits: 1

Blocking findings

(none)

Non-blocking findings

[NB-1] docs/sdk.md:181,261 — SDK docs not updated for the new with_last_used capability

lib.py's Client.list_tokens gained a with_last_used: bool = False keyword argument and result_models.py's TokenListEntryResult gained three new fields (last_used, last_used_event, last_used_status), but docs/sdk.md still describes the pre-PR shape at both the method-reference line (181) and the model-reference line (261). CONTRIBUTING.md's "Extending the importable SDK" checklist calls for documenting SDK additions in docs/sdk.md. Low risk (the fields are additive/optional, so nothing breaks), but an SDK consumer reading the docs would not discover the new parameter or fields exist. Fix: add a sentence to both reference lines mirroring the CLI's --with-last-used note.

[NB-2] src/keboola_agent_cli/server/routers/token.py:37-44 — no server-layer test for the new with_last_used query parameter

The route is a one-line passthrough (registry.token.list_tokens(alias=project, with_last_used=with_last_used)), and the underlying service is thoroughly unit-tested, so risk is low. Still, no test in the repo (tests/test_server_smoke.py or similar) asserts that GET /token/{project}/list?with_last_used=true actually forwards the flag through the FastAPI route — a future refactor of the router signature (e.g. renaming the query param) would not be caught by any test. Consider adding one thin TestClient assertion alongside the existing /token/{project}/create smoke coverage.

Nits

  • [NIT-1] src/keboola_agent_cli/services/_token_last_used.py:741dormancy_rank(...) -> tuple[int, float] is a new tuple-returning function. It's a conventional Python sort-key idiom (never destructured by name at a call site, only fed to .sort(key=...)), so it doesn't really trigger the "ambiguous positional return" problem the dataclass rule targets — flagging only because it's technically a new -> tuple[...] annotation the grep step surfaces; no action needed unless the author wants a named @dataclass(order=True) for symmetry with the rest of the module.

Verification log

  • gh auth status → authenticated as padak, scopes include repo/workflow
  • gh pr view 633 --json title,body,files,... → OPEN, 22 files, +1112/-59, feat(token): prefix matches new-feature scope ✓
  • git rev-parse --abbrev-ref HEADclaude/issue-622-exploration-b15e87 matches <branch>, working tree clean ✓
  • gh pr diff 633 → 1537-line diff, read in full ✓
  • Layer-violation greps (typer/formatter in services, httpx in commands, formatter/typer in clients) → all empty ✓ no violations
  • Magic-number / raw error-code-string / bare-except / print() / new-tuple-return / token-leakage greps → clean; the only "token" hits are the feature name itself, not secret values ✓
  • grep -nE '"token\.(list|create|delete|refresh)"' permissions.py → all four already registered (token.list: read, token.create: write, token.refresh: write, token.delete: destructive); this PR adds flags to an existing command, not a new command, so no new OPERATION_REGISTRY entry is needed ✓
  • commands/context.py, CLAUDE.md, commands-reference.md, gotchas.md all updated with matching (0.88.0+) / (since v0.88.0) tags ✓
  • wc -c plugins/kbagent/agents/keboola-expert.md → 61960 bytes (< 62000 cap); PR description's stated reason for skipping a new tool-matrix row (existing group, budget nearly exhausted) checks out ✓
  • pyproject.toml, plugin.json, marketplace.json all show 0.88.0 — version-sync consistent ✓
  • grep DEFAULT_MAX_PARALLEL_WORKERS / models.py max_parallel_workers default=10 → constant correctly mirrors the config default ✓
  • grep _resolve_max_workers services/base.py → confirmed BaseService helper exists and is reused (no duplicated worker-pool logic) ✓
  • make check5724 passed, 12 skipped, exit code 0 (lint + format + typecheck + skill-check + version-check + command-sync-check + changelog-check + check-error-codes + check-sentinel-guards + loc-check + full test suite, per Makefile's check target) ✓
  • grep list_tokens docs/sdk.md → stale, does not mention with_last_used or the three new result fields (see NB-1)
  • grep -rln "routers.token\|registry.token" tests/ → no dedicated server-router test exercises the new query param (see NB-2)
  • Did not independently re-run the live API reproduction (minting a token and asserting lastUsedStatus == "never") — the PR description already documents a live A/B (raw feed vs. q=token.id: narrowed feed) against a real GCP-stack token, and the described behavior matches the code exactly (derive_last_used in _token_last_used.py) and is covered by both tests/test_token_service.py::TestListTokensWithLastUsed and the updated tests/test_e2e.py scoped-token lifecycle test. Per repo convention, AI agents do not handle real API tokens directly, so this was verified by code/test reading rather than a fresh live call.

Open questions for the author

(none)

padak added a commit that referenced this pull request Aug 21, 2026
…review)

Selecting a derived column with no derivation behind it rendered every
row as "unknown (older than event retention)" -- `_last_used_cell` fell
through to its last branch because no row carried `lastUsedStatus`. That
is not merely an empty column: it is a definitive statement about event
retention for tokens nobody looked up. Reproduced live, where a token
proven `used` two minutes earlier reported as unknown.

Fail fast with exit 2 naming the missing flag rather than rendering the
column blank: someone who typed `--columns last_used` wants that data,
and a blank cell invites the equally wrong reading "this token has no
recorded activity". The cell renderer also now returns an empty string
for a row with no status at all, so it can never invent a verdict even
if one reaches it.

Also from review:
- docs/sdk.md documented `list_tokens()` and `TokenListEntryResult`
  without the new parameter or the three new fields.
- Add server-router tests asserting `?with_last_used=` reaches the
  service and defaults to False, so a future refactor cannot silently
  drop the kwarg (or silently opt every REST caller into the N+1).
padak added 4 commits August 22, 2026 00:16
`token list` could say what tokens EXIST but not which are still in use.
On a shared project with 25 tokens -- per-user master tokens, MCP tokens,
device tokens, `[_internal]` orchestration tokens -- nothing distinguished
one used four minutes ago from one last used five months ago, so nothing
could be revoked responsibly and the set only ever grew.

Storage token payloads carry no `lastUsed` (only the Manage API's PAT
response does), so `--with-last-used` derives it per token from
`GET /v2/storage/tokens/{id}/events`, fanned out over the existing
`max_parallel_workers` pool. Opt-in: one extra request per token, and a
plain `token list` that only wants an id must stay cheap. Rows sort
dormant-first so reading order is cleanup order.

The feed is narrowed SERVER-SIDE to `q=token.id:{id}` -- events the token
performed. Connection ORs in events performed ON the token
(EventsSearchQueryGenerator::getTokenEventsSearchQuery), so a fresh
token's newest raw event is its own `storage.tokenCreated` and reading
`events[0]` reports never-used as "used today". Filtering client-side
instead fails differently: right after a rotation the one event a
`limit=1` fetch returns is that rotation, leaving the filter with nothing
and calling an active token unused.

`lastUsedStatus` keeps `never` (minted inside the ~6-month retention
window with no activity, so provable) apart from `unknown` (older than
retention, unanswerable) and `error` (per-token lookup failed; the row
degrades, the audit completes). Collapsing never/unknown would be a
confident lie about exactly the tokens someone is about to revoke.

Known limit, documented in gotchas: the endpoint is not branch-addressable
and always resolves to the default branch, so activity inside a dev branch
is invisible and a branch-only token reads as dormant.

Also adds `--columns` (repeatable, human output only) and `Refreshed` to
the default table; `--json` already returned `refreshed` but the fixed
6-column table had no way to show it.

SDK: `Client.list_tokens(with_last_used=True)`, three new fields on
`TokenListEntryResult`. REST: `with_last_used` on `GET /token/{p}/list`.
…review)

Selecting a derived column with no derivation behind it rendered every
row as "unknown (older than event retention)" -- `_last_used_cell` fell
through to its last branch because no row carried `lastUsedStatus`. That
is not merely an empty column: it is a definitive statement about event
retention for tokens nobody looked up. Reproduced live, where a token
proven `used` two minutes earlier reported as unknown.

Fail fast with exit 2 naming the missing flag rather than rendering the
column blank: someone who typed `--columns last_used` wants that data,
and a blank cell invites the equally wrong reading "this token has no
recorded activity". The cell renderer also now returns an empty string
for a row with no status at all, so it can never invent a verdict even
if one reaches it.

Also from review:
- docs/sdk.md documented `list_tokens()` and `TokenListEntryResult`
  without the new parameter or the three new fields.
- Add server-router tests asserting `?with_last_used=` reaches the
  service and defaults to False, so a future refactor cannot silently
  drop the kwarg (or silently opt every REST caller into the N+1).
@padak
padak force-pushed the claude/issue-622-exploration-b15e87 branch from 54b9746 to c826c38 Compare August 21, 2026 22:17
@padak
padak merged commit 430cab2 into main Aug 21, 2026
4 checks passed
@padak
padak deleted the claude/issue-622-exploration-b15e87 branch August 21, 2026 22:28
padak added a commit that referenced this pull request Aug 22, 2026
Nothing between v0.87.0 and today was ever published: #629 bumped
pyproject to 0.88.0, #633 bumped it again to 0.89.0, and both changelog
blocks sat unreleased. Ship the whole span as one release instead of
publishing two versions retroactively.

- changelog: merge the 0.89.0 bullets into 0.88.0. Reordered so the
  #624 column-description fix leads -- it is the largest user-visible
  change of the span, and the first bullet is what `kbagent changelog`
  renders as the one-line summary. The internal prompt-budget note moves
  to the end.
- pyproject 0.89.0 -> 0.88.0, propagated to plugin.json,
  marketplace.json and uv.lock via `make version-sync`.
- rewrite every `0.89.0+` / `(since v0.89.0)` version gate in CLAUDE.md,
  the keboola-expert prompt, commands-reference.md, gotchas.md,
  context.py, docs/sdk.md and two test comments. A stale gate would make
  the agent refuse flags that do exist on the user's installed version.
padak added a commit that referenced this pull request Aug 22, 2026
Nothing between v0.87.0 and today was ever published: #629 bumped
pyproject to 0.88.0, #633 bumped it again to 0.89.0, and both changelog
blocks sat unreleased. Ship the whole span as one release instead of
publishing two versions retroactively.

- changelog: merge the 0.89.0 bullets into 0.88.0. Reordered so the
  #624 column-description fix leads -- it is the largest user-visible
  change of the span, and the first bullet is what `kbagent changelog`
  renders as the one-line summary. The internal prompt-budget note moves
  to the end.
- pyproject 0.89.0 -> 0.88.0, propagated to plugin.json,
  marketplace.json and uv.lock via `make version-sync`.
- rewrite every `0.89.0+` / `(since v0.89.0)` version gate in CLAUDE.md,
  the keboola-expert prompt, commands-reference.md, gotchas.md,
  context.py, docs/sdk.md and two test comments. A stale gate would make
  the agent refuse flags that do exist on the user's installed version.
padak added a commit that referenced this pull request Aug 22, 2026
…eport fix (#638)

Nothing between v0.87.0 and today had been published: #629 bumped pyproject to
0.88.0, #633 bumped it again to 0.89.0, and both changelog blocks sat unreleased.
Ship the whole span as one release instead of publishing two versions
retroactively.

- changelog: merge the 0.89.0 bullets into 0.88.0, reordered so the #624
  column-description fix leads -- the first bullet is what `kbagent changelog`
  renders as the one-line summary. Add (#issue) decorations to all 24 bullets;
  the `SDK:` prefix is not in \_PREFIX_RE so that bullet rendered uncoloured and
  becomes `New (#622):`. Add entries for #636 and #637, which carried no version
  bump and so arrived with no changelog at all.
- pyproject 0.89.0 -> 0.88.0, propagated to plugin.json, marketplace.json and
  uv.lock via `make version-sync`.
- rewrite every `0.89.0+` / `(since v0.89.0)` version gate across CLAUDE.md, the
  keboola-expert prompt, commands-reference.md, gotchas.md, context.py,
  docs/sdk.md and two test comments. A stale gate makes the agent refuse flags
  that do exist on the user's installed version.
- fix: `kbagent version --json` no longer advertises an `upgrade_command` when
  there is nothing to upgrade to. It was built unconditionally, so a caller on a
  pre-release read `up_to_date: true` beside a `--force --reinstall` command
  pinned to the OLDER stable wheel (a silent downgrade), and an unreachable
  release feed produced an unpinned `git+` default-branch install. `kbagent
  update` itself was never affected. Two tests added; both fail on the parent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

token list: no last-used signal; derivable from /v2/storage/tokens/{id}/events but no CLI path

1 participant