Skip to content

feat(chat): hand a private room over, or purge it, when its owner closes - #129

Merged
zaxovaiko merged 10 commits into
devfrom
feat/chat-room-ownership-handover
Sep 4, 2026
Merged

feat(chat): hand a private room over, or purge it, when its owner closes#129
zaxovaiko merged 10 commits into
devfrom
feat/chat-room-ownership-handover

Conversation

@mp-blurify

@mp-blurify mp-blurify commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

A private chat room now survives its owner's account being closed: ownership
passes to the longest-serving moderator, or the room enters a 30-day countdown and is
then permanently deleted along with its messages. Adds the player.account.closed
trigger event, two chat_room/chat_room_member columns, a chat-room-purge cron, and
in-app notifications for the inheriting owner and the affected members.

Why

Closing a player's account stranded every private room they owned. chatRoom.creatorId
kept pointing at the dead account, and since updatePrivateRoom and deletePrivateRoom
both assert ownership, nobody could rename the room, delete it, or take it over — it sat
in every member's room list forever with an owner who could never log in again.

None of this was fixable downstream. There was no "account removed" event to subscribe
to: PlayerService.remove set status = 'closed' and revoked sessions but emitted
nothing. Ownership is read two ways — chatRoom.creatorId gates room edit and delete,
chatRoomMember.role gates the moderation guards — and both are written only from inside
the chat services, so a transfer that moved one and not the other would leave a room its
new owner could not administer. Rooms had only ever been soft-deleted, and
NOTIFICATION_TYPES is a closed enum.

Alternatives considered

Trigger on player.account.closed alone, not also identity.user.deactivated.
Deactivation is the other real way an account stops being reachable, and an operator who
deactivates rather than closes would have left rooms stranded exactly as before. Both
route into one idempotent handler, so the two firing for the same person is a no-op the
second time. See the reversibility risk below.

Delete the closed owner's member row instead of marking it. Simpler, but
listRoomMembers backs the author names on that member's entire message history in the
room; dropping the row turns every message they ever sent into an unattributed one. The
row stays, carries accountClosedAt, and surfaces as isDeletedAccount on both roster
schemas so a client can render it as a deleted user.

Soft-delete the room at the deadline. Consistent with the rest of the chat domain,
but the requirement is permanent deletion of the room and its messages. The purge is a
real hard delete; the chat.private_room.purged audit record is the only surviving trace.

Expose scheduledDeletionAt as the roster's deletion signal. The contract carries
isDeletedAccount as a boolean instead — clients only ever branch on it, and a second
date on the roster invites a per-member countdown UI that does not exist.

Route the countdown notification through notificationEventMap. A map entry produces
exactly one notification, and this event notifies every member. It stays a bespoke
subscription, but enqueues onto the same notifications-dispatch queue — one job per
recipient, keyed notifications-dispatch:<eventId>:<userId>, since a single key per
event would collide and silently drop every member after the first.

An hourly or per-room purge tick. The deadline has day granularity, so a finer tick
buys nothing and one missed run only deletes a day late. Daily at 03:15.

Risks

The transfer is not reversible, deactivation is. A reactivated player does not get
their room back — the transfer stands and a running countdown keeps running. Accepted
with the ticket. If it turns out wrong in practice the fix is to narrow the trigger to
player.account.closed alone, not to add an undo path.

The purge is the only hard delete in the chat domain. It is guarded on the room being
private, having a non-null scheduledDeletionAt, and that deadline having passed; the
full guard is re-read under the room's advisory lock before anything is deleted, and a
test pins that a public room, a null-deadline room, a future-deadline room and a
soft-deleted room all survive a cycle. Any future loosening of that where clause
destroys data irrecoverably.

Only the handler writes the countdown, and only once. scheduledDeletionAt is set
where scheduledDeletionAt is null, which is what makes the deadline immune to member
activity. A second write path against that column would silently reintroduce a resettable
countdown.

Case B parks the room with creatorId = null on purpose. For the 30 days it is
writable for chat and frozen for administration — members talk, nobody can rename it or
delete it early. That is the intended reading of "the chat remains fully active", but it
is a state no room could previously be in, so clients must tolerate a null creatorId.

Notification duplicates on a partial retry. The countdown fan-out dedupes on the
queue key, not on the notification.event_id unique index, so a worker that inserts and
then dies before acking can re-notify that one member. Same property as the existing
kyc-resubmission path; an in-app chat notice is not a money path.

Deferred: the event's memberIds is now the notification audience rather than the
roster — closed accounts and the previous owner are excluded — so the audit record's
after.memberCount counts recipients, not members. Renaming both to recipient* would
remove the ambiguity and costs nothing while this is unreleased.

@damianrzepka damianrzepka left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified pnpm check:types (clean) and the unit tests for the touched files (map-event.test.ts, notification-event-map.test.ts, 56/56 passing) in packages/core. Didn't run the *.int.test.ts suite (needs real PG, didn't want to run it against a shared DB), but it reads as thorough - transfer by roleAssignedAt with joinedAt fallback, skipping an already-closed moderator, no-op on replay, public rooms untouched, and the full purge cycle including "nothing due" and "purge only what's due".

Overall this is a careful, well-locked piece of work: idempotency guard on chatRoomMember.accountClosedAt, shared advisory lock key with join/leave/promote, deadline written once and never rewritten, full audit coverage, per-member idempotency key on the notification dispatch. Two non-blocking notes left inline. Also noticed merge conflicts are currently showing against dev on system-design.md and notifications/plugin.ts - probably just needs a rebase.

// Emitted after the status write commits: subscribers (chat's room-ownership handover)
// act on a closure that has actually happened, never on one that could still roll back.
this.events.emit('player.account.closed', {
playerId,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove() isn't transactional: the status UPDATE, sessionCommands.revokeAll, and this emit are three separate steps. If revokeAll throws after the status write commits, this event never fires and any private chat room owned by that player is stranded permanently - nothing later reconciles "closed player whose rooms were never handed over." Given how carefully idempotency is handled elsewhere in this PR (the account-closed handler, the purge job), might be worth a follow-up: either wrap this in a real transaction and emit post-commit, or add a periodic reconciliation scan for closed players with no matching chat_room_member.account_closed_at.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified, and it turned out to be worse than the partial-failure case you described - thanks for pulling the thread.

remove() is indeed three unsynchronised steps. But while checking it I found a second closure route that fails deterministically: PlayerService.update() accepts status: 'closed' (the contract permits the full PlayerStatusSchema), revokes sessions for it via BLOCKING_PLAYER_STATUSES, and never emitted player.account.closed at all. Chat's handover subscribes to the event, not the route, so an admin who closed an account by PATCH instead of DELETE stranded every private room that player owned - exactly the bug this PR exists to fix, reachable 100% of the time rather than only on a crash between steps.

Fixed in cb27b82: both routes now emit the same payload. The handover handler already guards on chatRoomMember.accountClosedAt, so a player reached by both paths is a no-op the second time. Added four tests (emission from update, identical payload from remove, no emission for suspended, no re-emission when already closed) and corrected the event schema's docstring, which still said PlayerService.remove was the only emitter.

On your two suggested remedies: the transaction one is not directly available here - emitInTransaction() throws unless the outbox is bound (server/kernel/event-bus.ts:98, create-app.ts:209), and OUTBOX_ENABLED is off by default. So the reconciliation scan is the practical option, and it is also the broader one: there is a third hole neither of us listed, which is that the subscriber in chat/plugin.ts:105 is fire-and-forget with .catch(log), so a handleAccountClosed that throws mid-flight strands the room with only a log line and no retry. A periodic scan for closed players with no matching chat_room_member.account_closed_at covers all three. Leaving that for the follow-up you suggested rather than growing this PR.

.where(eq(chatRoomMember.roomId, roomId));
// chat_message, chat_platform_ban and chat_mute point at chat_room without a
// cascade, so they go first or the room delete trips their foreign key. Everything
// else (chat_room_member, _rule, _configuration, _ban, _mute, _remove) cascades.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this comment's "_mute" (in the cascade list) actually means chatRoomMute, while the chatMute deleted explicitly on line 101 below is a different table entirely - one without an FK to chat_room at all (it's nullable, "null = global chat" per its own schema comment). Behavior is correct either way, but the name overlap between the two "mute" tables could mislead someone grepping the schema for chatRoomMute later.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and the comment was wrong in a second way I had not noticed until I checked the schema.

You are right that the _mute in the cascade list means chat_room_mute (schema/index.ts:213, FK with onDelete: 'cascade') while line 101 deletes chat_mute (schema/index.ts:280), a different table. But the sentence above it also claimed chat_mute "points at chat_room without a cascade, so they go first or the room delete trips their foreign key" - chat_mute.roomId is a bare nullable uuid() with no .references() at all, so there is no foreign key to trip. That delete is orphan cleanup, not ordering. Only chat_message and chat_platform_ban actually carry a cascade-less FK, and for those the original reasoning holds.

Rewritten in cb27b82 to separate the two reasons and to name chat_room_mute in full where it appears in the cascade list. Behaviour unchanged - the deletes were already correct, this is comment-only.

Mateusz Paś and others added 2 commits September 2, 2026 07:31
A private room used to be stranded by its owner's account closing: `creatorId`
kept pointing at a dead account, so nobody could rename or delete the room, and
`PlayerService.remove` emitted no domain event for anything to react to.

- New `player.account.closed` domain event, emitted after the status write
  commits. The chat module subscribes to it and to `identity.user.deactivated`,
  routing both into one idempotent handler.
- `ChatRoomMembershipService.handleAccountClosed` stamps the closed account's
  member row and, when it was the owner's, hands the room to the moderator with
  the earliest `roleAssignedAt`. `chatRoom.creatorId` and the member role move
  together, since ownership is read through both.
- With no moderator to inherit it, the room keeps `creatorId = null` and gets
  `scheduledDeletionAt` 30 days out: writable for chat, frozen for
  administration. The deadline is written once under the room lock and never
  rewritten, so member activity cannot move it.
- New `chat-room-purge` cron hard-deletes rooms past their deadline with their
  messages, bans and mutes. It is the only hard delete in the chat domain, so it
  is guarded on the room being private with a deadline that has passed, and the
  `chat.private_room.purged` audit record is the room's only surviving trace.
- `ChatRoomSchema` gains `scheduledDeletionAt`; both roster schemas gain
  `isDeletedAccount`, so a closed account renders as a deleted user without
  losing the author names on its message history.
- Notifications go to the inheriting owner and to every member of a room on the
  countdown, through the dispatch queue so they are retried and pushed live.
  Members whose own account is already closed are left out of that audience:
  they have no session and could never read the notification.

BF-494

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012LmCQXcxHdRG3rtcYB5Kss
…ct the purge cascade note

Review follow-ups on the room-ownership handover.

`PlayerService.update` accepts `status: 'closed'` and revoked sessions for it, but
never emitted `player.account.closed` - only `remove()` did. Chat's handover
subscribes to the event, not the route, so an admin who closed an account by PATCH
left every private room that player owned stranded with a dead owner, exactly the
bug this feature exists to fix. Both routes now emit the same payload; the handover
handler already guards on `chatRoomMember.accountClosedAt`, so a player reached by
both paths is a no-op the second time.

The purge cascade comment named `chat_mute` as an FK that the room delete would
trip. It has no FK at all - `roomId` is a bare nullable column - so that delete is
orphan cleanup, not ordering. The comment also listed `_mute` among the cascading
tables, which is `chat_room_mute`, a different table from the `chat_mute` deleted
two lines below. Comment only; the deletes were already correct.

Ports both new integration tests off `InProcessRealtimeTransport`, dropped from
`@openora/core/testing` on dev, onto the Redis-backed transport the sibling chat
integration test already uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPoLHR63UFg1DNjRALheyD
@mp-blurify
mp-blurify force-pushed the feat/chat-room-ownership-handover branch from dcb197f to cb27b82 Compare September 2, 2026 05:52
@mp-blurify

Copy link
Copy Markdown
Collaborator Author

Rebased onto dev (9f922bd9) and pushed - the system-design.md and notifications/plugin.ts conflicts you spotted are resolved. Both were purely additive: dev's rg.limit.set handler now sits alongside this branch's chat.room.scheduled_for_deletion one, and the system-design.md line is generated, so pnpm regen rewrote it (check:drift clean). No chat migration collision - 0011 was still free and gen:drizzle reported no schema changes.

One thing worth flagging that the rebase surfaced: #125 removed InProcessRealtimeTransport from @openora/core/testing, which broke the build of both new integration tests. room-ownership-handover.int.test.ts now uses RedisPubSubRealtimeTransport on an isolated key prefix with the transports closed in afterEach, mirroring chat.service.int.test.ts; chat-room-notifications.int.test.ts uses makeRealtimeTransport() instead, since it only resolves the transport out of the container and never asserts on a publish.

Both inline notes are answered in their threads - the first one turned out to have a deterministic sibling bug, now fixed here; the second is comment-only.

Full pnpm verify green locally: 15/15 tasks, 1404 core integration tests (player.service.int.test.ts 38, room-ownership-handover.int.test.ts 17, chat-room-notifications.int.test.ts 4), plus check:drift.

`admin deletion publishes a tombstone and records audit data` subscribed and
published in the same tick. `RedisPubSubRealtimeTransport.addSubscriber` returns
synchronously and fires the real SUBSCRIBE on a floating promise, and Redis pub/sub
keeps no backlog, so a publish that beats it is dropped for good - not delivered
late. The test then spun in `waitFor` until it timed out, which is what it did on
CI here and on dev in run 33533827246.

Every other subscriber in this file already awaits `settle()` first; this one was
missed when #125 swapped the in-process transport for Redis. Raising the timeout
would not have helped, since the message is gone rather than slow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NPoLHR63UFg1DNjRALheyD
@mp-blurify

Copy link
Copy Markdown
Collaborator Author

CI is green now. One extra commit (e77c591a) touches a test this PR otherwise has nothing to do with, so flagging why.

chat.service.int.test.ts > ChatService moderation > admin deletion publishes a tombstone failed the first verify here, and failed again identically on a re-run - so not a transient flake. It also took down the most recent pipeline on dev itself (run 33533827246), before this branch was rebased.

Cause is a subscribe/publish race, not slowness: RedisPubSubRealtimeTransport.addSubscriber (redis-pubsub-realtime-transport.ts:224) returns synchronously and fires the real SUBSCRIBE on a floating promise the caller cannot await. Redis pub/sub keeps no backlog, so a publish that beats it is dropped for good rather than delivered late - the test then spun in waitFor until the 2s deadline. Raising that deadline would not have fixed it.

The test subscribed and published in the same tick. Every other subscriber in the file already awaits settle() first; this one was missed when #125 swapped the in-process transport for Redis. The fix is that one await settle(), test-only, matching the file's existing idiom. Full suite 137/137 locally.

Worth noting separately: subscribe() giving callers no handle on subscription readiness is a real gap in the transport contract - every consumer has the same race, and the tests paper over it with a 150ms sleep. That belongs in its own issue rather than here, so I have not touched the transport.

.set({ role: 'member', roleAssignedAt: null })
.where(and(eq(chatRoomMember.roomId, roomId), eq(chatRoomMember.userId, userId)));

if (successor) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The handover lock does not cover every membership-changing path. removeMember takes no chat-room:${roomId} lock, and banMember authorizes before taking it. A closing owner can therefore remove or ban successor after it is selected here; this update can affect zero rows, but we still set creatorId to that removed user and demote the prior owner. The room then has no owner member and no deletion deadline, so it is stranded permanently. Please serialize those paths on the same room lock, re-check ban authorization inside it, and require the promotion update to affect exactly one row before updating creatorId (otherwise reselect/schedule deletion). This is the concurrency-invariant requirement in docs/standards/database.md.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against the branch - all three parts hold, and the end state is as bad as you describe: the room keeps a creatorId who is not on the roster, no owner member and no scheduledDeletionAt, and it is permanent, because chatRoomMember.accountClosedAt is already stamped by then and a re-run of handleAccountClosed skips the room outright.

removeMember took no lock at all (its transaction went straight to the role reads), banMember ran assertModerator before withAdvisoryXactLock, and the promotion UPDATE result was never inspected before creatorId followed the user the select had picked.

Fixed in abff41f:

  • removeMember runs under the same chat-room:${roomId} lock as join, leave, setMemberRole and the handover.
  • assertModerator now takes the executor, and banMember calls it inside the lock - a request authorized while its caller was still the owner is re-checked against the demoted row and rejected.
  • the handover promotes the first candidate whose update actually affects a row, and when none does it falls through to the ownerless countdown instead of naming a creator that was never written. With every writer on the lock this is unreachable in practice, which is the point: it is the guard that keeps it unreachable if a future writer forgets the lock.

Also dropped the stale comment in setMemberRole that pointed at removeMember as the lock-free path.

Three integration tests in room-ownership-handover.int.test.ts, all three failing on the previous code:

  • the promotion guard - a DELETE on the successor's row is held uncommitted, so the handover's select still sees the row (READ COMMITTED) while its UPDATE blocks on the row lock and then matches nothing. That is the exact select-then-vanish interleaving, not an approximation of it. Asserts the countdown starts rather than a creator being named.
  • removeMember blocks while the room lock is held, then rejects with ChatRoomNotModeratorError once the handover's writes commit, leaving the new owner's row intact.
  • banMember the same, which before the fix would have deleted the member row of the owner the handover had just installed.

pnpm verify green, 15/15 including check:drift.

@damianrzepka damianrzepka left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR currently conflicts with the latest dev (the generated catalog reference and the concurrent chat test change). Please rebase, resolve the overlaps, run pnpm regen, and rerun verification.

Mateusz Paś and others added 2 commits September 3, 2026 08:32
Resolves the two conflicts from the cms-banner and chat-tombstone work on dev:
- chat.service.int.test.ts: both branches added the same `await settle()` before
  the tombstone publish; keep it once, with the comment explaining why.
- system-design.md: regenerated the catalog reference line so it counts both
  branches' routes and events.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SyLqsaeoprPoPXehsU5BbR
…ndover

A closing owner's `removeMember` or `banMember` could land between the handover
picking a successor and promoting them: `removeMember` took no room lock at all,
and `banMember` authorized before taking it. The promotion then matched zero rows
while `creatorId` still followed the selected user, leaving a private room with a
creator who is not a member, no owner on the roster and no deletion deadline -
permanent, since `accountClosedAt` is already stamped and a re-run skips the room.

- `removeMember` runs under the same `chat-room:${roomId}` advisory lock as join,
  leave, setMemberRole and the handover.
- `banMember` asserts moderator authority inside that lock, against rows nobody
  can move underneath it.
- the handover promotes the first candidate whose update actually affects a row,
  and falls through to the ownerless countdown when none does, so `creatorId`
  never names a row that was not written.

Three integration tests reproduce the interleavings - the promotion one blocks the
UPDATE on an uncommitted delete's row lock, so it is the exact select-then-vanish
case rather than an approximation. All three fail on the previous code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SyLqsaeoprPoPXehsU5BbR
@mp-blurify

Copy link
Copy Markdown
Collaborator Author

Conflicts with dev resolved in 0f67d70, and the lock finding answered in abff41f.

Merge (0f67d70). Two conflicts, both from work that overlapped this branch:

  • chat.service.int.test.ts - dev (f250122) and this branch (e77c591) landed the same await settle() before the tombstone publish independently. Kept once, with the comment explaining why the subscribe has to settle first.
  • system-design.md - the generated catalog reference line. Not merged by hand: pnpm gen:catalog rewrote it to 247 routes / 107 events, which is both sides' counts. docs/catalog.json auto-merged and regeneration left it byte-identical, so nothing drifted.

Lock finding (abff41f). Confirmed on the branch and fixed - removeMember now takes the room lock, banMember authorizes inside it, and the promotion has to affect a row before creatorId follows it, otherwise the room gets the ownerless countdown. Details and the three regression tests are in the thread.

pnpm verify green locally on the merged tree, 15/15 tasks including check:drift, core integration 1427 and @openora/testing 149.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need it. Let's remove.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed against the history rather than taken on trust, and you are right: the convention lapsed and I was following the old one.

Every PR up to the 9f922bd9 release carried a changeset (#119, #82, #86, #69 among them), and that release consumed fourteen of them at once. Nothing merged to dev since has added one - #127, #132 and dd9cb296 all touch packages/core and leave .changeset/ empty. This branch was the only one still writing them.

Removed in b88c0bd. The user-facing summary it carried is not lost - it is what the PR description says.

Comment on lines +103 to +105
await t.delete(chatPlatformBan).where(eq(chatPlatformBan.roomId, roomId));
await t.delete(chatMute).where(eq(chatMute.roomId, roomId));
await t.delete(chatRoom).where(eq(chatRoom.id, roomId));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, maybe we should enable cascade?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - and the manual deletes were the odd ones out, not the cascade. Six of the eight room-scoped tables already declare onDelete: 'cascade'; chat_message and chat_platform_ban were the only two carrying a bare .references(() => chatRoom.id), which is exactly why the purge had to delete them by hand and in the right order.

Enabled in b88c0bd (migration 0013_chilly_sprite, drop and re-add both constraints with ON DELETE cascade). The purge now deletes the room and lets the FKs do the rest.

Two notes on scope:

  • chat_mute stays a manual delete. It has no foreign key at all - its roomId is a bare nullable column, null = global chat - so nothing cascades to it and its rows would outlive the room as orphans.
  • Room-scoped platform bans cascade; global ones (roomId IS NULL) are untouched, as they must be.
  • The cascade only ever fires on the purge. Every other room deletion in the domain is a soft delete via chatRoom.deletedAt, which leaves messages exactly where they are.

The existing purge test asserts messages, members, bans and mutes are all gone, so it now covers the cascade rather than the delete order.

Comment on lines +42 to +57
const due = await this.drizzle.db
.select({ id: chatRoom.id })
.from(chatRoom)
.where(
and(
eq(chatRoom.isPublic, false),
isNull(chatRoom.deletedAt),
// `isNotNull` is redundant next to `lte` in SQL, but it is the invariant this
// job exists to respect - a room with no deadline is never eligible - so it is
// stated, not implied.
isNotNull(chatRoom.scheduledDeletionAt),
lte(chatRoom.scheduledDeletionAt, new Date()),
),
);
let purged = 0;
for (const { id } of due) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, why isn't this a job per room? A cron schedule ends up with attempts: 1 since schedule in kernel/bullmq-job-queue.ts never passes attempts, so a room that fails to delete just waits for tomorrow's tick. And because purgeRoom throws straight out of the loop, one bad room blocks every room behind it, on every run.

Let's make the tick a dispatcher instead: select the due rooms with a limit, enqueue one job per room keyed chat-room-purge:<roomId> with attempts and a backoff, and let the worker purge a single room. Then retries are per room, and the batch limit stops one tick from serially chewing through the whole backlog after downtime.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified, all three parts, and the first one is worse than "waits for tomorrow".

schedule() builds its BullMQ options as { name, data } only (kernel/bullmq-job-queue.ts:151-155) - attempts is never passed, so a cron job runs with BullMQ's default of 1. And runCycle had no try/catch around purgeRoom, so the throw left the loop: one bad room dropped every room behind it in that tick, and the tick that would retry them is a day away with no retry of its own. The select also had no limit, so a tick after downtime chewed through the whole backlog serially.

Restructured as you describe in b88c0bd. ChatRoomPurgeService is now two entry points - listDueRooms(limit) and purgeRoom(roomId) - and the plugin owns the dispatch: the cron lands on chat-room-purge-scan, which enqueues one chat-room-purge job per room with attempts: 5 and exponential backoff, and a dead-letter log. Batch limit is 500; a deadline that already passed does not get worse for being purged on the next tick.

One deviation from your suggested key, because chat-room-purge:<roomId> alone would have reintroduced the bug in a quieter form. The driver retains failed jobs for seven days (RETAIN_FAILED), and BullMQ's add with an existing jobId returns the existing job instead of enqueueing - so a room whose job exhausted its retries would collide with its own dead job and be skipped for a week. The key is chat-room-purge:<roomId>:<yyyy-mm-dd>: per room so two replicas ticking together enqueue once, per tick so tomorrow's dispatch is a fresh id.

Tests: the batch limit returns the oldest deadlines first, a second job for an already-purged room is a no-op, and a job carrying an id that stopped qualifying refuses to delete (the scan is only half the guard - the room is re-read under the lock).


// Daily at 03:15. The deadline the job checks has day granularity, so a finer tick buys
// nothing and one missed run only deletes a day late.
const CHAT_ROOM_PURGE_CRON = '15 3 * * *';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15 3 * * * in which timezone? schedule accepts timezone and we don't pass it, so it follows the process local time and two replicas in different zones won't agree on when it runs. Nothing gets purged early thanks to the lte(scheduledDeletionAt, ...) guard, but let's pin timezone: 'UTC' so the schedule is predictable.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in b88c0bd: { cron: CHAT_ROOM_PURGE_CRON, timezone: 'UTC' }. The port has carried timezone all along (contracts/adapters/job-queue.ts:53) and the driver maps it to BullMQ's tz (bullmq-job-queue.ts:153) - the call site simply never passed it, so it followed process-local time.

Your read on the blast radius is right: nothing could be purged early, because lte(scheduledDeletionAt, now) decides that independently. The cost was only that "daily" meant a different hour per replica.

Worth flagging for a separate pass: no cron in the repo passes a timezone - tag.daily-evaluation, notifications.retention-purge, wallet-custody-sweep and wallet-reconciliation are all in the same position. I have only pinned chat's here rather than widening this PR.

Comment on lines +112 to +118
// Emitted before the realtime cleanup: this event is what writes the audit record, and
// after a hard delete that record is the room's only surviving trace - it must not go
// missing because a transport happened to be down.
this.events.emit('chat.private_room.purged', {
roomId,
messageCount: result.messageCount,
});

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a hard delete, but the audit record comes from emit after the transaction commits, and emit is best-effort. If the process dies right after the commit, the room and its messages are gone with no trace at all - the opposite of what the comment above says. Let's write the audit row inside the same transaction via AUDIT_WRITER.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, and the comment above it was the tell - it claimed the emit came first so the trace could not be lost to a transport being down, while the thing that could actually lose it was a crash between the commit and the emit.

Fixed in b88c0bd. purgeRoom now calls AUDIT_WRITER.recordInTransaction inside the deleting transaction, before the delete, so a failure to record fails the purge rather than producing an untraceable deletion. Message count is taken with a count() before the rows go, since it is the one quantity the record carries about what was destroyed.

That forced a decision on the other end: chat.private_room.purged was in the audit plugin's subscribed-topic list, so leaving it there would have written the row twice. It is out of that list and out of mapEventToRecord, with a comment on both sides saying chat owns this one transactionally and why. The event itself stays - overlays and clients can still react to a purge; it just no longer carries the trace.

Two tests: the audit call is asserted with its messageCount, and an audit writer that rejects leaves the room standing.

Comment on lines +410 to 428
async handleAccountClosed({ userId, closedAt }: { userId: Uuid; closedAt: Date }) {
const rooms = await this.drizzle.db
.select({ id: chatRoom.id })
.from(chatRoomMember)
.innerJoin(chatRoom, eq(chatRoom.id, chatRoomMember.roomId))
.where(
and(
eq(chatRoomMember.userId, userId),
isNull(chatRoomMember.accountClosedAt),
eq(chatRoom.isPublic, false),
isNull(chatRoom.deletedAt),
),
);
for (const { id: roomId } of rooms) {
const result = await this.closeAccountInRoom(roomId, userId, closedAt);
if (result) {
await this.announceAccountClosed(roomId, userId, result.handover);
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same shape as the purge, and here there is no retry anywhere behind it at all - plugin.ts calls handleAccountClosed fire-and-forget with .catch(log), and player.account.closed is itself a best-effort in-process emit. So if the event drops, or this throws on room 3 of 10, the rest stay stranded with a dead owner, which is the bug we're fixing here.

Let's put it on JOB_QUEUE keyed by the user with attempts and a backoff - we already do exactly that for the deletion notifications in notifications/plugin.ts.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and this is the same hole damianrzepka and I traced upthread from the PlayerService side - I had deferred it to a follow-up, which was the wrong call given it is the bug this PR exists to fix.

Confirmed as described: plugin.ts:105 called handleAccountClosed(...).catch(logger.error), the emit behind it is best-effort and in-process, and the room loop had nothing wrapping it - so a throw on room 3 of 10 left rooms 4-10 with a dead owner and a log line.

Moved onto JOB_QUEUE in b88c0bd, matching the notifications/plugin.ts shape you pointed at: attempts: 5, exponential backoff from 1s, dead-letter log. Keyed chat-account-closed:<userId> as you suggested - the two triggers (player.account.closed and identity.user.deactivated) collapse onto one job per person, which is what we want, and the handler's own idempotency covers the case where they do not.

One consequence worth naming: closedAt now comes off the job payload rather than new Date() at handler time, so every retry of one closure carries the same instant. That is what makes the re-announce in the sibling thread able to recognise its own work.

Comment on lines +440 to +455
// The stamp is the idempotency guard for the whole handler: a row already marked
// is a room a previous run already finished, so it is skipped outright.
const [stamped] = await t
.update(chatRoomMember)
.set({ accountClosedAt: closedAt })
.where(
and(
eq(chatRoomMember.roomId, roomId),
eq(chatRoomMember.userId, userId),
isNull(chatRoomMember.accountClosedAt),
),
)
.returning({ role: chatRoomMember.role });
if (!stamped) {
return null;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this one: accountClosedAt is the idempotency guard for the whole handler, but the notifications and the realtime signals only go out after the transaction commits. If we die in between, a redelivery short-circuits right here, so members never hear the room is closing - and the purge still deletes it 30 days later. Shouldn't the announce be re-derivable instead of guarded away, eg. re-announce when the row is already stamped but the room is on a countdown?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, and it is not only the notifications - the realtime countdown banner goes the same way, and the purge still fires 30 days later either way. Guarding the announce behind the write stamp meant a redelivery could only ever make things quieter.

Made it re-derivable in b88c0bd. Losing the stamp race no longer returns; it hands over to rederiveHandover, which reads (never writes) and re-announces only what THIS closure caused. Three stamps make that provable:

  • the member row carries accountClosedAt = closedAt;
  • an owner-less countdown is exactly closedAt + OWNERLESS_ROOM_RETENTION_DAYS;
  • a successor promoted by this closure carries roleAssignedAt = closedAt - I changed the promotion from new Date() to closedAt for this, which is also the more honest timestamp on a retry: the room changed hands when the account closed, not when the retry ran.

So the transfer case is covered too, not just the countdown. A room changed by somebody else's closure matches none of the three and is left alone. closedAt is stable across retries now that it rides the job payload (sibling thread), which is what makes the comparison meaningful.

The room scan in handleAccountClosed had to stop filtering on accountClosedAt, or a redelivery would never reach the rooms it needs to re-announce.

Four tests: countdown re-announced with the original deadline and audience and nothing rewritten; transfer re-announced naming the successor the first delivery installed; silence when the countdown was somebody else's closure; silence for a non-owner, where there was nothing to announce. The existing "no-op the second time" test still holds and now says something sharper - it uses a different closedAt, so it is the case the re-derivation declines.

Mateusz Paś and others added 2 commits September 4, 2026 10:09
Renumber the branch's chat migration to 0012 so it lands after dev's
0011_light_changeling, and regenerate it from the merged schema.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015x4vmKHYFVSkc2BHuMaWKm
…ge in-transaction

Answers the seven review comments on #129. Four of them are one problem seen from
four sides - work that deletes data or is a member's only warning was running with
no retry behind it - so they are fixed together.

- The purge cron is a dispatcher. `listDueRooms` hands out a bounded batch, oldest
  deadline first, and one job per room calls `purgeRoom`. A room that fails now
  retries on its own (attempts + exponential backoff) instead of waiting for
  tomorrow's tick, and cannot block the rooms behind it.
- `handleAccountClosed` moves onto JOB_QUEUE, keyed per user. It was fire-and-forget
  behind a best-effort emit, so a throw on room 3 of 10 stranded the rest with
  nothing to retry it.
- The purge audit record joins the deleting transaction via AUDIT_WRITER. It is the
  room's only surviving trace, and a post-commit emit lost it exactly when it mattered
  most. `chat.private_room.purged` therefore leaves the audit plugin's topic list; the
  event itself stays for overlays.
- The countdown announce is re-derivable instead of guarded away. A redelivery
  re-announces what its own closure wrote - proven by `accountClosedAt`, the
  `closedAt + 30d` deadline, and a successor's `roleAssignedAt` - and writes nothing.
- `chat_message.room_id` and `chat_platform_ban.room_id` cascade like every other
  room-scoped table, so the purge no longer deletes them by hand.
- The purge cron pins `timezone: 'UTC'`; two replicas in different zones disagreed
  on when "daily" was.
- Drops the changeset: no PR merged to dev since the 9f922bd release carries one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015x4vmKHYFVSkc2BHuMaWKm
@mp-blurify

Copy link
Copy Markdown
Collaborator Author

Merged dev (3dbc187) and answered zaxovaiko's seven comments (b88c0bd). All seven checked out against the branch before fixing; details are in each thread.

Four of them turned out to be one problem seen from four sides - work that either deletes data or is a member's only warning was running with nothing to retry it - so they are fixed together:

  • Purge cron is a dispatcher. listDueRooms(limit) hands out a bounded batch, oldest deadline first; one chat-room-purge job per room calls purgeRoom, with attempts: 5, exponential backoff and a dead-letter log. A failing room retries on its own and cannot block the rooms behind it. Job key is per room and per tick - chat-room-purge:<roomId>:<date> - because a room-only key would collide with its own dead job in the driver's retained-failed set and skip the room for a week.
  • handleAccountClosed on JOB_QUEUE, keyed per user, same retry policy. It was fire-and-forget behind a best-effort emit.
  • Purge audit joins the deleting transaction via AUDIT_WRITER.recordInTransaction. It is the room's only surviving trace, so chat.private_room.purged leaves the audit plugin's topic list to avoid a double row; the event stays for overlays.
  • The countdown announce is re-derivable, not guarded away. A redelivery re-announces what its own closure wrote - provable from accountClosedAt, the closedAt + 30d deadline and a successor's roleAssignedAt - and writes nothing. Covers the transfer case too.

And three smaller ones: chat_message.room_id and chat_platform_ban.room_id now cascade like the other six room-scoped tables (migration 0013_chilly_sprite), the purge cron pins timezone: 'UTC', and the changeset is gone - no PR merged to dev since the 9f922bd9 release carries one.

Thirteen tests added or reworked. pnpm verify green, 15/15 including check:drift.

One thing left deliberately outside this PR: no cron in the repo passes a timezone. tag.daily-evaluation, notifications.retention-purge, wallet-custody-sweep and wallet-reconciliation are all in the same position chat's was.

…g repeat closures

Closing a player was one-way in a way neither of its triggers is. Nothing cleared
`chatRoomMember.accountClosedAt` or `chatRoom.scheduledDeletionAt`, so an account
reactivated by an admin still rendered as a deleted user on every roster it was on,
could never inherit a room again, and - worst - the 30-day countdown its closure had
started kept running until the purge hard-deleted the room and every message in it.
That countdown could not be stopped by hand either: `creatorId` is null while it runs,
so `updatePrivateRoom` and `deletePrivateRoom` both throw. A reversible admin action
therefore destroyed data.

Adds the inverse path. `player.account.reopened` fires when `PlayerService.update`
moves a player out of `closed` (the only route out - `remove()` has no undo), and chat
subscribes to it and to the existing `identity.user.reactivated`, mirroring how the two
closure triggers are handled. Both land on `chat-account-reopened`, which clears the
member stamp everywhere and, for a room still owner-less whose deadline is exactly
`accountClosedAt + OWNERLESS_ROOM_RETENTION_DAYS`, cancels the countdown and gives the
room back. A room a moderator inherited is left alone: that transfer still stands. The
deadline match is what keeps a plain member who reopens from being handed a room that a
different member's closure put on death row. `chat.room.deletion.cancelled` carries the
reversal into the audit log, and the countdown's realtime signal now takes a null
deadline so one client handler both raises and clears the banner.

Three smaller fixes from the same review:

- `chat-account-closed:<userId>` is now per user and per day. The driver retains
  completed jobs for 24h and failed ones for 7 days, and an enqueue whose id is already
  in either set is a silent no-op, so a second closure - after a reactivation, or after
  the first job dead-lettered - vanished without an error. Same reasoning that already
  made the purge key per room and per tick.
- `unbanMember` authorizes inside the room lock, like `banMember` already does.
- The purge scan worker logs its dead letter, and an account-state event dropped because
  the job queue is not bound yet is logged instead of returning silently.
The reasoning lives in the PR description and the commit messages; the code is
read on its own terms.
`soft-stale: returns the cached value immediately and refreshes in the background`
asserted the refreshed row after a fixed `wait(80)`. The provider resolves at 30ms but
the write behind it lands whenever the pool hands a connection back, so on a loaded CI
runner the row was still the cached 1.1 when the assertion ran. Swapped for `vi.waitFor`,
the idiom the cross-leg refresh test in the same file already uses for this race.
@zaxovaiko
zaxovaiko merged commit 4bf99d0 into dev Sep 4, 2026
2 checks passed
@zaxovaiko
zaxovaiko deleted the feat/chat-room-ownership-handover branch September 4, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants