Skip to content

fix(notify): drain only the notifications this run claimed - #804

Merged
somethingwithproof merged 1 commit into
Cacti:developfrom
somethingwithproof:fix/notification-queue-claim
Aug 17, 2026
Merged

fix(notify): drain only the notifications this run claimed#804
somethingwithproof merged 1 commit into
Cacti:developfrom
somethingwithproof:fix/notification-queue-claim

Conversation

@somethingwithproof

Copy link
Copy Markdown
Member

Fixes the claim half of #784.

thold_notify.php stamps its own identifier on the rows it intends to handle:

db_execute_prepared('UPDATE notification_queue
    SET process_id = ?
    WHERE event_processed = 0',
    [$pid]);

and then drained with

thold_notification_execute();

No argument. $pid defaults to zero, which drops the filter, so every stage selected every unprocessed row regardless of who claimed it. poller_thold.php launches this script on each poller cycle with no check that the previous one has finished, so two overlapping runs both mail the same notifications.

--thread is never passed to this script — only thold_process.php takes it — so the collector branch always runs and the filter was always empty in practice.

Three changes

The identifier is passed to the drain. Both stages that read the queue now carry AND process_id = <pid>.

The claim follows the registration. It ran before register_process_start(), so a second instance stamped its identifier over the first instance's rows before discovering it should exit. It also now takes only rows nobody holds (AND process_id = 0), so a claim cannot steal work in progress.

An instance that cannot check for a peer stands down. The liveness check needs posix_getpgid() or posix_kill(). Where neither exists the old code fell through the whole guard and drained the queue a second time. It now exits, which is the safe reading of a question it cannot answer.

Not in this change

#784 also reports that a failed send is discarded: mailer() returning an error still marks the row event_processed = 1, so the notification is lost, which is the symptom behind #754.

Retrying properly needs somewhere to count attempts, and notification_queue has no column for it. Leaving event_processed = 0 without a counter turns a permanently failing notification into one that retries every cycle forever. That is a schema change with its own migration, so it belongs in its own pull request rather than smuggled into this one.

Tests

Seven tests. The scoping is behavioural — the drain is driven with and without an identifier and every queue query is checked for the filter. The ordering and liveness changes are guarded structurally, because thold_notify.php is a top-level script that runs on include and cannot be loaded without a full Cacti runtime.

For the same reason the changed file is outside the coverage scope, which covers thold_functions.php only; every other plugin file either includes ./include/auth.php or executes at top level. The gate correctly reports no measured lines changed.

Run against PHP 8.1, 8.2, 8.3 and 8.4 on Debian, all green.

thold_notify.php stamps its own identifier on the rows it intends to handle,
then called thold_notification_execute() with no arguments. That defaults the
identifier to zero, which drops the filter, so the drain selected every
unprocessed row regardless of who claimed it. poller_thold.php launches this
script on every poller cycle without checking whether the previous one has
finished, so two overlapping runs both mailed the same notifications.

The claim also ran before register_process_start(), so a second instance
stamped its identifier over the first instance's rows before discovering it
should exit, and it now takes only rows nobody holds.

Where neither posix_getpgid() nor posix_kill() is available there is no way to
ask whether the recorded process is alive. That case fell through and drained
the queue a second time; it now stands down, which is the safe reading of an
unanswerable question.

Refs Cacti#784

Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@somethingwithproof
somethingwithproof merged commit 4edd133 into Cacti:develop Aug 17, 2026
1 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants