Skip to content

fix: keep counter sample time pairs synchronized - #817

Open
somethingwithproof wants to merge 18 commits into
developfrom
fix/counter-sample-time-pair
Open

fix: keep counter sample time pairs synchronized#817
somethingwithproof wants to merge 18 commits into
developfrom
fix/counter-sample-time-pair

Conversation

@somethingwithproof

@somethingwithproof somethingwithproof commented Aug 18, 2026

Copy link
Copy Markdown
Member

Sample values and their timestamps were persisted separately, so a poll could pair a value with the wrong time. They are now written as a pair in both the poller and daemon paths.

Stale, unavailable, duplicate, and transformed-unknown rates fail closed instead of being read as a real value, and an alert holds its state across an unknown sample rather than falsely recovering.

Backward sample clocks are re-anchored for recovery, and rate validation uses the RRD heartbeat rather than the poll interval.

Operational note: thresholds that previously alerted on a manufactured zero will now hold their prior state through unavailable samples. Expect fewer false recoveries and more THOLD log volume at medium verbosity on flaky devices.

Closes #815.

@somethingwithproof somethingwithproof added bug daemon Related to thold daemon confirmed Issue that has been confirmed by the dev team labels Aug 18, 2026
@somethingwithproof somethingwithproof self-assigned this Aug 18, 2026
@somethingwithproof
somethingwithproof marked this pull request as ready for review August 18, 2026 14:01
Copilot AI lite review requested due to automatic review settings August 18, 2026 14:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes issue #815 by ensuring counter/derive/absolute thresholds persist raw sample values and timestamps as a synchronized causal pair across both poller and daemon execution paths, preventing mismatched intervals from producing false alerts. It also updates unknown/stale/backward-clock handling so invalid rates fail closed without incorrectly clearing active alert state, and extends expression-related resolution behavior with safer fallbacks.

Changes:

  • Introduces shared sample-pair persistence + transition logging helpers, and applies heartbeat-aware rate validation in thold_get_currentval().
  • Updates both poller (includes/polling.php) and daemon (thold_process.php) persistence paths to use the same sample-pair rules, including a status-only prepared update batch.
  • Updates unit tests and documentation to reflect the new sampling, availability, and alert-state behavior.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
thold_process.php Switches daemon persistence to a centralized helper and fetches rrd_heartbeat for rate validation.
thold_functions.php Adds sample-pair persistence/logging helpers, heartbeat-aware rate validation, fail-closed handling for unknowns, and expression/percent/lower-upper hardening.
includes/polling.php Uses centralized poller batching row builder; adds prepared status-only batching and unified cleanup.
tests/Unit/ThresholdTimeBasedCharacterizationTest.php Updates unknown-reading characterization to assert alert state is preserved.
tests/Unit/ThresholdHiLowCharacterizationTest.php Updates unknown-reading characterization to assert alert state is preserved.
tests/Unit/TholdGetCurrentvalTest.php Adds broad coverage for sample-pair persistence, heartbeat-aware validation, expression fallback behavior, and unknown propagation.
tests/Unit/TholdCalculatePercentTest.php Updates behavior expectations for non-numeric denominators to yield the no-value sentinel.
tests/Unit/GetCurrentValueTest.php Adds coverage for the new “missing value” sentinel parameter.
tests/bootstrap-unit.php Adds missing constant stub + improves cacti_log() recording + adds substitute_snmp_query_data() stub.
README.md Documents the updated sampling/rate validation and alert-state preservation behavior.
CHANGELOG.md Adds issue #815 entry summarizing the behavior change.
Suppressed comments (1)

thold_functions.php:952

  • Same as in thold_daemon_persist_sample(): calling thold_log_unavailable_transition() before thold_sample_persistence() causes a backward-clock sample (numeric reading, but rate discarded and pair re-anchored) to log both the backward-clock warning and the generic "sample is unavailable" warning. Skipping the transition warning when currenttime < lasttime avoids duplicated/misleading warnings.
	thold_log_unavailable_transition($thold_data, $currentval);

	$sample = thold_sample_persistence($thold_data, $item, $currenttime);


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread thold_functions.php Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug confirmed Issue that has been confirmed by the dev team daemon Related to thold daemon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missed counter samples advance lasttime without oldvalue

2 participants