Skip to content

fix(api): stamp emissions with measurement time, not send time - #1341

Closed
davidberenstein1957 wants to merge 1 commit into
masterfrom
scaling/01-emission-timestamps
Closed

fix(api): stamp emissions with measurement time, not send time#1341
davidberenstein1957 wants to merge 1 commit into
masterfrom
scaling/01-emission-timestamps

Conversation

@davidberenstein1957

Copy link
Copy Markdown
Collaborator

Part of #1338.

ApiClient.add_emission built EmissionCreate(timestamp=get_datetime_with_timezone(), ...), ignoring the timestamp the row already carries from measurement time (emissions_tracker.py:1047). So the hosted API and dashboard dated every point by receipt, and already disagreed with the CSV output for the same run.

Proof

End-to-end repro against a real stdlib HTTP server, with requests.post wrapped in a 5s delay, driven through CodeCarbonAPIOutput.live_out:

measured at : 2026-08-12T09:00:00
BEFORE  stored as : 2026-08-12T16:55:39.446260+02:00   # off by 7h55m
AFTER   stored as : 2026-08-12T09:00:00+02:00          # exact

Unit test test_add_emission_keeps_measurement_timestamp is red before, green after.

One deviation worth flagging

The original proposal preferred fixing this at the producer, by making EmissionsData.timestamp offset-aware. That changes the timestamp column format in everyone's emissions.csv — a user-visible output change for a bug that lives entirely in the API client. This takes the proposal's own fallback instead: normalise inside api_client, leave CSV untouched.

The fallback around the conversion is deliberately broad (KeyError/TypeError/ValueError → now), because add_emission is a public dict-taking method and an existing test already passes timestamp="222".

_create_run still stamps "now", which is correct there; there's now a comment saying so, so it doesn't get "fixed".

Tests

uv run pytest tests/ -q -k "output or http or api or tracker" → 252 passed, 2 skipped, over three consecutive runs. (tests/test_viz_data.py excluded — fails collection on master too, dash not installed.) One run showed two test_emissions_tracker_flush / test_logging_output failures; they pass consistently on re-run and in isolation on both branches — timing flakes. Ruff clean on both touched files, same as master.

Risk

Server-side rows now carry measurement time rather than receipt time, so a dashboard spanning the upgrade sees a one-time discontinuity equal to the old send latency — sub-second in practice.

Not included

Client-side batching, which the original proposal bundled with this. carbonserver has only the single-row POST /emissions — no /emissions/batch — so the batch path would be dead code that 404s and falls back to the loop it replaces, and the proposal's own api_batch_size=1 default makes it a no-op for anyone who doesn't opt in. It should land with the server endpoint or not at all.

Also left alone: the return False dedent in add_emission (api_client.py:186) — a real separate bug, not something to smuggle into a timestamp commit.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.40%. Comparing base (eb2cfef) to head (697e1e8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1341      +/-   ##
==========================================
- Coverage   91.43%   91.40%   -0.04%     
==========================================
  Files          49       49              
  Lines        5057     5062       +5     
==========================================
+ Hits         4624     4627       +3     
- Misses        433      435       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ApiClient.add_emission discarded carbon_emission["timestamp"] and called
get_datetime_with_timezone() instead, so every row stored the moment the
payload was built rather than the moment it was measured. Harmless while
the two are milliseconds apart; wrong by the full latency as soon as a
send is slow, retried or queued.

Normalise the measurement timestamp to offset-aware instead, falling back
to now when the payload carries none or an unparseable one (the method is
public and takes a plain dict). CSV output is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidberenstein1957
davidberenstein1957 force-pushed the scaling/01-emission-timestamps branch from 0658642 to 697e1e8 Compare August 19, 2026 09:18
@davidberenstein1957

Copy link
Copy Markdown
Collaborator Author

Closing as folded into #1340. These two conflicted textually in codecarbon/core/api_client.py — both inserted a module-level block immediately after get_datetime_with_timezone() and both touched add_emission — so one had to rebase onto the other regardless. Since both are small and both are about the API client behaving correctly, they are now one review instead of a review, a rebase and a second review. The measurement-timestamp fix and its test came across unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant