Skip to content

Fix kernel async statement telemetry handle - #923

Open
jay-xiao446 wants to merge 1 commit into
mainfrom
jay/kernel-async-telemetry-handle
Open

Fix kernel async statement telemetry handle#923
jay-xiao446 wants to merge 1 commit into
mainfrom
jay/kernel-async-telemetry-handle

Conversation

@jay-xiao446

@jay-xiao446 jay-xiao446 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Use the retained owning kernel async handle for the first in-process async result fetch so ExecuteStatementAsync telemetry is finalized
  • Fall back to attach-by-id for re-fetch, fresh cursor, and missing-handle cases
  • Update unit/e2e coverage and lifecycle comments for the new ownership behavior

Testing

  • .venv/bin/python -m pytest tests/unit/test_kernel_client.py -q
  • python3.10 -m py_compile src/databricks/sql/backend/kernel/client.py src/databricks/sql/backend/kernel/result_set.py tests/unit/test_kernel_client.py tests/e2e/test_kernel_backend.py

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — the owning-handle-first / attach-by-id-fallback change is correct: the claim is check-and-set under _async_handles_lock, cleanup is consistent across close_command/close_session, and the new unit + e2e tests cover the main paths. One low-severity note on an unclear telemetry-finalization edge when result-set construction fails after a successful await_result().

stream = handle.await_result()
except Exception as exc:
if uses_owning_handle:
with self._async_handles_lock:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The owning-handle failure path only discards _async_result_stream_started when await_result() raises. If await_result() succeeds (marker stays set) but the subsequent KernelResultSet.__init__arrow_schema() raises and is re-wrapped, the guid remains marked as started. A later retry then takes the attach-by-id (no-op telemetry) branch.

Whether this loses the ExecuteStatementAsync telemetry row depends on when the kernel finalizes it: if finalization happens when await_result() returns, this is harmless (telemetry already committed). If finalization only completes once the result stream is drained, the telemetry is lost on this retry because the owning handle is never reused. The PR's own comments ("first in-process result stream", "clear the claimed marker so a retry can still use the telemetry-bearing owning handle") are ambiguous on this point, and the added test_get_execution_result_owning_handle_failure_can_retry_owning_handle only exercises the await_result()-raises case, not the construct-failure-after-await case. Worth confirming the finalization semantics and, if drain-based, discarding the marker on the construction-failure path too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant