Add azure.ai.loom extension for Foundry experiment tracking - #9795
Add azure.ai.loom extension for Foundry experiment tracking#9795HarshaVardhanBabu (babu-namburi) wants to merge 8 commits into
Conversation
Add authenticated CLI coverage for run inspection, agent traces, OTLP ingestion, and W&B compatibility APIs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: 7 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
Thank you for your contribution HarshaVardhanBabu (@babu-namburi)! We will review the pull request and get back to you soon. |
There was a problem hiding this comment.
Pull request overview
Adds Foundry experiment-tracking operations to the azure.ai.projects extension.
Changes:
- Adds 18 run, ingestion, trace, span, and W&B API commands.
- Adds bearer/API-key authentication and Foundry request handling.
- Adds tests and user documentation.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
README.md |
Documents experiment-tracking usage. |
internal/exterrors/codes.go |
Adds experiment error codes. |
internal/experimenttracking/client.go |
Implements the authenticated HTTP client. |
internal/experimenttracking/client_test.go |
Tests client behavior. |
internal/cmd/root.go |
Registers new command groups. |
internal/cmd/experiment.go |
Implements experiment commands. |
internal/cmd/experiment_test.go |
Tests command helpers and registration. |
docs/environment-variables.md |
Documents API-key authentication. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move experiment tracking commands under azd ai loom run while preserving the existing azure.ai.projects command surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exercise every azd ai loom run surface with configurable project, run, trace, and payload inputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Create temporary OTLP protobuf and JSON fixtures so the smoke test exercises ingestion and W&B APIs without caller-provided payload files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: 5 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Preserve authoritative run targeting, surface OTLP partial success, escape dot segments, and complete endpoint fallback documentation and coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 32 changed files in this pull request and generated 3 comments.
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
cli/azd/extensions/azure.ai.loom/internal/cmd/project_endpoint.go:28
url.Parseerrors include the original URL, so a malformed endpoint containing user info or a sensitive query value can echo that secret in this structured error. Return a generic parse failure here; the existing suggestion already tells the user how to correct it.
fmt.Sprintf("invalid project endpoint URL: %s", err),
cli/azd/extensions/azure.ai.loom/internal/cmd/run.go:337
strconv.ParseFloatacceptsNaNand infinities, and each bypasses this ordering check. The request then always fails injson.Marshalas an unsupported value and is reported as an internal request error; reject non-finite--min/--maxvalues as invalid parameters before building the body.
if maxStep < minStep {
return invalidExperimentParameter("max", "--max must be greater than or equal to --min")
}
Prevent credential-bearing redirects, harden validation errors, and add command-level HTTP contract coverage for every experiment tracking surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed the latest review feedback in c067118:
|
There was a problem hiding this comment.
🟡 Changes recommended
Endpoint path safety and cancellation classification issues remain unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
cli/azd/extensions/azure.ai.loom/internal/exterrors/errors.go:72
- A canceled request reaches this fallback and is converted to an internal structured error, which removes the
context.Canceledchain and makes Ctrl+C appear as an unexpected failure. Check cancellation before this fallback and return a user-cancellation error, asazure.ai.routines/internal/exterrors/errors.go:93-95does.
- Files reviewed: 32/33 changed files
- Comments generated: 2
- Review effort level: Balanced
| path := strings.TrimRight(endpoint.EscapedPath(), "/") | ||
| return fmt.Sprintf("https://%s%s", host, path), nil |
| return exterrors.Validation( | ||
| exterrors.CodeInvalidExperimentPayload, | ||
| message, | ||
| "provide a valid JSON object using the documented request schema", |

Fixes #9796
Summary
azure.ai.loomextension with all 18 Foundry experiment-tracking API surfaces underazd ai loom run.trace,span,ingest, andwandbcommand groups.azure.ai.projectscommand and provider behavior by removing experiment commands from that extension.AZURE_AI_PROJECT_API_KEYauthentication.CLI hierarchy
Validation
go build ./...andgo test ./...fromcli/azd/extensions/azure.ai.loom.go test ./...fromcli/azd/extensions/azure.ai.projects.golangci-lint run ./...and extension cspell checks.mage checkDependencyVersions.azd x buildforazure.ai.loom, including artifact packaging and local installation.fdp-command-job-westus2-projproject using project-key authentication.Smoke testing
Run the PowerShell smoke-test script from the Loom extension directory:
The script builds and installs the extension, generates temporary synthetic OTLP protobuf, agent-trace, GraphQL, and W&B file-stream payloads, exercises every CLI surface, and removes the fixtures afterward. Use
-SkipWriteOperationsto run only inspection, trace, and span checks.Release handoff
This PR adds repository build and release wiring for
azure.ai.loom. Publishing the extension to the official registry remains a separate release/registry step.