Add hosted voice agent support - #9697
Conversation
|
Azure Pipelines: Successfully started running 1 pipeline(s). 20 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Final validation statusAll automated CI checks are green, including:
Manual regression validation also passed:
Review status:
The PR remains Draft and therefore still requires normal human review/approval before merge. |
Synced with latest #9634 Voice API surfacePR #9697 now contains the latest #9634 head ( Hosted Voice adaptation completed:
Validation:
Updated private package: SHA256: Private bundle commit: |
Final standard-manifest regression result — PASSHosted Voice init now matches the existing Hosted Agent / Scenario 1: empty folder + manifest
Scenario 2: existing source/project
CI / quality
Private package
SHA256: The public GitHub sample manifest URL remains pending publication. The private package contains the validated standard manifest and sample source. |
Renamed private test packageThe final private test ZIP now includes the PR number in its filename:
SHA256:
Checksum file:
The previous non-PR-numbered ZIP path has been removed. |
Updated rigorous private test package
SHA256:
The Test Plan now includes:
Package validation:
|
20ea3c8 to
f9eca38
Compare
|
Azure Pipelines: Successfully started running 1 pipeline(s). 20 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Adds hosted voice wrappers to the Azure AI Agents extension, integrating them with manifest initialization, dependency ordering, deployment validation, and cleanup.
Changes:
- Adds hosted-agent voice schema and API mappings.
- Resolves and validates deployed hosted targets through
uses. - Extends initialization workflows, documentation, and tests.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
schemas/azure.ai.agent.json |
Defines hosted voice configuration. |
README.md |
Documents hosted voice usage. |
internal/project/service_target_agent.go |
Deploys and validates wrappers. |
internal/project/service_target_agent_test.go |
Tests inline round-tripping. |
internal/project/hosted_voice_target_test.go |
Tests target resolution and validation. |
internal/project/foundry_dependencies.go |
Resolves deployed target dependencies. |
internal/project/agent_policies_test.go |
Updates voice policy validation fixture. |
internal/project/agent_definition.go |
Maps hosted voice fields. |
internal/pkg/agents/agent_yaml/yaml.go |
Adds authoring model types. |
internal/pkg/agents/agent_yaml/parse.go |
Validates hosted voice manifests. |
internal/pkg/agents/agent_yaml/parse_voice_test.go |
Tests manifest validation. |
internal/pkg/agents/agent_yaml/map.go |
Builds hosted wrapper API requests. |
internal/pkg/agents/agent_yaml/map_voice_test.go |
Tests request serialization. |
internal/pkg/agents/agent_api/models.go |
Adds wire-model target references. |
internal/cmd/listen.go |
Deep-clones service configuration. |
internal/cmd/init.go |
Integrates hosted voice initialization. |
internal/cmd/init_test.go |
Tests manifest code configuration. |
internal/cmd/init_hosted_voice.go |
Detects compatible manifests. |
internal/cmd/init_hosted_voice_test.go |
Tests manifest detection. |
internal/cmd/init_from_templates_helpers.go |
Adds the interactive init choice. |
internal/cmd/init_from_templates_helpers_test.go |
Tests hosted voice selection. |
internal/cmd/init_from_code.go |
Generates target and wrapper services. |
internal/cmd/init_from_code_test.go |
Tests wrapper generation. |
internal/cmd/delete.go |
Cleans target pin variables. |
internal/cmd/delete_test.go |
Tests marker cleanup. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
cli/azd/extensions/azure.ai.agents/internal/cmd/init_hosted_voice.go:42
fmt.SprintacceptsvoiceLiveCompatible: trueas compatible, but the deployment mapper only copies string metadata values (agent_yaml/map.go:899-905). Such a manifest gets a wrapper, then deploys the target without this marker and the wrapper fails its post-deploy compatibility check. Normalize these marker values to strings before deployment, or reject non-string metadata during detection.
voiceCompatible := strings.EqualFold(strings.TrimSpace(fmt.Sprint(metadata["voiceLiveCompatible"])), "true")
bridgeVersion := strings.TrimSpace(fmt.Sprint(metadata["bridgeProtocolVersion"]))
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.
Suppressed comments (5)
Previously missed (3) — in code that hasn't changed since the last review.
cli/azd/extensions/azure.ai.agents/internal/project/foundry_dependencies.go:465
- This added line is 126 characters and will fail the repository's 125-character
lllcheck (cli/azd/AGENTS.md:115-126). Wrap the call before merging.
return nil, fmt.Errorf("hosted voice target service %q is deployed to a different Foundry project", targetServiceName)
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/parse.go:456
- This added line is 130 characters and exceeds the repository's 125-character Go limit (
cli/azd/AGENTS.md:115-126), solllwill reject it.
This issue also appears in the following locations of the same file:
- line 458
- line 467
errors = append(errors, "template.target_agent.service is required when model_type is 'hosted_agent'")
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go:765
- This added line is 192 characters and exceeds the repository's 125-character Go limit (
cli/azd/AGENTS.md:115-126), solllwill reject it.
return nil, fmt.Errorf("model, instructions, structured_inputs, tools, tool_choice, parallel_tool_calls, max_output_tokens, include, and handoff belong to the target hosted agent")
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/parse.go:458
- This condition is 131 characters and exceeds the repository's 125-character Go limit (
cli/azd/AGENTS.md:115-126). Split it so the enforcedlllcheck passes.
if agent.TargetAgent != nil && agent.TargetAgent.Version != "" && agent.TargetAgent.Version != "deployed" {
cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/parse.go:467
- This added line is 202 characters and exceeds the repository's 125-character Go limit (
cli/azd/AGENTS.md:115-126), solllwill reject it.
errors = append(errors, "instructions, structured_inputs, tools, tool_choice, parallel_tool_calls, max_output_tokens, include, and handoff belong to the target hosted agent")
There was a problem hiding this comment.
🟡 Changes recommended
Both documented initialization paths omit wrapper generation, and adoption can misclassify supported service shapes.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
cli/azd/extensions/azure.ai.agents/internal/cmd/init.go:2096
- The manifest-driven flow still persists only the hosted target. After this merge it falls through to
addToProject, which adds one hosted service and never creates theprompt-voicewrapper documented in the README and required by #9762. Add hosted-voice manifest detection and create the wrapper (including its project/targetusesedges) before init completes.
if hostedAgent, ok := agentManifest.Template.(agent_yaml.ContainerAgent); ok &&
- Files reviewed: 21/21 changed files
- Comments generated: 2
- Review effort level: Balanced
51644a5 to
0429bfb
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Hosted Voice initialization and validation still have blocking gaps.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 21/21 changed files
- Comments generated: 3
- Review effort level: Balanced
There was a problem hiding this comment.
🟡 Changes recommended
Adoption can misclassify referenced or legacy voice wrappers and overwrite them with hosted deployment settings.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 21/21 changed files
- Comments generated: 1
- Review effort level: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
Wrapper-level protocols currently pass schema validation but are silently discarded during deployment.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json:229
- Reject
protocolsas a target-owned field here. The schema currently accepts wrapper-level protocols, butAgentDefinitionInline.toVoiceAgentomitsProtocols, so azd silently discards the setting even though the hosted target is the component that must declareinvocations_ws/1.0.0. Add matching schema and Go validation before the inline/standalone definitions are converted.
- Files reviewed: 21/21 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Summary
Closes #9762.
Adds Hosted Voice Agent support to the
azure.ai.agentsextension while preserving the existing Hosted Agent andinvocations_wslifecycle.A Hosted Voice Agent consists of:
invocations_ws/1.0.0The target follows the same code-agent package, provision, deploy, show, and delete patterns as other Hosted Agents. The wrapper is an additional explicit
azure.ai.agentservice so azd can distinguish Hosted Voice from a genericinvocations_wsagent and developers can customize the Voice surface.Cross-team design context and open questions are documented in the Draft Spec: https://github.com/coreai-microsoft/foundrysdk_specs/pull/322
User experience
Init from a sample
azure.yamlazd ai agent init -m .\path\to\azure.yaml azd provision azd deployAfter the sample is published, the local path can be replaced by the public GitHub URL for the sample
azure.yaml.Existing Hosted Voice azd project
When
azure.yamlalready exists, init reuses the existing project configuration. It does not regenerate target/wrapper services; the existingazure.yamlremains the deployment source of truth.Sample
azure.yamltargetAgent.serviceis an azd-local reference to the Hosted Agent service that runs the developer code and receives Voice Bridge turns. During deployment, azd resolves it to the deployed target name/version and creates or updates the managed wrapper.Developers primarily edit target code. They can also configure wrapper-owned Voice settings such as audio formats, transcription, VAD, interruption behavior, output voice/locale/style/speed, modalities, storage, greeting, and avatar where supported.
Validation
azure.yamlazure.yamlproject init reuse/helprouted through the wrapper to target-ownedCommands:output