Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Validating a Foundry Hosted Agent Sample · microsoft | DecimalAI

testingby microsoft

Validating a Foundry Hosted Agent Sample

microsoft/foundry-hosted-agent-validation

Source microsoft/agent-framework ↗License MIT

Step-by-step process for validating a Python Foundry hosted agent sample (under python/samples/04-hosting/foundry-hosted-agents/) end to end — running it locally (native runtime and `azd ai agent run`) and after deploying it to an Azure AI Foundry project with `azd`. Use this when asked to validate a hosted agent sample.

SkillScore95Maintained

Automatic

AI Engineer pack

Last verified 8/29/2026

What it changes

Passed 20 of 21 test cases with the skill, and 10 of 21 without it.

Without skill45%

With skill95%

Checks it passed that the model alone missed

Advises against running azd provision.
Recommends azd deploy instead of azd provision.
Passes -m pointing to the agent manifest YAML file.

and 21 more in the full report

11 fixed10 passed either way1 still failing0 broke

Pass-rate lift: +50 pp
Tokens: +106%
Agent turns: 0%
Response time: -45%

Model gemini-3.6-flashMeasured 8/29/2026v1 · latest is v221 of 22 cases passed with skill · 21 comparable

Tokens, turns and time describe resource use, not a dollar price.

This benchmark measured an earlier version of the skill.

Use this skill

Save it into your project's skills folder. Claude Code picks it up in its next session.

curl -fsSL --create-dirs -o .claude/skills/microsoft-foundry-hosted-agent-validation/SKILL.md https://app.decimal.ai/s/microsoft-foundry-hosted-agent-validation/SKILL.md

Want it pinned and updatable? pip install "decimalai>=0.15", then decimalai skills add microsoft-foundry-hosted-agent-validation, and start Claude Code with claude --plugin-dir "$(git rev-parse --show-toplevel)/.agents"

1 · One-time — install the CLI (docs ↗)

pip install "decimalai>=0.15"

2 · Get the skill — writes .agents/skills/microsoft-foundry-hosted-agent-validation/ and pins it in .decimal/skills.lock

decimalai skills add microsoft-foundry-hosted-agent-validation

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/microsoft-foundry-hosted-agent-validation/SKILL.md https://app.decimal.ai/s/microsoft-foundry-hosted-agent-validation/SKILL.md

Raw file: /s/microsoft-foundry-hosted-agent-validation/SKILL.md ↗ · machine-readable summary at /s/microsoft-foundry-hosted-agent-validation.json ↗

Safety checks: 3 of 3 passed

Code scan CleanIntent PassedContent Clean

See it in action

One of 11 test cases the skill fixed. Same model, same prompt, with and without the skill.

1 of 11

Prompt

After deploying a hosted agent to Azure AI Foundry, which CLI command reveals the environment variables registered in the deployed agent's active definition?

What was checked, without and with the skill
Checked	Without skill	With skill
Directs executing azd ai agent show <agent-name> --output json.	Missed	Passed

case-18 · gemini-3.6-flash · 8/29/2026 · answers are capped and redacted

What's inside

The instructions your model reads when the skill loads.

SKILL.md303 linessingle file

Validating a Foundry Hosted Agent Sample

A hosted agent sample is "validated" when it passes three independent checks, plus cleanup:

Local, native runtime — run the sample's own entry point (python main.py) and invoke it over HTTP.
Local, via azd ai agent run — the azd local dev loop.
Deployed — azd deploy to Foundry, then invoke the hosted agent.

Each check must succeed for single-turn and multi-turn (session / previous_response_id) conversation. Always end with cleanup (delete the deployed agent, remove the temp azd project, restore the sample dir).

Read the sample's own README.md and the parent .../foundry-hosted-agents/README.md first — they define the run/deploy commands and any sample-specific payload. This skill captures the process and the non-obvious gotchas the READMEs don't.

Inputs you need before starting

Gather these:

Foundry project endpoint, e.g. https://<account>.services.ai.azure.com/api/projects/<project>.
Foundry project resource id (for non-interactive azd ai agent init): /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<account>/projects/<project>. Find it with az cognitiveservices account list + the project name.
A real, deployed model name in that project (e.g. gpt-4.1-mini). This is often different from the model id in agent.manifest.yaml — the actual deployment name wins.
An existing ACR to reuse for deployment (login server, e.g. myacr.azurecr.io). Reusing one avoids azd provision creating resources.
Whether a like-named agent already exists in the project (remove it first for a clean validation — see below).

Tooling / auth

Track record

Benchmark

20 / 21

test cases passed, vs 10 / 21 without

Adoption

Users: 2
Times used: 25

Safety and source details

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v5 — 10/8/2026

View 15 scanner checks

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

microsoft / agent-framework

python/scripts/sample_validation/skills/foundry-hosted-agent-validation/SKILL.md

View on GitHub ↗

License MITImported 10/8/2026Version v2

More skills from the same source and nearby in the registry, with their headline benchmark lift.

Ready to supercharge your AI agents?

See it in action

What's inside

Validating a Foundry Hosted Agent Sample

Inputs you need before starting

Tooling / auth

Track record

Benchmark

Adoption

SkillSafety

Source

Related skills