Warranty Desk

Seven parts. Every arrow is a managed identity and a role assignment. Nothing holds a key.

In this order. If a check fails, stop. A mistake in step 4 shows up in step 6 as a failure you cannot explain.

1. Foundry project and model

   Create a Foundry resource and project in a region with gpt-5.4-mini quota. Deploy the model, Standard SKU, capacity 10. Copy the project endpoint.

   az login
   azd auth login
   azd ext install microsoft.foundry
   pip install "azure-ai-projects>=2.3.0" azure-identity python-dotenv

   The model answers in the Foundry playground.
2. Orders data

   Cosmos DB, serverless. Database retail, container orders, partition /customerId. Load the twenty sample orders from the project repo.

   A query for customerId = "c-0007" returns three orders.
3. Two tools as Azure Functions

   Python Function App, two routes: GET /orders?customerId= and GET /warranty?orderId=. Warranty returns { "covered": true, "endsOn": "2027-03-14" }. System-assigned identity with Cosmos DB Built-in Data Reader. Publish an OpenAPI 3 document; the agent reads it to learn what the tools do, so the descriptions are prompts.

   curl both routes with an Entra token. Confirm the date for order o-1042 by hand.
4. Policy documents

   Upload the three policy PDFs (returns, extended warranty, exclusions) to a file search vector store. Read the exclusions PDF yourself; it has the clause the agent will get wrong later.

   The vector store shows three files indexed.
5. The agent

   Scaffold a hosted agent from the Agent Framework sample. Add instructions, the file search tool, and the OpenAPI tool pointing at your Function App. Run it locally first.

   azd ai agent init -m "https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/agent-framework/responses/01-basic/azure.yaml" --deploy-mode code
   azd ai agent run      # local
   azd deploy            # to Foundry Agent Service

   The instructions must say: check the order before stating any date; quote policy only from the documents; if a tool fails, say so and offer a person; never guess.

   "Is order o-1042 still covered?" produces a tool call and the date you computed in step 3.
6. Memory

   Create a memory store, then attach the memory_search_preview tool to the agent. The store needs a chat model and an embedding model deployment, and creating one needs an opt-in header, Foundry-Features: MemoryStores=V1Preview. Set the tool's scope and send the customer's id as x-memory-user-id on every call.⁴

   Two things, in this order. First: memory is region-gated and is not in the tool-by-region table. It failed for us in East US with "Memory Tool is not supported in this region", after the store and the agent had both been created without complaint, and worked in West US 3. Second, and this is the one that matters: sign in as one customer, tell the agent something private, then ask for it as a different customer. A second customer must get nothing. When we ran this, they got everything. Do not go on until you have run this test yourself and seen the result with your own eyes.
7. Identity

   Streamlit signs the customer in with Entra ID and calls the agent as that customer. The agent's platform identity gets Foundry User on the project. The Function App accepts only the agent's identity. Remove every key you used while getting things working.

   grep -ri key .env* returns nothing. An anonymous call to the Function App gets a 401.
8. Observability

   Connect Application Insights, turn on tracing, open the Agent Monitoring Dashboard. Run ten conversations; stop the Function App for three of them.

   You can point at one trace and show the tool error, the model's next step, and what the customer was told.
9. Front end

   Streamlit on Container Apps with a system-assigned identity. Sign in, start a conversation, stream replies, show a "handed to a person" banner on the escalation phrase.

   Someone who has never seen the project asks about their warranty from a phone and gets the right date.
10. Tear it down

    azd down

    Then check the resource group. azd down only removes what azd created; the Cosmos account and Function App are yours to delete.

    The resource group is empty. Today's cost is under $5.

Cause each one on purpose. Tick it when you can describe the failure and the fix from memory.

• Silent tool error Stop the Function App, ask for a date. Early instructions let the model invent one. Fix: forbid dates without a tool result, make error responses unmistakable, confirm in the trace.
• Confident misquote Ask if water damage is covered. The exclusions PDF says no, except for one product line. Watch the agent miss the exception. Fix: chunking, and an instruction to quote the clause, not summarise it.
• Memory bleed The one that would end a pilot. We built this exactly as documented, scope {{$userId}} plus an x-memory-user-id header per customer, and customer B was told customer A's name, order number and what had happened in her kitchen. Isolate the cause before trusting any of it: give a second agent a hard-coded scope and confirm it is blind to the first customer. That worked for us, so scope itself is sound and it was the per-user template that never took effect. Until your own two-customer test comes back clean, treat the template as unproven and set an explicit scope per customer.⁴
• Schema drift Rename orderId to order_id in the Function but not the OpenAPI document. The agent gets a 400 it cannot explain. Fix: generate the document from the code.
• The missing role Owning the subscription does not grant Foundry data-plane access, and the refusal is a 403 with an empty body, no message at all. Assign Foundry User at project scope, then wait: it took about a minute to take effect for us, and retrying too early looks exactly like still being denied.
• The 500 that goes away One tool call came back server_error, "the server had an error while processing your request", and the identical call then succeeded three times in a row. Transient failures are normal at this layer: retry with backoff before spending an hour debugging a request that was never broken.
• Cost surprise Send a 40-turn conversation and read the token count in the trace, before and after memory summarises. Decide, with numbers, whether history lives in the prompt or in memory.

Hosted agent, or your own container?

Hosted gives you the dashboard, per-user memory and a platform identity for free, in preview. Your own loop on Container Apps gives control and no preview risk, and you build memory and tracing yourself. This build chooses hosted because observability is the point.

OpenAPI tools, or an MCP toolbox?

OpenAPI is the short path when the APIs already exist. A toolbox over MCP wins once several agents share tools. Project 04 makes that move.

File search, or Azure AI Search with Foundry IQ?

Three PDFs do not need a search service. Three thousand do, and so does anything with document permissions. Project 02 makes the switch.

gpt-5.4-mini, or the larger model?

Lookups and quotations. The small model does them at a fraction of the cost. Test the larger one on the exclusions clause only, and prove it in Project 03.

Managed memory, or history in the prompt?

Prompt history is simple, grows linearly in cost, and leaks nothing between customers because there is nothing shared to leak. Managed memory summarises, persists across sessions, and is in preview: region-gated, behind an opt-in header, and in our run its per-user scoping did not isolate. For a support desk holding order numbers and names, prompt history until your own isolation test passes is the defensible choice, and "I tested it and it failed, so I did not ship it" is a better interview answer than a feature you never checked.

Figure 2, redrawn by you

From memory, on paper, then compared.

One trace screenshot

The tool-failure conversation: the error, the model's next step, the final message.

The repo

Instructions file, OpenAPI document, Bicep for Functions and Cosmos, and a README that opens with the trade-offs, not the install steps.

One sentence for the résumé

"Built and deployed a customer support agent on Microsoft Foundry with tool calling, per-user memory, managed-identity auth and end-to-end tracing; tested failure handling for downstream API outages."