Care Gap Report

Seven parts. The agent only ever sees a patient with the name taken out first.

In this order. Steps 3 and 4 are the ones a demo skips and a regulator asks about. Use synthetic data throughout; nothing here should ever touch a real patient record.

1. Foundry project and model

   Create a Foundry project, deploy gpt-5-mini, confirm it against the Learn page for the day; naming and quota behave the same as Project 01.

   The model answers in the playground.
2. Synthetic patient data

   Build twenty fake patients in Azure SQL: age, sex, condition flags, and a short free-text visit note per patient that sometimes mentions a screening date and sometimes a name, a relative, or a phone number, on purpose.

   Note for patient row 12:
   "Seen for follow-up. Pt states her sister Maria had a BRCA-positive
   result in 2024. Last mammogram was March 2023 per pt report.
   Call back number on file: patient's own, not a relative's."

   Twenty rows, each note between 30 and 80 words, at least six containing a name that is not the patient's.
3. Measure documents

   Write three short PDFs, one per quality measure: breast cancer screening (mammogram within 27 months, ages 50 to 74), colorectal cancer screening (within 10 years for a colonoscopy, ages 45 to 75), and diabetes HbA1c testing (within 12 months for anyone with a diabetes flag). Upload to a file search vector store.

   The vector store shows three files indexed.
4. Redact before anything else runs

   Create an Azure AI Language resource. Run Text PII detection over every note. Store the redacted note next to the structured fields; store the original note and the name-to-pseudonym mapping in a separate table with its own, tighter role assignment.

   Pull up a redacted note and confirm every name, phone number, and date of birth is replaced with a placeholder. Confirm the mapping table is not reachable by the identity the agent runs as.
5. Extract the clinical facts

   Run Text Analytics for health over the same redacted notes. Pull out any mentioned screening type and date as structured entities, and merge them with the structured fields from step 2.

   For patient row 12, the extracted entity is a mammogram date, March 2023, attached to the pseudonymous ID, with no name attached anywhere in this table.
6. The agent

   Create an agent with the file search tool over the measure specs. Instructions: for each patient, using only the structured fields and extracted entities provided, decide per measure whether the patient is due, overdue, or not applicable, and cite the exact date or value used. Never infer a screening from context that is not an extracted date or a structured field.

   Run it against patient row 12 for the breast cancer measure. It should say overdue, using March 2023 as the last mammogram, since more than 27 months have passed, and it should not mention Maria.
7. Audit every decision

   Log each agent decision to an append-only table: pseudonymous patient ID, measure, verdict, the date or value cited, and a timestamp. Nothing in this table is ever a real name.

   After a full run, the audit table has one row per patient per measure, and you can answer "why did you flag row 12" by reading one row.
8. Human review, then re-identification

   Build the population report from the audit table: counts and percentages by measure, no names. Give the compliance officer role a separate view that, for a specific pseudonymous ID they select, joins back to the mapping table from step 4 and shows the real name, only for that one row, only after they select it.

   A second, lower-privileged role can see the population report and cannot open a single record. The compliance officer role can, one row at a time, and that access is itself logged.
9. Identity

   Three separate roles: the agent's managed identity (structured and extracted data only, no mapping table), the reviewer role (population report only), the compliance officer role (population report plus one-row re-identification). Remove any key used while building.

   grep -ri key .env* returns nothing. Signing in as the reviewer role and attempting to read the mapping table fails.
10. Tear it down

    azd down

    Then check the resource group by hand, including the Azure AI Language resource, which azd down may not own if you created it separately.

    The resource group is empty. Today's cost is under $8.

Cause each one on purpose. This is the section a compliance officer will actually ask you about.

• The agent infers a date from context Feed it a note that says "screened again this spring" with no extracted date, and ask for a verdict. Early instructions let it guess a date and call the patient current. Fix: the agent may only cite an extracted entity or a structured field, never prose; verify in the audit log that every citation is a real value, not free text.
• Redaction misses a nickname A note that calls the patient "Patty" instead of her chart name may not get redacted. Fix: no redaction pass is perfect; add a second, narrower regex pass for the fields your organization cares most about, and manually sample a percentage of redacted notes before trusting the pipeline at scale.
• The reviewer role reaches the mapping table A permissions mistake in step 9 lets the lower-privileged role join to real names. This is the single worst failure in the project; test it by trying to break your own access control, not by reading the role assignment and assuming it worked.
• The measure logic drifts Change the breast cancer measure's lookback window in the PDF from 27 to 24 months and rerun. If the agent's verdicts do not change, it cached an old answer or is not actually reading the document per run. Fix: confirm the file search tool is called on every request, not memorized in the conversation.
• The missing role Deploy without Foundry Project Manager and watch the error. Same failure as Project 01, worth seeing twice.
• Batch cost surprise Run all twenty notes through Text Analytics for health and PII detection and read the per-call cost. At clinic scale, thousands of notes a day, this number is the one a CFO asks about before the accuracy number.

Agent judgment, or a deterministic rules engine?

A rules engine is exact and fully auditable for numeric criteria like age bands and lookback windows, and brittle every time a measure's wording changes. This build lets the agent read the measure PDF and reason in language, but the age and date-window math is done in code, in step 6's instructions, not left to the model to compute.

Redact first, or analyze raw text and redact the output?

Redacting before the model ever sees the text means PHI never appears in a prompt, a model log, or a trace. The cost is losing context, and it is bigger than a name: in our run the redactor masked the word "sister" as well as "Maria", so "her sister was BRCA-positive" came out as "her ****** ***** was BRCA-positive" and the family history was gone. This build always redacts first and accepts that loss, because the alternative puts PHI in front of a model call.

Review every gap, or only the borderline ones?

Reviewing every flagged gap is safest and slowest. Once you have measured the agent's precision against a labeled set, the way Project 03 measures an agent, sampling only low-confidence verdicts is a defensible middle ground. This build reviews everything, because it has no measured precision yet.

Synthetic data here, or a real de-identified export?

This project is built entirely on fabricated patients so anyone can run it without a business associate agreement or an institutional review. A real deployment on real PHI is a different undertaking: it needs a signed BAA with Microsoft, use of only HIPAA-eligible services, and your organization's own compliance sign-off, not this page.

HIPAA, or a non-US reader's framework?

HIPAA is a US law and this page is written for it. Under GDPR, health data is "special category data" under Article 9, needing an explicit legal basis and likely a Data Protection Impact Assessment before this kind of processing starts. The architecture, redact first, log everything, gate re-identification behind a human, travels across frameworks. The specific legal basis you are relying on does not.³

A redacted-note pair

The original synthetic note next to its redacted version, side by side.

One audit log row

A single decision, the cited value, and the timestamp, screenshotted.

The repo

Instructions file, the three measure PDFs, the role assignment script, and a README that states plainly this project uses synthetic data only.

One sentence for the résumé

"Built a document-grounded quality-measure agent that de-identifies clinical text before any model call, cites every decision to a source value, and gates patient re-identification behind human approval and an audit log."