Skip to content

Part 9, Human-in-the-loop & condition checker โ€‹

Knowledge-share outline โ€‹

  1. Why HITL is more than an approval button, deciding when a human is truly needed.
  2. Risk-based escalation, confidence, risk, policy, irreversibility.
  3. What the human needs to see to make a good decision.
  4. Suggestions as evidence, not automatic truth, every suggestion carries a confidence.
  5. The package condition checker flow, driven by structured package data.
  6. Approval, rejection, correction, and the audit trail, capturing the decision and rationale.
  7. How this ties together context, memory, MCP, tools, A2A, and governance.
  8. Wrap-up and next steps.

At a glance โ€‹

Start stateEnd stateFallback baseline
Open starter/09-1-hitl-agent.py as your starting fileA condition-checker workflow that pauses for human confirmation before customer-impacting or irreversible actions, with an audit trailbaseline/09-1-hitl-agent.py

Learning goals โ€‹

  • Explain human-in-the-loop (HITL) as meaningful oversight, triggered by confidence, risk, policy, or irreversibility, not a one-click approval button.
  • Define risk-based escalation rules that decide when the workflow must pause.
  • Understand what the human needs to see to make a good decision.
  • Treat the checker's suggestion as evidence, not automatic truth.

Human-in-the-loop is more than an approval button โ€‹

A rubber-stamp "Approve?" prompt is not oversight. Meaningful HITL decides when a human is needed and gives them what they need to decide well. The workflow pauses when one of four things is true: the model's confidence is low, the risk is high, a policy demands review, or the action is irreversible or customer-impacting. Everything else can flow; these cases stop and wait for a person. The point is accountability for the decisions that matter, not friction on the ones that don't. (HITL workflow concepts come from the official documentation; see Source-backed background.)

The package condition checker โ€‹

A package arrives damaged or ambiguous. The PackageConditionChecker inspects the structured package data (status, priority, fragile flag, truck assignment). From that evidence it suggests a condition, but it never finalizes a customer-impacting outcome on its own. At Dunder Mifflin, Darryl wants the system to do the legwork and then put a clear recommendation in front of a human for the calls that count.

The suggestion is evidence, not automatic truth โ€‹

The checker's suggestion is evidence the human weighs, not a verdict. It reports a suggestion with a confidence, explicitly marked as not-yet-confirmed. We do not claim a model can reliably determine real-world package condition on its own, a person validates every consequential classification.

Lab steps โ€‹

The focus here is the agentic behaviour, not the Python. Each step below gives you the exact snippet to paste over the matching TODO stub in hitl_agent.py, plus a short explanation of why the code looks the way it does. Work top to bottom; after every paste you can re-run the command to see the workflow grow.

  1. Start from the provided scaffold. Open starter/09-1-hitl-agent.py as your working file for this part. It is a fresh condition-checker scaffold with TODO stubs for the checker and the HITL gate already in place.

    bash
    copy starter\09-1-hitl-agent.py hitl_agent.py
    bash
    cp starter/09-1-hitl-agent.py hitl_agent.py

    Expected output: hitl_agent.py is in your root folder. The TODO comments confirm which parts are not yet implemented: the condition-checker and HITL parts are marked TODO.

  2. Build the PackageConditionChecker. This is three small functions: fetch the live package data, suggest a condition with a confidence, and map that condition to a proposed action. Paste each snippet over the matching stub (fetch_package, suggest_condition, proposed_action_for).

    First, fetch_package calls the MCP server's lookup_package tool, the same server you built in Part 6, and turns its key=value text into a dict the checker can read. Notice the agent never touches the database directly; it goes through the standard MCP interface.

    python
    async def fetch_package(package_id: str) -> dict[str, Any] | None:
        """Fetch live package data via the running MCP server."""
        async with stdio_client(SERVER_PARAMS) as (read, write):
            async with ClientSession(read, write) as session:
                await session.initialize()
                result = await session.call_tool("lookup_package", {"package_id": package_id})
                text = result.content[0].text if result.content else ""
                if text.startswith("Error:"):
                    return None
                # Parse "key=value, key=value" text into a dict
                fields: dict[str, Any] = {}
                if ": " in text:  # strip the "Package PKG-XXXX: " prefix
                    text = text.split(": ", 1)[1]
                for part in text.split(", "):
                    if "=" in part:
                        k, v = part.split("=", 1)
                        fields[k.strip()] = v.strip()
                return fields

    Next, suggest_condition inspects the evidence and returns a category with a confidence: it is a suggestion, never a verdict. The ordering matters: the most specific, highest-risk signals are checked first so a clear "missing label" never gets downgraded to a vague "unclear".

    python
    def suggest_condition(package_data: dict[str, Any]) -> tuple[str, float]:
        """Suggest a condition and confidence from available package fields.
    
        Returns (condition, confidence) where condition is one of VALID_CONDITIONS.
        """
        raw     = " ".join(str(v) for v in package_data.values()).lower()
        status  = package_data.get("status", "").lower()
        fragile = str(package_data.get("fragile", "false")).lower() == "true"
        truck   = str(package_data.get("truck", package_data.get("truck_id", ""))).strip().lower()
    
        if "missing label" in raw or "missing_label" in raw:
            return "missing label", 0.95
        if "wrong address" in raw or "wrong_address" in raw:
            return "wrong address", 0.90
        if status == "damaged" or "damag" in raw:
            return "damaged", 0.88
        if status == "returned":
            return "needs inspection", 0.80
        if fragile and truck in ("none", ""):
            return "unclear", 0.72  # fragile with no truck assigned: ambiguous
        if status in ("order_created", "packaged", "ready_for_shipping", "shipped", "delivered"):
            return "OK", 0.96
        return "unclear", 0.60

    Finally, proposed_action_for maps a condition to the action the workflow would take: this is what a human will approve, reject, or correct later.

    python
    def proposed_action_for(condition: str) -> str:
        """Return the standard proposed action for a condition category."""
        return {
            "OK":               "proceed-normally",
            "damaged":          "mark-damaged + queue-customer-followup",
            "unclear":          "schedule-inspection",
            "missing label":    "hold + queue-customer-followup",
            "wrong address":    "hold + queue-customer-followup",
            "needs inspection": "route-to-inspection-bay",
        }.get(condition, "schedule-inspection")

    Keep the MCP server and backend running

    The condition checker launches your Part 6 MCP server (mcp_server.py) for you over stdio, and that server reads live data from the package-this backend API. Before running the command below, make sure both are ready:

    • mcp_server.py is in your root from Part 6 (if you skipped Part 6, copy it: copy baseline\06-1-mcp-server.py mcp_server.py).
    • The backend API is running at http://localhost:8000. Start it from your fork with .\scripts\dev.ps1 (Windows) or ./scripts/dev.sh (macOS / Linux). See Running the app for Docker and other options.

    A Connection closed or package ... not found error usually means one of these two is missing.

    bash
    python hitl_agent.py --check-condition --order PKG-2024-009
    bash
    python hitl_agent.py --check-condition --order PKG-2024-009

    Expected output: a suggestion such as Order PKG-2024-009 -> suggested: damaged (confidence 0.88), not confirmed, with no claim that any status was changed.

  3. Confirm the condition categories. The allowed outcomes are already declared near the top of the file: you only need to confirm they are complete. No edit is needed unless one is missing.

    python
    VALID_CONDITIONS = frozenset({
        "OK", "damaged", "unclear", "missing label", "wrong address", "needs inspection",
    })
    HIGH_RISK_CONDITIONS = frozenset({"damaged", "missing label", "wrong address"})

    Expected output: --check-condition only ever returns one of these categories (or escalates when it cannot choose confidently).

  4. Define the escalation rules. This is the heart of meaningful HITL: it decides when the workflow must pause. Paste this over the escalation_reason stub. It returns the first rule that fires (or None when it is safe to proceed), so the reviewer always sees a single, clear reason.

    python
    def escalation_reason(
        condition: str,
        confidence: float,
        proposed_action: str | None,
    ) -> str | None:
        """Return the first escalation rule that fires, or None if safe to proceed."""
        if confidence < CONFIDENCE_THRESHOLD:
            return (
                f"Rule 1: confidence {confidence:.2f} is below threshold "
                f"{CONFIDENCE_THRESHOLD} - human confirmation required"
            )
        if condition in HIGH_RISK_CONDITIONS:
            return (
                f"Rule 2: condition '{condition}' is high-risk - "
                "human review required before any downstream action"
            )
        if proposed_action and any(kw in proposed_action for kw in CUSTOMER_IMPACTING_KEYWORDS):
            return (
                f"Rule 3: proposed action '{proposed_action}' is customer-impacting or "
                "irreversible - human confirmation required"
            )
        return None

    See Escalation rules below for the reasoning behind each rule.

  5. Wire the confirmation gate. Now connect the pieces: compute the escalation reason, and when a rule fires (and an action is proposed), pause and emit the review payload instead of acting. Paste this over the # TODO (step 5) block at the bottom of cmd_check_condition. It also wires the decision path you complete in step 7.

    python
        esc = escalation_reason(condition, confidence, action if propose_action else None)
    
        if not propose_action and not decide:
            return
    
        if propose_action and not decide:
            if esc:
                print(f"\nAwaiting human confirmation: proposed: {action}. No change made yet.")
                print(f"Escalation reason: {esc}")
                payload = build_review_payload(
                    package_id, package_data, condition, confidence, esc, action
                )
                print("\nHuman review payload:")
                print(json.dumps(payload, indent=2))
            else:
                print(f"\nNo escalation required. Safe to proceed with: {action}")
            return
    
        if decide:
            audit = record_decision(
                package_id=package_id,
                original_condition=condition,
                decision=decide,
                corrected_to=corrected_to,
                by=decided_by,
                note=note,
                action=action,
            )
            print("\nAudit artifact:")
            print(json.dumps(audit, indent=2))
    bash
    python hitl_agent.py --check-condition --order PKG-2024-009 --propose-action
    bash
    python hitl_agent.py --check-condition --order PKG-2024-009 --propose-action

    Expected output: the workflow pauses: Awaiting human confirmation: proposed: mark-damaged + queue-customer-followup. No change made yet. Nothing happens without a decision.

  6. Build the human review payload. The pause is only useful if the human can decide well. Paste this over the build_review_payload stub. It gathers everything the reviewer needs in one place: evidence, the suggestion and confidence, why it escalated, and the options, and marks the case confirmed: False. (Template: Human review payload below.)

    python
    def build_review_payload(
        package_id: str,
        package_data: dict[str, Any],
        condition: str,
        confidence: float,
        escalation_rule: str,
        action: str,
    ) -> dict[str, Any]:
        """Assemble the human review payload so the reviewer can decide well."""
        return {
            "package_id":        package_id,
            "evidence":          package_data,
            "suggested":         condition,
            "confidence":        confidence,
            "confirmed":         False,
            "escalation_reason": escalation_rule,
            "proposed_action":   action,
            "options":           ["approve", "reject", "correct"],
        }

    Re-run the --propose-action command from step 5: the payload is now filled in instead of empty.

  7. Capture the decision as an audit artifact. Paste this over the record_decision stub. A correction keeps both the original suggestion and the corrected value, so the audit trail never loses what the model proposed versus what the human decided.

    python
    def record_decision(
        package_id: str,
        original_condition: str,
        decision: str,
        corrected_to: str | None,
        by: str,
        note: str,
        action: str,
    ) -> dict[str, Any]:
        """Create an audit artifact capturing the human decision."""
        final_condition = (
            corrected_to if (decision == "correct" and corrected_to) else original_condition
        )
        final_action = (
            proposed_action_for(final_condition) if decision == "correct"
            else (action if decision == "approve" else "no-action")
        )
        return {
            "package_id":         package_id,
            "decision":           decision,
            "original_condition": original_condition,
            "final_condition":    final_condition,
            "corrected_to":       corrected_to,
            "decided_by":         by,
            "rationale":          note,
            "action":             final_action,
            "timestamp":          datetime.now(timezone.utc).isoformat(),
        }

    Then run both decision paths: approve, and correct (see Expected output):

    bash
    python hitl_agent.py --check-condition --order PKG-2024-009 --decide approve --by "Darryl"
    python hitl_agent.py --check-condition --order PKG-2024-006 --decide correct --to damaged --by "Darryl" --note "Crushed corner found on inspection"
    bash
    python hitl_agent.py --check-condition --order PKG-2024-009 --decide approve --by "Darryl"
    python hitl_agent.py --check-condition --order PKG-2024-006 --decide correct --to damaged --by "Darryl" --note "Crushed corner found on inspection"

    Expected output: an audit record, e.g. decision=correct, original_condition=unclear, final_condition=damaged, decided_by=Darryl, rationale="Crushed corner found on inspection", action=mark-damaged + queue-customer-followup.

Escalation rules โ€‹

Define at least three risk-based rules. When any fires, the workflow pauses and routes the case to a human instead of acting:

  1. Low confidence, the suggested condition's confidence is below a threshold (e.g. < 0.85), so a human confirms before anything downstream happens.
  2. High risk / policy, the package is high-value, flagged by policy, or classified wrong address / missing label, where a wrong call has outsized consequences.
  3. Irreversible or customer-impacting action, anything that would change a status a customer sees, or trigger a follow-up to a customer, always requires confirmation.

Even a high-confidence suggestion still escalates when rule 2 or 3 applies, confidence alone never bypasses a rule.

Human review payload โ€‹

You already built this payload in code in step 6 above. This section is the design pattern behind that code: a reusable, model-agnostic checklist of what a human needs to see to decide well. Nothing to paste here, use it as a reference when you design oversight for any HITL system, and check that your step 6 output covers every line below (evidence, the suggested classification with confidence, why it escalated, the recommended action, and the approve/reject/correct options).

Human review payload template โ€‹

When the workflow pauses, the reviewer can only decide well if they see everything at once. Use this as the checklist for what an escalated case must contain โ€” every field below should appear in the payload your code emits. A pause without these is a rubber stamp, not oversight.

  • Case โ€” what is being reviewed. Example: "Order #1037 โ€” condition check before customer notification."
  • Evidence โ€” the structured data the checker used. Example: "Order record (status=damaged, fragile=true, no truck assigned); status history note 'Damaged during sorting'."
  • Suggested classification + confidence โ€” what the checker proposes, never as fact. Example: "Suggested: damaged (confidence 0.71) โ€” suggestion only, not confirmed."
  • Why it escalated โ€” which rule fired. Example: "Low confidence (< 0.85) and a customer-impacting action would follow."
  • Recommended action โ€” what would happen on approval. Example: "Mark order damaged and queue a customer follow-up for a human to send."
  • Decision options โ€” what the reviewer may choose. Example: "Approve / Reject / Correct (set the right category)."
  • What is NOT decided here โ€” the boundary. Example: "No refund, customer message, or status change happens until a human confirms; nothing is auto-sent."

Expected output โ€‹

  • Suggested classification: one of the condition categories, with a confidence, marked not-yet-confirmed.
  • Escalation / pause: when a rule fires, the workflow holds and reports the proposed action with no change made yet.
  • Review payload: the reviewer sees evidence, suggestion + confidence, the firing rule, and the approve / reject / correct options.
  • Recorded decision (audit artifact): the choice, who made it, the rationale, and the resulting action. A correction keeps both the original suggestion and the corrected value.

Happy path (no rule fires) vs escalation path (a rule fires), side by side:

bash
# Happy path, confident, low-risk, nothing customer-impacting
python hitl_agent.py --check-condition --order PKG-2024-004 --propose-action
# -> suggested: OK (confidence 0.96); no escalation; safe to proceed

# Escalation path, high-risk condition + customer-impacting action
python hitl_agent.py --check-condition --order PKG-2024-009 --propose-action
# -> suggested: damaged (confidence 0.88); RULE FIRED: high-risk condition
# -> PAUSED, awaiting human confirmation; review payload emitted; no change made
bash
# Happy path, confident, low-risk, nothing customer-impacting
python hitl_agent.py --check-condition --order PKG-2024-004 --propose-action
# -> suggested: OK (confidence 0.96); no escalation; safe to proceed

# Escalation path, high-risk condition + customer-impacting action
python hitl_agent.py --check-condition --order PKG-2024-009 --propose-action
# -> suggested: damaged (confidence 0.88); RULE FIRED: high-risk condition
# -> PAUSED, awaiting human confirmation; review payload emitted; no change made

(This lab uses a deterministic local run, not a network simulation, so both paths are reproducible for everyone.)

Final checkpoint โ€‹

  • You added or inspected the PackageConditionChecker and saw a suggested, not-yet-confirmed classification.
  • You defined the condition categories and at least three escalation rules.
  • The workflow pauses for human confirmation before a customer-impacting or irreversible action.
  • You designed a human review payload showing what the reviewer needs.
  • You captured an audit artifact (decision + rationale), and ran a happy path and an escalation path.

See How checkpoints work.

Fallback baseline โ€‹

Try the build yourself first, then copy the fallback if you run out of time:

bash
copy baseline\09-1-hitl-agent.py hitl_agent.py
bash
cp baseline/09-1-hitl-agent.py hitl_agent.py

The baseline includes the completed PackageConditionChecker with the escalation rules, the human-confirmation gate, the review payload, and the audit artifact, so you can adopt it and still finish the day wrap-up reflection.

Apply it: De LeesLantaarn โ€‹

Apply it to your own solution

For De LeesLantaarn, which actions truly need a human in the loop? A member borrowing an in-stock book should flow without friction, but waiving a fine, approving an age-restricted title, or overriding a subscription quota are exactly the customer-impacting, sometimes irreversible calls that should pause for a librarian. Write two or three escalation rules for the library, and decide what evidence the review payload must show (member history, the title's age category, the current backorder state) so the librarian can decide well.

Troubleshooting โ€‹

  • The workflow never pauses, confirm the proposed action is marked customer-impacting/ irreversible and that an escalation rule covers it. More help on Troubleshooting.
  • Reviewer can't decide from the payload, the payload is missing context; add the evidence, confidence, and firing rule so the human doesn't have to look elsewhere.
  • Correction lost the original, the audit artifact must retain both the original suggestion and the corrected value.
  • Out of time, copy baseline/09-1-hitl-agent.py and move on to the reflection.

Optional stretch task โ€‹

Add a fourth escalation rule of your own (for example: escalate any needs inspection regardless of confidence), or extend the audit artifact into a short, append-only log across several cases, a small step toward real governance. Keep every consequential action behind the human gate.

Source-backed background โ€‹

  • HITL workflow: the official Microsoft Agent Framework documentation on Microsoft Learn and its official repository, for human-in-the-loop workflow concepts (pausing, approval gates, and resuming).
  • Earlier patterns: MCP from modelcontextprotocol.io (Part 6) and the Agent-to-Agent handoff built in Part 8.

For the full source list, see References.

What we will not cover โ€‹

  • A production claims system, or any automation of refunds, customer messages, or legal decisions, those stay with people.
  • HITL as a checkbox, oversight here is risk-based and meaningful, not a rubber stamp.
  • Storing personal data, the lab keeps evidence in-memory for the exercise and stores nothing; any real retention would need an explicit retention period and privacy boundary.

Day wrap-up reflection โ€‹

You built the whole arc, and this final workflow uses all of it:

  • Context engineering (Part 2), the agent works from the right package/order context.
  • Memory (Part 3) and Specs & personas (Part 4), Darryl's needs and prior decisions shape the workflow.
  • MCP (Parts 5 & 6), the package tools are served over a standard interface.
  • Tool calling (Parts 7 & 8), the agent and the native condition checker call tools to get facts.
  • Agent-to-Agent (Part 8), bounded delegation hands specialist work off with an explicit contract.
  • Human-in-the-loop (Part 9), meaningful oversight pauses for a person when it matters, with an audit trail for governance.

Each pattern solves a specific problem, reach for the smallest one that does the job, and keep a human in the loop for the decisions that carry real-world consequences. Keep exploring with the References.


Previous: Part 8, Native tools & A2A handoff ยท Next: References