Define how to monitor an AI agent and what evidence it must log
Monitoring metrics, review routines, alerts, and the evidence an AI agent should produce for every task it completes.
The prompt
Help me plan how to monitor an AI agent once it is running and what evidence it should produce.
What the agent does: [TASKS]
Its objective and success measures: [e.g. "resolve password reset tickets in under 5 minutes with no wrong resets"]
Volume: [e.g. "200 tasks a day"]
Who will monitor it and how much time they have: [ROLE, e.g. "ops lead, 30 minutes a week"]
Tools available for logs and dashboards: [e.g. "Excel", "Power BI", "the platform's built-in logs"]
Return:
1. Evidence per task - what the agent must record each time: inputs received, sources used, steps and tools called, actions taken, output, confidence or uncertainty, approvals requested, escalations, and errors. Mark what a reviewer needs to reconstruct a decision.
2. Metrics - a table: Metric | Definition | Target | Alert threshold | Check frequency. Cover quality (accuracy from sampled reviews), speed, escalation rate, approval rejection rate, human rework, cost, complaints, and drift over time.
3. Review routine - daily, weekly, and monthly checks, including how many outputs to sample by hand and how to pick them.
4. Alerts - what triggers an immediate alert and who receives it.
5. Pause criteria - conditions under which the agent is paused until reviewed.
6. A simple weekly report template for the agent's owner.
Do not set targets as facts; propose starting values and mark them [SET BASELINE AFTER PILOT]. Accuracy must be measured by human review of samples, not by the agent's own self-assessment.
Highlighted [brackets] are blanks to replace with your own details.
Fill in the blanks and use this promptMore Working with AI prompts
What to fill in
- TASKS
- e.g. "200 tasks a day"
- ROLE, e.g. "ops lead, 30 minutes a week"
- e.g. "Excel", "Power BI", "the platform's built-in logs"
- SET BASELINE AFTER PILOT
How this prompt is built
This prompt follows the Few-Shot structure (Few-shot prompting). A few-shot prompt includes one or more worked examples of the input and the output you want, so the model can copy the pattern instead of guessing it.
Related prompts
- Build an AI adoption roadmap for my team or functionA phased plan for adopting AI in sales, marketing, HR, finance, legal, engineering, support, or operations — with use cases, guardrails, and metrics.
- Plan what to do when AI gets something wrong at workAn escalation path for AI errors: how to spot, contain, report, fix, and learn from mistakes made with AI tools.
- Run a risk check on an AI-assisted workflow before rolloutBuilds a risk register for an AI workflow covering accuracy, data privacy, bias, security, compliance, and over-reliance, with controls.
- Create a personal AI capability roadmap for my roleA staged plan to grow from occasional AI user to someone who builds and directs AI workflows in your job.