checklist
AI chatbot change approval checklist for small teams
A practical change approval checklist for customer-facing AI chatbots, covering change types, risk tiers, approval records, source changes, prompt changes, tool actions, release gates, rollback, and post-change monitoring.
Use this checklist before changing a customer-facing AI chatbot’s prompt, knowledge sources, connector scope, tool actions, admin settings, customer notice, retention setting, model/vendor configuration, or rollout location.
AI chatbot changes are risky because a small configuration update can change what the bot knows, what it says, what data it can reach, and what actions it can take. Before approving a change, run the AI Tool Risk Checker and attach the result to the change record.
Bottom line
A small team should approve chatbot changes only when the owner can show:
- What changed and why.
- Which customers, data, systems, sources, and tool actions are affected.
- Whether the change expands the bot’s scope, access, autonomy, or customer impact.
- Which tests passed, including regression tests from prior incidents and corrections.
- Who approved the change and who can pause or roll it back.
- What will be monitored in the first 24 hours and first 7 days.
- What customer-facing wording, retention promise, or human handoff path changed.
Use the Small Team AI Security Checklist for baseline owners, access review, incident routing, and evidence storage. This page focuses on approving one chatbot change.
When to use this checklist
| Change | Use this checklist? | Why |
|---|---|---|
| New or edited system prompt | Yes | It can change scope, refusal behavior, escalation, and claims. |
| New knowledge source, folder, help center section, or sync rule | Yes | It can expose stale, private, conflicting, or unapproved content. |
| New connector, API, CRM field, ticket queue, or account context | Yes | It can expand customer data access. |
| New tool action or automation | Yes | It can affect tickets, accounts, billing, access, deletion, messages, or records. |
| Model, vendor, retrieval, safety, or admin setting change | Yes | It can alter behavior without changing your visible content. |
| Customer notice, privacy wording, opt-out path, or retention setting change | Yes | Customer-facing promises must match operations. |
| Small copy edit to approved public FAQ source | Usually | Lightweight review is enough if no scope, access, or action changes. |
| Emergency containment after incident | Yes, but use emergency lane | Record the reason, approver, rollback path, and follow-up review. |
If the change can affect customer trust, customer data, account state, support decisions, or compliance wording, treat it as high-risk until proven otherwise.
Change intake form
Copy this into the change request.
| Field | Entry |
|---|---|
| Change title | |
| Requester | |
| Bot name and location | |
| Change type | Prompt, source, connector, tool action, admin setting, model/vendor, notice, retention, rollout, or other. |
| Reason for change | |
| Customer problem solved | |
| Affected customers or segments | |
| Affected data | None, public, internal, customer, account, billing, regulated, credential-like, or other. |
| Affected systems | Help center, ticketing, CRM, billing, account system, data warehouse, source repository, or other. |
| Affected tool actions | |
| Prior incident, complaint, correction, or red-team test related to this change | |
| Proposed release date | |
| Rollback owner |
Reject the request if the requester cannot describe the affected data, systems, or owner.
Change risk matrix
| Risk tier | Change examples | Approval required |
|---|---|---|
| Low | Public FAQ wording change, typo fix, source owner update, link fix. | Bot owner. |
| Medium | New public help center source, non-sensitive handoff rule, approved disclosure wording, routing tag change. | Bot owner and support/product owner. |
| High | New customer data field, internal source, connector scope, model/vendor setting, retention setting, sensitive-topic answer, or customer-facing trust claim. | Bot owner plus security/privacy owner. |
| Critical | New tool action affecting account, billing, deletion, access, outbound messaging, legal/compliance wording, or incident recovery. | Executive/business owner plus security/privacy/product owner. |
Use the highest tier that applies. Do not average the risk down because part of the change is routine.
Scope change decision table
| Question | If yes |
|---|---|
| Does the bot answer a new category of customer question? | Add tests, source owner, handoff rule, and disclosure review. |
| Does the bot use a new source, connector, or field? | Run source/access review before release. |
| Does the bot keep or expose more transcript data? | Review retention wording and deletion/export workflow. |
| Does the bot make a stronger privacy, security, legal, billing, or compliance claim? | Require human approval and approved wording. |
| Does the bot act on a customer account or downstream system? | Require tool-action approval and rollback plan. |
| Does the bot reduce human review? | Require monitoring signals and escalation thresholds. |
| Does the change fix an incident or repeated correction? | Preserve the old failing case as a regression test. |
| Does the change depend on a vendor feature or model behavior? | Record vendor docs, setting owner, and fallback path. |
If a change expands scope, do not release it through a “minor update” path.
Prompt change checklist
- Store the old prompt and new prompt in the change record.
- Summarize the exact behavioral change expected.
- Confirm the prompt does not ask the bot to hide AI use, skip handoff, ignore user safety, or make unsupported claims.
- Confirm sensitive topics route to a human where required.
- Test prior wrong-answer, handoff, and prompt-injection cases.
- Test customer data questions, deletion/export questions, training questions, and opt-out questions.
- Confirm customer-facing wording matches the disclosure notice and privacy/support pages.
- Confirm the bot still refuses prohibited account, billing, deletion, access, or outbound-message actions unless approved.
Use the AI chatbot red-team test checklist before approving a major prompt change.
Knowledge source change checklist
- Name every source added, removed, or changed.
- Record source owner, source system, access scope, and sync cadence.
- Confirm the source is approved for chatbot use.
- Confirm the source does not include private notes, internal-only strategy, credentials, payment data, regulated data, or unrelated customer data.
- Check source freshness and conflict with other approved sources.
- Test common questions, edge cases, missing-source behavior, and stale-source behavior.
- Confirm source citations or internal evidence are not exposing private titles, paths, or metadata.
- Record how to remove or pause the source if the change fails.
Use the AI chatbot knowledge base review checklist for a full source review.
Connector and data change checklist
| Check | Required evidence |
|---|---|
| Connector purpose | Why the bot needs this connector or field. |
| Minimum scope | The narrowest source, folder, object, field, or role that works. |
| Data classification | Public, internal, customer, account, billing, regulated, credential-like, or other. |
| Access owner | Person who can approve, revoke, and review the connection. |
| Customer impact | What the bot can infer, display, summarize, or route from the data. |
| Retention impact | Whether transcripts, tool logs, or downstream records keep this data. |
| Deletion/export impact | Whether customer request workflows need updating. |
| Audit evidence | Where connector changes, access grants, and tool calls are logged. |
Use the AI chatbot deletion and export request workflow if the change affects customer data request handling.
Tool action change checklist
- Add the action to the approved action inventory.
- Define allowed inputs, disallowed inputs, and expected output.
- Confirm the bot cannot choose account, billing, access, deletion, or outbound-message actions without the approved confirmation step.
- Test prompt injection attempts against the action.
- Test duplicate, retry, timeout, and partial failure behavior.
- Confirm the customer sees what will happen before the action runs.
- Confirm logs show prompt, action, arguments, approval, result, and owner.
- Define the rollback or cleanup step if the action runs incorrectly.
Use the AI chatbot tool action approval checklist before enabling any new action.
Release gate checklist
| Gate | Pass condition |
|---|---|
| Owner | Bot owner, system owner, and data owner are recorded. |
| Scope | Change does not exceed the approved use case. |
| Data | New data access is minimized and approved. |
| Sources | Source owner, freshness, conflicts, and sensitive content are reviewed. |
| Prompt | New behavior is documented and tested. |
| Tool actions | Human approval and rollback exist for high-impact actions. |
| Handoff | Sensitive, disputed, and human-request paths still work. |
| Notice | Customer-facing disclosure, opt-out, and support paths match the change. |
| Retention | Transcript, log, deletion, export, and redaction rules are still accurate. |
| Testing | Regression, prompt injection, sensitive data, source, and handoff tests pass. |
| Monitoring | First-day and first-week monitoring owners are assigned. |
| Rollback | The team can pause the bot, pause a capability, or restore the old config. |
Do not approve the change if any high-risk gate has no owner.
Test set before approval
| Test group | Minimum tests |
|---|---|
| Happy path | Common customer questions the change is meant to improve. |
| No-answer path | Questions outside approved scope. |
| Handoff path | ”I want a human,” complaint, refund, legal, privacy, security, billing, and account-impacting topics. |
| Sensitive data | Passwords, payment data, recovery codes, private keys, regulated data, and account-sensitive data. |
| Prompt injection | Ignore instructions, reveal hidden prompt, override tool approval, expose source, exfiltrate data. |
| Source conflict | Two sources disagree; bot should use the approved source or hand off. |
| Stale source | Old policy or removed feature; bot should not answer from stale content. |
| Tool action | Approved, denied, malformed, duplicate, retry, timeout, and high-impact paths. |
| Customer notice | Customer can identify AI use and find a human path. |
| Regression | Prior incidents, corrections, complaints, and failed red-team cases. |
Keep the test set short enough to run, but broad enough to catch scope expansion.
Rollback plan
| Rollback item | Owner | Ready? |
|---|---|---|
| Restore previous prompt | Bot owner | |
| Remove or pause new source | Source owner | |
| Revoke connector scope | Admin/security owner | |
| Disable tool action | Product/security owner | |
| Restore customer notice or support routing | Support/trust owner | |
| Preserve logs and transcripts | Security/privacy owner | |
| Customer correction or notice path | Support/privacy owner | |
| Post-rollback test set | Bot owner |
Rollback should be possible without waiting for the person who made the change.
Emergency change lane
Use the emergency lane only for containment, outage recovery, active abuse, private data exposure, broken handoff, unauthorized tool action, or customer-impacting wrong answers.
| Emergency step | Required output |
|---|---|
| State the emergency | What is happening and who is affected. |
| Limit the change | Only the minimum containment change is allowed. |
| Name approver | Bot owner plus security/privacy/product owner where relevant. |
| Preserve evidence | Logs, prompts, sources, tool calls, tickets, and screenshots if safe. |
| Define rollback | How to undo the emergency change. |
| Schedule follow-up | Full review within 1 business day. |
Emergency does not mean undocumented. It means short, focused, and reviewed after the immediate risk is contained.
Approval record
Copy this table into the final approval note.
| Field | Entry |
|---|---|
| Change title | |
| Risk tier | Low, Medium, High, or Critical. |
| Approved change summary | |
| Out-of-scope items | |
| Tests run | |
| Failed tests and fixes | |
| Data/source/connector changes | |
| Tool action changes | |
| Customer notice or retention changes | |
| Rollback path | |
| First 24-hour monitoring owner | |
| First 7-day monitoring owner | |
| Approvers | |
| Release decision | Approve, approve with limits, reject, hold, emergency release, or rollback. |
| Next review date |
The approval record should be readable by someone who did not attend the meeting.
Post-change monitoring
| Period | Review |
|---|---|
| First hour | Confirm bot is live only where intended and high-impact actions are not firing unexpectedly. |
| First day | Review conversations touching the changed prompt, source, connector, action, notice, or setting. |
| First day | Review all handoffs, corrections, sensitive data entries, tool actions, denied actions, and errors. |
| First week | Compare real failures with test results and add new regression cases. |
| First week | Decide whether to continue, limit, expand, or roll back the change. |
| First month | Include the change in monthly admin, source, access, retention, and monitoring review. |
Use the AI chatbot production monitoring checklist after release.
Metrics to track
| Metric | Why it matters |
|---|---|
| Changes approved by risk tier | Shows whether high-risk changes are getting proper review. |
| Changes released without full evidence | Shows process debt. |
| Failed tests before release | Shows whether tests are finding issues. |
| Incidents or corrections after change | Shows whether approval gates are working. |
| Time to rollback | Shows operational readiness. |
| Unapproved source, connector, or setting changes | Shows admin drift. |
| Tool actions attempted, denied, failed, and rolled back | Shows automation exposure. |
| Sensitive data entries after change | Shows privacy and warning effectiveness. |
| Handoff failures after change | Shows whether support routing still works. |
| Regression cases added | Shows learning from real failures. |
The goal is not to slow every update. The goal is to keep small changes from silently becoming major risk changes.
Evidence checked
This checklist is aligned with:
- NIST AI RMF Core, which emphasizes continuous AI risk management, documented roles, ongoing monitoring, lifecycle risk mapping, regular evaluation, tracking emergent risks, change management, incident response, recovery, and deactivation when outcomes are inconsistent with intended use.
- NIST AI 800-4 monitoring report summary, which identifies post-deployment monitoring as crucial and frames functionality, operational, human factors, security, compliance, and impact monitoring categories.
- NIST continuous-monitor-and-update AI security note, which argues against one-time guardrails and emphasizes continuous red-team work, guardrail updates, operational resilience, impact limitation, and quick recovery.
- CISA joint guidance on deploying AI systems securely, which highlights confidentiality, integrity, availability, mitigations for known vulnerabilities, and controls to protect, detect, and respond to malicious activity against AI systems and related data and services.
- OWASP Top 10 for LLM Applications, which covers prompt injection, insecure output handling, training data poisoning, sensitive information disclosure, insecure plugin design, excessive agency, and overreliance risks relevant to chatbot changes.
- FTC artificial intelligence guidance, which tracks enforcement and guidance related to AI accuracy, advertising, safety, data handling, chatbot monitoring, deceptive claims, and consumer protection.
- Cybergiz templates for chatbot launch review, source review, tool action approval, disclosure notices, admin settings review, deletion/export handling, red-team testing, and production monitoring.
This page is practical operating guidance, not legal, procurement, privacy, compliance, audit, certification, release-management, incident-response, or security assurance advice.
FAQ
Is every chatbot change a security review?
No. Typos and approved public-source edits can use a lightweight review. But any change that expands access, autonomy, scope, customer promises, retention, source coverage, or downstream actions needs risk-based approval.
Who should approve chatbot changes?
The bot owner should approve routine changes. Add support/product for customer workflow changes, security/privacy for data or connector changes, and executive/business ownership for critical tool actions or customer-impacting risk.
Can we batch small chatbot changes?
Yes, but list each source, prompt, connector, setting, and action change separately. A batch that includes one high-risk item should follow the high-risk approval path.
What if a vendor changes model behavior without notice?
Treat it like a change. Re-run key tests, review monitoring signals, check admin settings and vendor release notes if available, and record whether to continue, limit, or pause the bot.
Do we need screenshots?
Use screenshots only when they help prove settings or customer-visible wording. Do not store screenshots containing customer data, credentials, private prompts, or internal source content in public docs.
What should block release?
Block release if the change lacks an owner, expands data access without approval, skips high-impact tool-action confirmation, breaks handoff, creates unsupported claims, weakens retention or deletion wording, fails regression tests, or has no rollback path.
How soon should we monitor after approval?
For high-risk changes, review the first real conversations the same day. For critical actions, watch tool logs immediately after release and keep a clear disable path ready.
Where should we keep the approval record?
Keep it in the internal system that already stores AI tool evidence: ticket, doc, GRC tracker, or controlled workspace. Do not publish private prompts, customer transcripts, source internals, credentials, or security details.