checklist

AI chatbot replacement cutover checklist for small teams

A practical cutover checklist for replacing a customer-facing AI chatbot, covering target controls, data migration boundaries, shadow testing, connector and action mapping, customer continuity, rollback, and old-vendor closure.

Published: Aug 14, 2026 Last reviewed: Aug 14, 2026 Audience: Founders, support leads, product owners, engineering owners, security owners, privacy owners, and admins moving a customer-facing AI chatbot to a new platform Risk: High Evidence: NIST AI RMF Core and Playbook, FTC small-business cybersecurity guidance, CISA secure AI system development guidance, OWASP Top 10 for LLM Applications, and Cybergiz chatbot operations templates

Use this checklist when replacing a customer-facing AI chatbot vendor, model, platform, or orchestration layer without losing control of customer data, human handoff, tool actions, or rollback.

A replacement is ready for cutover only when the new route has been tested against the old route’s real use cases, the data transfer is minimized and approved, connectors and actions have explicit boundaries, support can handle fallback, and the old route can be disabled or restored. Before approving the change, run the AI Tool Risk Checker and attach the result to the cutover record.

Bottom line

Do not switch a customer-facing chatbot because the new vendor has a better demo. Switch only after the team can show that the target system handles expected questions, sensitive topics, no-answer cases, human handoff, customer data, prompt injection, source conflicts, tool actions, logging, retention, and rollback within the team’s risk tolerance.

Use the Small Team AI Security Checklist for baseline ownership, access review, and incident routing. Use the AI chatbot vendor exit checklist for the old vendor’s data, access, subprocessor, and closure work. This page focuses on the change window and cutover decision.

When to use this checklist

Situation	Use this checklist?	Main risk to control
Replace the chatbot vendor	Yes	Data, behavior, permissions, and fallback can change together.
Move from a hosted bot to an internal orchestration layer	Yes	New code and identities may expand access or action scope.
Change the model behind the same chatbot	Yes	Answer quality, refusal, latency, and tool behavior may shift.
Move from one channel to another	Yes	Disclosure, logging, handoff, and customer expectations may differ.
Add a new source or connector during migration	Yes	Scope creep makes it hard to attribute failures.
Fix a typo in an existing disclosure	Usually no	Use the normal change approval process.
Move an internal-only bot with no customer data	Maybe	Scale the review to the actual data and action boundary.

Keep the replacement change narrow. A vendor switch, new data source, new connector, new channel, and new tool action should not all be first-day changes.

Cutover decision matrix

Outcome	Use when	Required evidence
Go	Target meets the minimum quality, safety, access, and continuity gates.	Signed gate record and test results.
Limited pilot	Target is ready for one channel, topic, or customer group but not broad use.	Explicit boundary, pilot owner, expiry date, and rollback trigger.
Shadow only	Target can generate suggestions but should not answer customers or execute actions.	Human review process and comparison scorecard.
Defer	A material gap is understood but not yet fixed.	Owner, due date, compensating control, and decision expiry.
No-go	Data, action, handoff, logging, or rollback control is missing or fails high-risk tests.	Failed test evidence and remediation decision.

No-go findings include unknown customer data flow, unbounded connector access, unsafe write actions, failed sensitive-topic handoff, missing logs for high-impact events, or no tested way back to the old route.

Cutover intake form

Copy this into the chatbot change record.

Field	Entry
Review date
Change owner
Old vendor or platform
New vendor or platform
Channels affected
Topics and customer groups affected
Data classes involved	Public, internal, customer, account, billing, regulated, or credential-like.
Sources to migrate
Connectors to migrate
Tool actions to migrate
Human fallback route
Shadow-test window
Cutover date and timezone
Rollback owner
Customer communication owner
Evidence location

If the team cannot complete the data, connector, or action fields, keep the change in discovery rather than calling it a low-risk migration.

Target control comparison

Compare the old and new systems by control, not by feature count.

Control	Old system	New system	Decision or gap
Customer data input boundary
Conversation retention
Deletion and export path
Source approval and freshness
Connector read scope
Connector write scope
Human approval for actions
Prompt injection controls
Sensitive-topic routing
Audit and action logs
Admin roles and SSO
Vendor support access
Pause and rollback
Customer notice and disclosure

Do not mark a control equivalent because both vendors use the same label. Record what the setting does, who owns it, how it is tested, and what limitation remains.

Data migration boundary

Choose what the new system actually needs. A replacement is not a reason to copy every historical conversation or upload.

List the exact customer, account, billing, support, and public data fields required by the new use case.
Exclude secrets, credentials, unrelated tickets, raw exports, and obsolete records.
Define whether historical transcripts are needed for continuity, testing, support, or legal/incident evidence.
Redact or tokenize sensitive fields before transfer where possible.
Set access, retention, deletion, and export rules in the target system before import.
Protect temporary migration files and assign an expiry date.
Validate a small sample before importing the approved data set.
Record data owner approval and the person responsible for deleting temporary copies.

The FTC advises small businesses to specify vendor data use, sharing, retention, and deletion, and to limit a vendor’s access to the data it needs. Apply that same boundary to the replacement system rather than inheriting the old vendor’s access by default.

Source migration checklist

Check	Pass condition
Source inventory	Every migrated source has an owner and scope.
Freshness	Important sources have a review date or update process.
Access	Private sources are available only to approved routes and users.
Conflict handling	Conflicting documents have a resolution rule.
Retrieval test set	Known questions retrieve the expected source.
Sensitive source rule	Internal notes, customer records, legal drafts, and secrets are blocked or handled deliberately.
Update workflow	Source changes trigger a retest when needed.
Retirement cleanup	Old indexes and copied sources have a removal or retention decision.

Do not change source content during the first cutover unless the change is separately approved. Otherwise a bad answer cannot be attributed to the platform replacement.

Connector and action mapping

Map every read, write, outbound message, and background job.

Capability	Old route	New route	Minimum permission	Human approval	Rollback
Search support records
Read account status
Create a support ticket
Update a CRM record
Send an email or message
Billing or refund request
Account or access change
Delete or export data

The new system starts with the minimum read scope.
Write and outbound actions are disabled until separately tested.
High-impact actions require an explicit human approval step.
Duplicate, retry, timeout, and partial-failure behavior is known.
Action attempts, arguments, approvals, denials, outcomes, and errors are logged.
Old and new service identities cannot both write the same workflow accidentally.
Connector owners know how to revoke access during rollback.

OWASP identifies excessive agency, prompt injection, and sensitive information disclosure as relevant LLM application risks. A model or vendor swap does not remove those risks; it can change where the controls must be implemented.

Shadow test and comparison scorecard

Run the target in shadow mode where possible. Do not send its output to customers or execute its actions during this phase.

Test group	Minimum sample	What to compare
Normal questions	10-20 representative questions	Accuracy, source use, tone, latency, and answer completeness.
No-answer cases	5 or more known unknowns	Refusal, uncertainty, and human route.
Sensitive topics	Privacy, security, billing, access, legal, HR, health, or safety as relevant.	Correct routing and no unsafe advice.
Customer data	Approved redacted examples	Correct use, non-use, and leakage resistance.
Source conflicts	Conflicting or stale documents	Conflict handling and escalation.
Prompt injection	Direct and indirect attempts	No unauthorized instruction following or data access.
Tool actions	Approval, denial, retry, duplicate, timeout, and rollback cases.	Permission, idempotency, logs, and safe failure.
Handoff	Customer asks for a person or disputes an answer.	Context transfer, queue creation, and response target.
Abuse	Spam, harassment, manipulation, and oversized inputs.	Rate limits, safe response, and escalation.

Record the old and new outputs, source trace, test input classification, reviewer decision, and follow-up. Do not store real customer secrets in the public repository or in an unprotected comparison file.

Go-live gates

No single green dashboard should override a failed customer-data, action, handoff, or rollback test.

Customer continuity plan

Customer journey	During cutover	Fallback owner	Test result
Public FAQ
Account-specific support
Billing and refunds
Privacy requests
Security reports
Existing open conversations

Publish a clear maintenance or route-change notice where needed.
Provide a human route for account, billing, privacy, security, and disputed questions.
Give support the expected response time and escalation owner.
Reconcile conversations that begin before the cutover and finish after it.
Do not silently drop pending tool actions or deletion/export requests.

Cutover runbook

Freeze unrelated prompt, source, connector, and UI changes.
Confirm the decision record, owners, fallback, and rollback trigger.
Capture the old configuration, routes, source versions, permissions, queues, and metrics baseline.
Import only the approved minimum data and verify the sample.
Enable the target in shadow or limited pilot mode.
Review shadow results and close or accept each gap explicitly.
Disable target write actions unless their approval gates passed.
Announce the cutover to support, customer success, engineering, and security/privacy owners.
Switch one low-risk route or customer segment first.
Monitor answers, handoffs, action attempts, errors, latency, and customer feedback.
Expand only after the agreed observation window passes.
Disable the old customer route and begin the vendor exit sequence.

Sequence the change so the team can identify whether a failure comes from the target, the routing layer, the data transfer, or the old system being left active.

Rollback triggers

Trigger	Immediate action	Decision owner
Customer data appears in an unauthorized output	Pause the affected route, preserve evidence, and escalate.	Security/privacy owner.
Sensitive topic is not handed to a human	Force human route and pause the topic.	Support/product owner.
Unauthorized or duplicate tool action	Disable the action and reconcile downstream state.	Engineering/security owner.
Target answer quality falls below gate	Return traffic to the old route or human-only fallback.	Product owner.
Logs cannot reconstruct a high-impact event	Pause high-impact capability until logging works.	Security owner.
Support queue or customer wait time exceeds limit	Limit traffic and activate fallback staffing.	Support owner.
Old route cannot be disabled safely	Keep the pilot narrow and resolve the identity/routing issue.	Engineering owner.

Rollback is a control, not a failure. A replacement without a practical rollback path is a migration still in progress.

Post-cutover verification

Cutover record

Field	Entry
Old system
New system
Approved scope
Explicitly blocked scope
Data migrated
Sources migrated
Connectors and actions enabled
Shadow-test result
Go-live decision
Cutover date and timezone
Rollback trigger
Fallback route
Customer notice
Old-vendor exit owner
Residual risk
Approvers
Final verification date
Next review date

Keep the record with the AI tool inventory, vendor review packet, test results, and exit record. Do not put customer exports, credentials, API keys, or private vendor responses in the public repository.

Metrics to track

Metric	Why it matters
Shadow answer agreement	Shows whether the target behaves like the accepted baseline.
Answer pass rate	Shows quality within the approved scope.
Sensitive-topic handoff rate	Shows whether risky conversations reach people.
Tool action attempts and failures	Shows automation exposure and control effectiveness.
Customer disputes and corrections	Shows harmful or misleading output.
Fallback wait time	Shows continuity during transition.
Old-route traffic after cutover	Finds stale links, clients, or routing.
Unapproved data fields observed	Finds migration or connector boundary failures.
Rollback events and reopened tasks	Shows whether the replacement is stable.

Do not optimize only for deflection or lower cost. A successful cutover also needs acceptable safety, support, privacy, and recovery evidence.

Evidence checked

This checklist is aligned with:

NIST AI RMF Core, which addresses mapping risks across AI components, third-party software and data, monitoring, accountability, and safe decommissioning.
NIST AI RMF Manage Playbook, which describes managing third-party AI resources, contingency processes, monitoring, and documented risk controls.
FTC cybersecurity guidance for small businesses, which recommends specifying vendor data use, sharing, retention, deletion, security requirements, verification, and least-necessary access.
CISA and UK NCSC secure AI system development guidance, which covers secure practices across AI design, development, deployment, and operation.
OWASP Top 10 for LLM Applications, which covers prompt injection, sensitive information disclosure, insecure plugin design, excessive agency, and overreliance.
Cybergiz templates for chatbot launch, red-team testing, tool action approval, human handoff, pause and fallback, vendor exit, and retirement decisions.

This page is practical operating guidance, not legal, privacy, compliance, audit, certification, customer-support, procurement, incident-response, or security assurance advice.

FAQ

Should the old chatbot and new chatbot run at the same time?

They can run in shadow or limited pilot mode, but avoid two systems independently writing to the same downstream workflow. Separate identities, actions, queues, and monitoring so the team can stop one route safely.

Do we need to migrate historical conversations?

Only when a documented support, customer, migration, or evidence need requires them. Minimize the fields, protect temporary files, set retention, and do not copy history merely because the new vendor accepts it.

What is the most important cutover test?

There is no single test. For a customer-facing bot, prioritize sensitive-topic handoff, customer-data boundaries, tool action approval and rollback, prompt injection resistance, source conflicts, logging, and the ability to return to a human or controlled old route.

Can we add a new connector during the replacement?

Preferably not during the first cutover. If it is necessary, treat it as a separate high-risk change with its own data map, minimum permissions, test set, approval, monitoring, and rollback.

How long should shadow testing run?

Long enough to cover normal traffic, sensitive topics, no-answer cases, support handoffs, action attempts, and likely failure modes. A fixed number of hours is not sufficient if the bot has low or highly variable traffic.

When should we disable the old vendor?

After the target passes its go-live gates and the team has verified the customer route, fallback, monitoring, and rollback. Then follow the old vendor’s exit sequence for access, data, support, subprocessor, backup, and closure tasks.

What if the new system is better but has weaker admin controls?

Treat the control gap as a no-go or limited-pilot finding. Better answer quality does not compensate automatically for unknown retention, broad access, missing logs, or unsafe actions.

Who approves the cutover?

At minimum, include the product or support owner, engineering owner, and security/privacy owner. Add customer success, procurement, legal, finance, or vendor owners when their workflows or obligations are affected.