Chatbot development: acceptance testing and handover

Delivering a chatbot is not the moment you stop working: it is the moment you prove the work holds up without you. The difference between a well-delivered project and an "approximately done" one comes down to a verifiable checklist: what was ordered, what was tested, what was validated, and whose responsibility remains what. This article gives you that checklist, the definitions to agree on in writing, and examples of failures that do or do not block delivery.

Why "prepared" is not "validated"

Four words come up in every project and rarely mean the same thing to everyone. Define them in writing from the brief onward, or you will debate them on acceptance day, from a weak position.

• Prepared: the feature exists in a test environment with fictional data. Nothing guarantees its behaviour in real conditions.
• Tested: pre-defined scenarios have been run and their results recorded, including deviations.
• Validated: you, or the designated business owner, have accepted the test results against the brief's criteria.
• Delivered: the chatbot runs in the intended production environment, with its handover dossier, transferred access and a known recovery plan.

This separation is not bureaucracy. A chatbot can be tested internally by the vendor and still be unacceptable to you, because the criteria differed. If you write a testable brief from the start — as detailed in our guide on writing a useful custom AI agent brief — validation becomes a verification, not a negotiation.

The delivery checklist, item by item

Here is the framework to check before any acceptance. Every line must be verifiable by you, not merely declared by the vendor.

1. Order and brief: the delivered scope matches the ordered scope. Every deviation is documented and accepted in writing, not discovered during acceptance testing.
2. Version: the exact production version number is known, with the list of changes since the tested version. A chatbot validated as version 3 and delivered as version 7 is not validated.
3. Tools and channels: the list of connected services (knowledge base, CRM, messaging channel) matches what was tested. A connection added at the last minute invalidates the acceptance run.
4. Permissions: the accesses used by the chatbot are listed, minimal and documented. See the dedicated section below.
5. Tests: the acceptance report lists each scenario, its expected result, its observed result and its status. Failures are classified as blocking or non-blocking, with written agreement.
6. Human: the handover-to-a-human cases are defined, tested, and covered by someone who knows they will receive those requests.
7. Handover: a complete delivery dossier, transferred access, a rollback plan and agreed support terms.

If your vendor refuses to provide any of these items, that is a signal about the rest of the relationship, not just the delivery. Our grid for comparing AI agent agency proposals helps you set these expectations before signing.

Permissions and access: the most commonly oversold point

Permissions are where misunderstandings cost the most. An interface label is not a permission boundary. Take an example outside chatbots but structurally identical: in the Gmail API, the gmail.compose and gmail.modify scopes can both allow sending messages. If you want the chatbot to only prepare drafts, restricting behaviour requires application-level controls in the code, not a checkbox in an authorisation screen. The official documentation on Gmail API scopes describes these access levels.

Transposed to your chatbot: ask for the exact list of authorisations granted to each connection, and ask how behaviour is restricted in practice. "The bot can only display" must be proven by code or configuration, not by the connector's name. Three distinct things must also be verified separately: the technical connection (transport works), the provider authorisation (the account is allowed to act), and proof of operation (a trace shows the action actually happened). A chatbot can have the first two without the third, and you would never know whether a follow-up was actually sent.

Acceptance testing: scenarios, blocking and non-blocking failures

Acceptance testing runs on scenarios written before development, not improvised in front of a demo. Each scenario has a measurable expected result. For a support chatbot, for instance: "when the user asks about an order status with a valid order number, the answer cites the correct status and offers tracking".

Classify failures into two categories, and have that classification validated before testing:

• Blocking: the chatbot gives a wrong answer on a case covered by the brief, acts on data without the agreed permission, sends a message to the wrong recipient, or leaves the user with no way out (no answer, no human handover).
• Non-blocking: an awkward phrasing on a rare case, a correct answer slower than the desired threshold, a lack of variety in reformulations.

A non-blocking failure is not an ignored failure: it goes into an improvement list with a dated commitment, or it is explicitly accepted. What is forbidden is discovering after delivery that a failure the vendor considered minor was major for your customers. Our article on evaluating AI chatbot answers against business criteria offers a method for building these acceptance criteria.

Main worked example: acceptance testing an order-tracking chatbot

Consider a hypothetical, entirely fictional case. An e-commerce SME orders a custom chatbot that answers order-tracking questions on its website and prepares a draft complaint email when a customer is unhappy. The brief defines four test scenarios.

At acceptance, version 5, the results are as follows:

Scenario	Expected result	Observed result	Status
Valid order number	Correct status cited	Correct status cited	Pass
Non-existent order number	Error message and offer of human contact	The bot invents a plausible status	Blocking
Refund request	Email draft prepared, not sent	Draft prepared, not sent, application-level control verified	Pass
Out-of-scope question (product advice)	Polite refusal and redirection	Approximate answer given	Non-blocking

Comment on each line. Scenario 2 is blocking because an invented answer on a case covered by the brief destroys trust and generates complaints: it is exactly the risk the brief was meant to cover. Scenario 4 is non-blocking because the approximate answer concerns an out-of-scope topic, with no action taken and no sensitive data involved; it goes on the improvement list with a commitment to fix within fifteen days — a fictional deadline here, to be set according to your context. Scenario 3 deserves a specific check: the vendor shows in the code that sending is impossible, not merely that the interface displays "draft". This is the same principle as the Gmail scopes above: the label is not enough.

Delivery happens only after scenario 2 is fixed and all four scenarios are re-run, on a documented version 6.

The handover dossier: what you must receive

The handover dossier is what makes you independent. Without it, you have not bought a chatbot; you have rented a dependency. It must contain, at minimum:

• The production version number and the version history.
• The inventory of connected tools, each with: what it is for, what permissions it has, who owns the account.
• Credentials and access transferred to your name, with password changes and removal of the vendor's access after an agreed period.
• The code or configuration, in a repository you can access, or failing that a clear clause on its return.
• The final acceptance report, signed.
• Documentation of the covered scenarios and, just as usefully, of the cases deliberately left uncovered.

Also check contractual reversibility: what happens if you switch vendors in two years? Who can export the conversation history, in what format, and under what conditions? A custom chatbot that cannot leave its agency is not custom.

Rollback: plan the way back before you need it

Rollback is the ability to return to the previous version if the new one misbehaves in production. It is prepared before delivery, not during the incident. Three questions to ask, with written answers:

1. How? A documented procedure with a target duration — for example, reverting to the previous version in under an hour, a hypothetical figure to validate against your architecture.
2. What happens to the data? Are conversations received during the faulty period kept, replayable, or lost? This is the same problem as managing retries and duplicates in automated workflows: see our article on retries and duplicates in workflow automation.
3. Who decides? Name a single person who can trigger the rollback without a meeting. A rollback nobody dares trigger on a Friday evening is not a rollback.

Test the procedure once, in real conditions, before considering delivery complete. A rollback that has never been executed is an assumption, not a mechanism.

Support and maintenance: who does what after delivery

Delivery opens a warranty period and a durable operating model. Both must be contracted. On warranty: duration, what it covers (for example, fixing deviations found during acceptance) and what it does not cover (new needs, changes to connected tools). On maintenance: who monitors the chatbot, how often, and who is alerted in case of malfunction.

Two points deserve particular attention. First, external changes: an API provider can modify its terms, pricing or permissions; your contract must state who carries that risk and who is notified, with what lead time. Second, continuous improvement: a chatbot lives off its conversations. Plan a periodic review of questions that received no satisfactory answer — monthly at first, a figure to adjust — and a simple channel for updating content. Our guide on building your first AI agent stresses this point: the project does not end at go-live, it changes pace.

The human side: the handover to people is part of the product

A chatbot that can say "let me hand you over" is more robust than one that always answers. But the handover is only real if someone on your team knows they will receive those requests, knows what to do with them, and has the conversation context. Test that circuit during acceptance, not after: deliberately trigger a human handover and measure what happens next.

Also check channel exit. If your chatbot runs on WhatsApp Business Platform, the 24-hour service window is only reopened by a message from the user; beyond that, only approved templates allow re-contacting them. Compliance, permission and channel-exit requirements should be checked in the WhatsApp Business documentation. Meta's Platform pricing is charged per delivered message and varies by category and market; not all messages are charged, but your support budget must account for it — check the Platform pricing page rather than relying on an old per-conversation rate. Finally, beware of a mere QR code presented as proof of an official Business Platform connection: it is not one.

Decision table: deliver, fix or postpone?

Situation found at acceptance	Recommended decision	Condition
Wrong answer on a case covered by the brief	Do not deliver, fix	Full re-test of the affected scenario
Unplanned action on data (send, modify)	Do not deliver, review permissions	Application-level control demonstrated in code
Improvable wording, rare case	Deliver with a dated commitment	Entry in the improvements register
Delivered version differs from tested version	Postpone delivery	Re-run acceptance on the exact delivered version
Connected tool added after testing	Postpone delivery	Additional acceptance on the new tool
Untested human handover	Postpone delivery	Circuit test with the relevant team
Rollback not demonstrated	Postpone delivery	Successful execution of a rollback
Incomplete handover dossier	Conditional delivery	List of gaps with a firm deadline

This table is a design grid, not a legal obligation or proof of compliance. Adapt the decisions to your context, but refuse "equal-trust" deliveries without verification.

If you want to frame a chatbot delivery with verifiable criteria from the start, you can define a project with Edeverse.

Frequently asked questions

What is the difference between acceptance testing and validation?

Acceptance testing is running the test scenarios and recording the results. Validation is your decision to accept those results against the brief. A vendor can present a flawless test run; only validation, which belongs to you, triggers delivery.

Can you deliver with non-blocking failures?

Yes, under two conditions: the blocking/non-blocking classification was validated in writing before testing, and each non-blocking failure appears in a register with a dated fix commitment or an explicit acceptance. Without that framework, every failure becomes a source of conflict.

What if the vendor does not provide a handover dossier?

Do not consider delivery complete. The dossier is part of the deliverable, just like the chatbot itself. Set a firm deadline, and if it is missed, activate the reversibility clauses in the contract — data export, access to the code, return of accounts.

How can I verify the chatbot really cannot send messages?

Ask for a demonstration of the application-level control: the restriction must exist in the code or configuration, not only in a connector's label. Have it checked by a competent person if you are not one. The principle goes beyond Gmail: a label is never a permission boundary.

How often should a chatbot be re-tested after delivery?

After every version change, every change to a connected tool and every update to the APIs in use, re-run at least the critical scenarios. Outside those triggers, a periodic review of questions that received no satisfactory answer is enough to guide fixes, without a systematic full re-test.