24 Questions to Ask an AI Agent Vendor Before You Sign

Use these questions to turn a sales conversation into evidence for a purchase. Keep a record with five columns: answer, supporting document or test, unresolved issue, owner and due date. A persuasive answer is still a claim until you can inspect its support.

Start with the workflow you want to improve. An agent label does not establish that the system can complete it reliably, and a conventional automation may be sufficient. Compare suppliers on the same job and acceptance criteria.

Scope and acceptance

1. Which of our tasks should stay manual or use ordinary automation? Ask for the reason: low volume, unclear rules, unavailable data or costly mistakes. Use your actual handling time and frequency to prioritize the work.

2. What is the first bounded workflow, and what counts as complete? Specify its trigger, inputs, final state, permitted actions and exclusions. “Manage customer support” is too broad; routing a defined category of billing tickets is testable.

3. What will we measure, against what baseline? Write the denominator, measurement period and buyer responsible for checking results. Subtract review and correction time when calculating hours saved. Count serious errors separately from the average success rate.

4. What stops automatic action, and who can restart it? Separate immediate incident stops from the minimum business result required at the pilot review. Name the owner, fallback process and evidence required to resume. Agree this before production access.

Evidence and delivery team

5. Can you demonstrate a comparable running workflow? Ask to follow one permitted example from input to confirmed outcome, including an exception. An internal system or an authorized, anonymized client example may help. Neither replaces a test on your use case.

6. What operating period and volume support the results? Ask for dates, eligible case count, failures, corrections and changes to the system. A demo proves a demonstrated path; it does not establish sustained performance.

7. Show a failure case and how the fix was tested. Look for the original error, its consequence and a repeatable check that catches it. A vendor can use a permitted test case without exposing another customer’s data.

8. Who builds, operates and covers absences? Meet the delivery owner and identify subcontractors, responsibilities and backup coverage. Put the agreed staffing and access arrangements in writing.

Data and security

9. Can our inputs or outputs be used for training? Request terms and settings for the exact products and plans. Review training use, retention and human access separately. A no-training promise does not establish zero storage.

10. Where does each data field go? Request a data-flow map covering providers, processing locations, logs, backups and deletion. Identify which subprocessors receive sensitive fields and whether those fields are necessary.

11. Which independent reports or certifications cover this service? Check the document, issuer, date or period, scope and exceptions with your security reviewer. SOC reports provide assurance information; a badge or a plan to obtain a report is not the report itself. Confirm whether your procurement requirements are met.

12. Who resolves data-processing and security questions? Request the data processing agreement, completed questionnaire and a named contact. Have the appropriate reviewer assess unresolved terms before data access. A fast reply alone does not prove good controls.

Actions and integrations

13. Which permissions are required in each system? Distinguish read access from creation, modification and deletion. Use the minimum required permissions and demonstrate that access can be revoked.

14. What happens after a timeout or duplicate event? For example, a CRM request times out after creating a contact. Have the vendor demonstrate how it checks the outcome before retrying, so the retry does not create a second contact. Name who maintains the integration.

15. What may the agent do on its own? Define allowed actions, limits, self-checks and exceptions sent to a person. Our approach is autonomy within agreed limits. Require evidence for those limits and separate authorization for expanding them.

16. Can we trace a result without asking the vendor to reconstruct it? Inspect a record of the input, source data, relevant model or configuration version, checks, actions and outcome. Agree access, export, redaction and retention for these records.

Price, ownership and changes

17. What is the price for the agreed first scope? Discovery may be needed before a reliable fixed quote. Ask what discovery delivers and costs, then record the build price, assumptions, exclusions and change process. A range is an estimate, not a fixed quote.

18. What does operation cost at normal and busy-month volumes? Include platforms, models, retries, fallbacks, hosting, maintenance and staff handling exceptions. Compare cost per accepted completed task; cost per attempted call understates it. The cost guide includes a first-year calculation.

19. What do we own, license and retain after termination? Separate commissioned code, prompts, configuration and evaluation cases from pre-existing or third-party components. Agree export formats, continuing rights, dependencies and any exit fees. “You own it” is incomplete if it cannot run without an undisclosed subscription.

20. How will additional workflows or scope changes be priced? Agree who approves changes, how estimates become quotes and what happens to the schedule. Do not treat an undefined future workflow as included in the first price.

Operation and exit

21. What does maintenance include? List monitoring, incident response, integration repairs, model changes, regression testing and reporting. Define exclusions and service hours. A model update should trigger relevant checks before wider use.

22. Can our operator run and recover the system after handover? Request documentation, account and permission transfer, training and an agreed handover test. Include an ordinary run and a failure scenario. State the cost and responsibilities for the transition.

23. How do you respond to a wrong action while the system remains online? Distinguish response time from resolution time. Ask how affected records are identified, automatic actions suspended and consequences corrected. An uptime commitment does not measure output correctness.

24. What evidence would make you recommend stopping? Name the result, cost or unresolved risk that would change the recommendation. Set a review date and a handover route. Extending a pilot needs a specific remaining question and budget.

Decide from the unresolved issues

A missing mandatory security requirement, unverified certification claim or refusal to define operational ownership needs resolution before access or commitment. An honest “we need to inspect the integration first” can be a useful answer when it comes with a bounded discovery plan.

Use the pilot procurement guide to turn accepted answers into scope and tests. A paid pilot can fund delivery work; a free technical test may answer a narrow feasibility question. Neither payment model establishes quality on its own.

We agree a fixed fee for the defined pilot scope. Confirm model usage, platform subscriptions, internal staff time and ongoing support separately in the written proposal. Our trust page describes our controls and offers a data processing agreement on request; it also states that we do not have a SOC 2 report. Our published ownership terms leave what we build with you. Confirm the exact deliverables and dependencies in the contract.

For implementation and handover scope, review AI agent development. If the first decision is which workflow merits investment, AI consulting covers that planning work. Bring the same evidence questions to either conversation.