How to Vet an AI Consultant: Shippers vs. Explainers

How to Vet an AI Consultant: Shippers vs. Explainers

Vet an AI consultant by asking for shipped work, evidence of repairs, and a plan your team can maintain. Every seller has the same tools now. Hire a shipper. They should connect the build to a business result, protect your data, and leave your people able to run it.

Across our AI engagements, the most reliable help comes from people who regularly ship small improvements. The least reliable can sound the most confident during the first meeting. You learn much more by seeing how a process survives an awkward customer request than by watching a polished demonstration.

A pattern we see: a consultant sells advice by the hour, and nobody on their side owns turning it into working changes. The meetings are good. The build never starts. Ask who does the building and how that work gets paid.

Vista's Knowledge-vs-Execution Gap. This is the distance between explaining what AI could do and delivering a working change your team can use, check, and maintain. Your hiring process should test that distance. Explainers stop at the first half. Shippers close the gap.

Use the questions below in a working conversation, requesting examples with sensitive details removed and following up when an answer stays abstract. You are checking practical experience, business judgment, and client management. A fluent explanation earns your attention. Evidence earns the work.

PROOF OF WORK

Can they show delivery and recovery?

Ask for working evidence first. Look for something used in normal operations, then hear what happened when it failed.

1. Show me something you shipped this month.

   Why it matters: A recent build shows how the consultant turns ideas into work that someone can actually use.

   What a good answer sounds like: "Here is a request-routing process, with customer details removed. Staff review the draft, approve the destination, and can take over manually." The consultant explains the starting problem, shows the changed process, names the person using it, and shares the handoff instructions. A demonstration built for selling is weaker evidence than work with routine users.

2. What broke, and how did you fix it?

   Why it matters: Repair experience reveals how the consultant tests assumptions and takes ownership when a build behaves unexpectedly.

   What a good answer sounds like: "An ambiguous request reached the wrong queue. We added a review step, changed the routing rule, and tested similar requests." You want the failure, its cause, the recovery, and what changed in the process afterward. An experienced builder explains the repair in terms your team understands, without blaming the model and leaving the business problem unresolved.

A customer changes a delivery address in a reply, and the draft confirmation still contains the original address. Ask your candidate who catches the mismatch and what they would do to keep the wrong address from reaching your customer. Where would your staff find the record needed to correct it?

That one question exposes their approval rules, their record handling, and how well they understand customer service. Hire someone who can follow the work through those details. The same standard applies when you assess whose advice to trust: look for experience that survives follow-up questions.

BUSINESS JUDGMENT

Who builds, and what should stay human?

Meet the builder before you commit. The person selling the engagement should explain who implements it and which decisions your team keeps.

1. Who on your team actually builds?

   Why it matters: Access to the person making implementation decisions matters most when your process does not match the sales presentation.

   What a good answer sounds like: "Our builder scopes the work, owns testing, and stays available through this support channel." Ask who gathers requirements, reviews access, and trains staff, even when those jobs belong to different people. What matters is that the handoffs are defined and someone owns the finished result. A mystery implementation team makes even a small correction harder to resolve.

2. What will you not automate?

   Why it matters: Knowing where to stop protects the judgment, customer relationships, and approvals that make your business work.

   What a good answer sounds like: "We would draft a response, but your team approves promises about delivery or exceptions to your pricing." Ask them to place that boundary in your operations, fixing an unclear process before adding AI when the current rule causes trouble. Treat "we can automate everything" as a red flag. It erases the line between routine admin and decisions with consequences.

In our delivery work, the tool stack rarely separates strong help from weak help. Strategic depth, sales experience, and client management do. Consultants who frame AI as support for your team's expertise understand that adoption takes work. Replacement promises often skip the person who must handle exceptions afterward.

Test that judgment with decisions you already face:

• A pricing exception requires approval before a quote leaves your business.
• A customer complaint needs context from the person who handled the job.
• A draft requires a reviewer who can recognize a promise you cannot keep.

Listen for how the consultant backs those people, including when the proposed automation stops and asks them for help.

AFTER THE HANDOFF

Can your team control the data and maintain the work?

Keep control of the working process. Data handling and maintenance belong in the scope before you approve the build.

1. How do you handle our data?

   Why it matters: Understanding what information moves through the process tells you who can access it and where it goes.

   What a good answer sounds like: "We will list the information needed, where it goes, who sees it, and how access ends." Ask the consultant to walk through a sample record without uploading customer information during the sales meeting. They should identify every service involved and explain which details can be removed. If they cannot answer a storage or access question yet, it should become a task to resolve before the build.

2. Who maintains it after you leave?

   Why it matters: An automation becomes your team's burden if routine repairs require a person you can no longer reach.

   What a good answer sounds like: "Your supervisor can update approved instructions, and we handle faults under the support agreement." Ask to see documentation, ownership of accounts, and the path for reporting trouble, with repairs defined separately from new work. Your staff should know what they can change themselves and what requires the builder, including how to switch to the manual fallback. Training should include an ordinary correction, not just a successful run.

Write the handoff expectations into the scope:

• Business accounts and access belong under your control.
• Instructions show how to pause the automation and finish the task manually.
• Support terms say who responds and which repairs are included.
• A named person on your team accepts and maintains the operating instructions.

Ask what your staff will practice before handoff, including how to correct an instruction or recover a stopped task. You will learn more from that explanation than from a promise that the system is easy. Your team should be able to operate the process on an ordinary working day.

BUYING THE RESULT

How will you pay and measure success?

Define the result before discussing the rate. You need an agreement that connects payment, scope, and acceptance to the work your team receives.

1. How do you price hours and outcomes?

   Why it matters: Pricing should tell you what gets built, which assumptions could change the scope, and who carries the implementation risk.

   What a good answer sounds like: "We charge for a defined workflow with agreed acceptance checks, and extra scope requires your approval." Hourly work can fit discovery or troubleshooting, but it becomes harder to judge when advice sessions keep replacing implementation. Ask what counts as finished and who handles another round of corrections before you accept the work. Paying for an outcome means naming it, not labeling an open-ended engagement with a new phrase.

2. How do you measure success?

   Why it matters: A busy build can leave your team with the same bottleneck and more steps to manage.

   What a good answer sounds like: "We will compare the existing process with the trial, including review effort and corrections." The consultant should ask how you currently handle the work and who will record the results. Choose a measure tied to the problem, such as requests waiting for review or drafts your staff must rewrite. Agree how you will judge the trial, since a faster draft is progress only if the whole task improves.

Use this table when the sales conversation becomes slippery. Each follow-up brings the answer back to evidence of finished work.

Red flag	What it usually means	The question that exposes it
Tool names dominate the explanation	The business process has received little attention	What changes for my staff when this is working?
Every example ends at a demonstration	Routine use and handoff may be missing	Who uses this now, and what can they maintain?
The promise is total replacement	Exceptions and approvals have been skipped	Which decisions stay with my team?
Advice continues without an acceptance point	The engagement has no defined finish	What must be delivered before this phase ends?
Accuracy is discussed without review effort	Staff corrections may erase the benefit	How will we record the work needed to approve output?

For general hiring warning signs, see our red flags for hiring a business advisor. Here, focus the test on AI implementation: working examples, controls, repairs, and a measurable handoff.

THE HIRING DECISION

What evidence earns a small first engagement?

Start with a bounded project. References and a plan for vendor changes should back your decision before you expand the work.

1. Can I speak with a business our size?

   Why it matters: Hearing how the work was put into use reveals how a consultant handles staffing and operating pressures like yours.

   What a good answer sounds like: "With their permission, you can ask a past customer about adoption, maintenance, and what they run today." Ask the reference how staff used the process after handoff, how failures were fixed, and who handled later changes.

   If confidentiality limits introductions, ask for a walkthrough with sensitive details removed and evidence of routine use. No references? Shrink the first project until you can judge the delivery yourself.

2. What happens if the model or vendor changes?

   Why it matters: Your business requires a recovery path when a service changes behavior or becomes unavailable.

   What a good answer sounds like: "We document the operating rules, test changes before applying them, and maintain a manual route." Ask which parts depend on a particular service and what replacing it would involve. Skip the model architecture lecture; ask who notices trouble, decides on changes, and helps your staff serve customers while the builder repairs the work.

Choose a first task with a named user, a review boundary, and an acceptance check. Put data access, support, and documentation in the agreement, then schedule a trial decision to keep it, repair it, or stop. That makes a bigger commitment something the consultant earns through the work.

The same test works on any outside help you pay for monthly. Our guide to judging what a media buyer delivers applies it to paid ads. To practice scoping and checking AI work with your team, start with Vista's AI Lab workshops.

COMMON QUESTIONS

Frequently asked questions

Buy evidence before a larger commitment.

What is the strongest evidence that an AI consultant can deliver?

Start with shipped work. Ask for a workflow people use, an explanation of what broke, and the instructions left with the team. Evidence of the build shows how the consultant handles normal operations, corrections, and staff handoff; a polished demonstration alone cannot establish that.

Should I reject an AI consultant who charges hourly?

Hourly pricing can fit a defined discovery or troubleshooting phase, with agreed deliverables and approval for extra work. The concern is an engagement that keeps selling advice without producing a working change. Match the payment structure to a defined task, acceptance checks, and a point when the work is finished.

Does an AI consultant need experience in my industry?

Experience with your operating pressures matters most when it helps the consultant design a workflow your staff can run. Ask about customer promises, approvals, exceptions, and limited staff time. A candidate from another field can earn a small trial by demonstrating sound judgment about those details in your process.

What should an AI consultant explain about data before starting?

The consultant should identify the information needed, the services receiving it, and the people with access before work begins. Ask how access ends and which details can be removed. Resolve unanswered questions before using business records, and establish the data path with a sample walkthrough that excludes customer information.

What should I include in a first AI consulting project?

Choose a bounded workflow with a named user and a review step, supported by acceptance checks your team understands. Agree on data access, documentation, support, and a manual fallback before starting the trial. At the trial review, decide to keep, repair, or stop it. Expand only after your team can run and maintain it.

What is Vista's Knowledge-vs-Execution Gap?

Vista's Knowledge-vs-Execution Gap is the distance between explaining AI and delivering a working business change. You test it through shipped examples, repair experience, business judgment, and handoff quality. A consultant closes that gap when your team can use the result, check its behavior, and maintain the process.