The AI Marketing Vendor Evaluation That Starts With Your Own Team

AI marketing vendor evaluation usually aims the scrutiny at the vendor. But the four questions that predict whether an AI agent will actually work are ones you have to answer about your own team first: who approves its actions, what gets logged, how you undo a bad one, and what caps its spend.

A company lost a paying customer to a support bot it had bought, switched on, and stopped watching. Not to a competitor. To its own AI.

The setup was Salesforce Agentforce, running the company’s public support channel. It had passed its demo and gone live. Then a customer asked how to cancel. Instead of handing the conversation to a person who might save the account, the agent opened an internal-only help article and gave the customer step-by-step instructions to cancel the service. The account left.

The implementation partner who got pulled in afterward found the cause fast. The agent was wired to a default retriever with access to every knowledge article in the org, including the ones no customer was ever meant to see. Nobody had decided what the agent could reach. Nothing logged the moment it went wrong. And there was no way to pull the action back before it reached the customer. “It is a black box we don’t have control over,” the partner wrote, before admitting the control had never been set up (1. r/salesforce, 2026).

The failure the demo hid

The demo tested the path where everything goes right. Production tested the governance nobody had built. The vendor set the ceiling on what the tool could do. The floor, the part that decides what it’s allowed to do and what happens when it acts wrong, belonged to the buyer. And the buyer hadn’t poured it.

That gap is the real story of AI in marketing right now, and it’s the one most buying conversations skip.

Why the three classes of AI marketing tools share one question

Real Story Group’s Apoorv Durga made a sharp point in his 2026 review of AI marketing vendors: the category has split into classes that don’t belong on the same shortlist (2. Durga, 2026). There’s the AI baked into the suite or CRM you already own. There are specialist tools built for one job. And there are platforms your own team configures. Different buying centers, different data arrangements, different roles once the pilot ends.

His advice for cutting through the demos is good. Make the vendor tell you who can approve an action the agent takes, what gets logged, how a bad action gets undone, and whether you can cap what it spends. Put those four questions to every vendor on the list.

Here’s the part that doesn’t get said out loud. Those are questions about your own operation as much as the vendor’s. The company that lost the customer could have asked Agentforce’s vendor all four and gotten clean answers. It wouldn’t have helped. The failure wasn’t in the vendor’s answer. It was that nobody on the buying side had decided the answers for themselves. And that holds across all three classes. Suite, specialist, or platform, the governance questions underneath are the same, and they’re yours to answer first. A vendor can hand you the controls. It can’t decide your policy.

When the data isn’t ready, that’s the buyer’s problem

The people who build these things say the same thing when you ask what goes wrong. It’s rarely the model. One consultant, describing a wave of Agentforce projects, put it flatly: most organizations can’t run these at scale because their data isn’t ready, and adoption stalls on data quality and process debt, not the AI (3. r/salesforce, 2026). Another, working on the same tool, said it responds differently to the exact same prompt, so you have to decide internally whether 80% right is good enough or whether you need it right every time.

That last one isn’t a setting a vendor ships. It’s a call the buyer makes about their own tolerance for a wrong answer in front of a customer. If you haven’t made it, no demo will make it for you.

Isn’t this just agent-washing?

Fair pushback: plenty of vendors oversell, so isn’t the churned customer really the vendor’s fault? There’s truth in it. A lot of what gets sold as “agentic” is a relabeled chatbot, and the market is crowded with them.

But look at why these projects collapse once they’re live. The pattern isn’t slick vendors winning demos. It’s cost running past the business case, value that never shows up, and controls nobody built. Those are buyer-side failures. Go back to the customer who walked. Even the people in that thread defending Agentforce landed on the same read: if the retrieval pulls something it shouldn’t, something was set up wrong. The setup sits on the buyer’s side of the line. A weak vendor makes this worse. It doesn’t cause it.

The other objection is that this belongs to IT, not marketing. It doesn’t, or at least not only. An agent doing journey work, offer selection, or send-time calls runs on identity, consent, channel ownership, and measurement that marketing owns or shares. Durga’s own advice is to watch who operates the system after the pilot team moves on. If the CMO can’t say what the agent is allowed to do in the brand’s name, nobody can.

Run the AI marketing vendor evaluation on yourself first

So before you shortlist anything, turn the four questions inward and answer them about your own operation. Who approves an action the agent takes for you. What you log, and who reads it. How fast you catch a bad action and reverse it. What caps its spend and contains the damage. If you can’t answer them, you’re not ready to judge a vendor yet, because you don’t know what a good answer sounds like. That readiness is internal muscle you build before the purchase , not something a vendor hands you after.

One reframe makes it concrete. You’re not trialing a reporting tool. You’re handing something the keys to your systems and your customers. Evaluate it the way you’d vet a new hire with that kind of access, not the way you’d kick the tires on a dashboard.

The category didn’t get harder to buy because vendors turned slippery, though some did. It got harder because buying an AI agent now shows you how ready you are to govern one. The company that lost a customer found that out in production. You can find it out first, on a whiteboard, before anyone signs.