Depending on who you ask, AI is either changing everything — or it’s just in the way.
Eliassen’s 2026 Technology Leadership Pulse Survey found that 64% of directors, vice presidents, and C-suite leaders said that their organizations had realized positive return on investment (ROI) from AI implementations. Meanwhile, a multi-year survey from Wharton reported that three quarters of leaders said their organizations were seeing positive ROI from AI investments, while a recent report from PwC found that 30% of CEOs said they’d seen increased revenue from AI, and a quarter said it helped them lower costs.
On the other hand, an MIT study found that 95% of AI pilots delivered no measurable return, while just “5% of integrated AI pilots are extracting millions in value. The vast majority remain stuck with no measurable P&L impact.”
So what gives? Are AI implementations delivering value, or aren’t they? And if they are delivering value, what form does that value take? What kinds of AI projects are leading to real wins? And why doesn’t anyone seem to agree on, well, anything when it comes to AI?
What Does “AI Implementation” Mean?
One foundational challenge today’s tech leaders, and the outlets that report on them, continue to contend with is the very definition of “AI implementation.” Does “implementing” AI mean deploying Claude Code, using something like OpenClaw to manage calendars and tasks, or “vibe coding” with Replit? Does an implementation have to be enterprise-wide, or does a select few engineers using a third-party AI solution count?
“At the moment, many of those options are valid examples of an AI implementation,” said Kolby Kappes, AI and data services practice leader at Eliassen Group. “Today, the best definition of ‘AI implementation’ may be ’a strategic, business-case-driven decision to build or buy an AI solution that functions as an enabling technology or a technology that empowers core business processes.’”
So what doesn’t count as an AI implementation?
“Over the past 12 to 24 months, almost every software vendor has bolted some AI functionality onto their existing solutions,” Kappes said. “And many of those deliver value — but those don’t really fall into the category of an implementation. In some cases, those AI features are being rolled out to customers who didn’t ask for them and even some who don’t want them. Since that’s not an intentional choice driven by a real business case, we can’t really put the ‘implementation’ label on those.”
Expectations Versus Reality: What Can AI Implementations Achieve?
If what constitutes an “AI implementation” is murky, what those implementations are expected to achieve is murkier still.
The MIT report referenced above states that: “Adoption is high, but transformation is rare. Only 5% of enterprises have AI tools integrated in workflows at scale and 7 of 9 sectors show no real structural change.”
“Transformation” and “structural change” are lofty goals, and some organizations are clearly getting there with AI. But are they the only measures of success? What about cost savings or increased efficiency for a single team or across a department? After all, our survey found that almost 70% of leaders who had seen positive ROI from AI implementations said that cost savings were a key outcome, while 64% said AI enabled them to be leaner and more efficient.
“What leaders expect from AI depends on which you’re talking to,” Kappes said. “Some seem to think AI has to revolutionize everything from the codebase to the org chart in order to be a success. Others are content to see AI eliminate manual processes for a team or reduce the demand on functions like customer service. They’re both right, in a way. Instead of looking for one overarching definition of ‘ROI’ when it comes to AI, companies should consider what’s reasonable — and feasible — today before looking to the future.”
In other words, Kappes noted, it may not be realistic for a 500-person logistics company to expect AI to completely automate business processes in the short term. But that company can likely see real ROI by using AI to automate some front-end customer service, order tracking, and data analysis.
“If ‘transformative’ comes later, that’s fine,” he added. “For now, projects with smaller stakes are often a great way to build internal AI competencies, solve data quality issues, and develop governance strategies. Once these foundational pieces are in place, then the transformative work can begin.”
What is AI Really Disrupting?
At Colgate-Palmolive, AI has been deployed selectively — but thoughtfully — across business functions with measurable business value, like creating marketing materials or aiding in innovation. In one case, the LLMs used by the teams at the CPG giant have been augmented with retrieval-augmented generation (RAG) content, like proprietary research, trend analysis, and syndicated data sources, among others. This enables their Gen AI to “quickly go through such material and describe market trends and unmet consumer needs.”
In another, Colgate-Palmolive:
“[F]ound that they could combine one AI system that surfaces unmet consumer needs with another proprietary AI system that develops new product concepts to meet those needs. In minutes, with human guidance, it can produce copy and imagery for a new concept, such as a new flavor of toothpaste.”
Meanwhile, at Liberty Mutual, AI is helping the insurance company’s underwriters “make faster, more informed decisions, while generative AI copilots assist claims teams through tasks such as document summarization and workflow automation—reducing friction and freeing up capacity for higher-value work.”
At Dun & Bradstreet, leaders worked with IBM Watson to develop an internal product called D&B Ask Procurement, a “a conversational chat experience that automates repetitive tasks and simplifies complex procurement processes.” This experience enables employees at the business data leader to deliver faster, more comprehensive supplier risk evaluations and revenue-based scores to its customers that are evaluating potential vendors.
When is AI Just Disruptive?
Not every organization’s experience with AI has been as successful as the ones listed above. One fintech startup famously replaced the employees within its customer service function with AI agents, only to do a very public about-face — meaning they rehired humans — shortly thereafter. “Cost unfortunately seems to have been a too-predominant evaluation factor when organizing this,” their CEO told Bloomberg. “What you end up having is lower quality.”
This startup is hardly alone. Some 55% of UK business leaders reportedly regret their decisions to replace humans with AI agents.
But it’s not just businesses that opted to replace humans with AI agents that are finding AI more disruptive than disrupting. A startling number of companies have been plagued by the phenomenon of “workslop,” or AI-derived output that “masquerades as good work, but lacks the substance to meaningfully advance a given task.”
“Workslop” most often occurs when employees enter a basic prompt into their AI of choice and uncritically deliver its output. This slows productivity and increases workplace frustration, since it forces other employees to review and correct the mistakes made by the AI.
There’s a throughline across the AI success stories and the stories of failure and frustration: Companies that deploy AI selectively and intentionally to solve discrete business challenges tend to succeed. They’re even more likely to succeed when they give tech leaders the time, tools, and, often, the expert partners, to get the implementations right. On the other hand, those that simply replace teams wholesale with untested AI agents, or those that give employees access to AI tools without training, governance, or clear expectations, tend to suffer.
“AI isn’t an expert,” Kappes stressed. “It’s an intern that can, eventually, do what you train it to do. But you can’t expect it to do everything, and you can’t expect it to complete even limited tasks without human oversight.”
Takeaways for Tech Leaders
Depending on the day, media coverage and industry experts alike will depict AI as either a transformative technology on par with electricity, or a messy, expensive distraction that makes work more complicated. AI’s ability to generate ROI may be painted as either grim or grand.
The truth, as it almost always is, lies somewhere in the middle.
“AI isn’t going to transform your business simply because you ask it to,” Kappes said. “What it may do is transform a process or a function, or it may streamline a task in a way that gives humans more time to focus on higher-order things.”
Either of those outcomes may make an investment in AI worthwhile, but getting there means organizations need to:
- Identify a clearly defined business problem that AI can solve or help solve
- Dedicate the time, resources, and expertise to implementing the right AI solution with the right capabilities
- Develop clear benchmarks and metrics for success or failure
“When companies take these actions and deploy AI smartly, they tend to succeed,” Kappes said. “When they ‘implement AI’ by giving every employee a license for a third-party LLM with no training or guardrails, they tend to conclude that AI is more trouble than it’s worth.”
To get more expert insights like these on AI, cybersecurity, tech talent, and more, visit our resources page today.
