AI success: Real or hallucination?

Ever consider how much of AI is a hallucination? Not how much AI output is hallucination, but rather how much of what we hear or read about it is genuine? Could it be that AI success is mostly hype, and that all the hype may be hurting the technology rather than helping? Enterprises need to get to the bottom of AI, not in technical terms but in terms of making a business case. How that happens is what will determine whether AI is real or maybe a hallucination.

We’ve all heard the stories about how AI is revolutionizing work, how it’s saving companies tens of thousands of hours of labor, and how professionals all need to be learning to be effective users of AI. While that may save your job for a while, the storylines say that AI is probably going to eliminate it eventually, and maybe eliminate you, too. Most people I talk to dismiss the extreme stories about AI harm, but how about the extreme stories about its benefits?

AI is mostly hype. And the hype is hurting AI, in the sense that it’s misdirecting a lot of effort and thus hindering adoption. But AI is revolutionizing work, and it is saving time…often lots of it. It’s something you should learn to use, and maybe some should learn to fear. It’s not coming for your job, though. And, in fact, right now the most popular uses of AI couldn’t make a business case. Does any of this make sense? We’ll see.

Hype isn’t necessarily a bad thing. Building buzz generates interest, gets new technologies tried, helps your salespeople get appointments, and keeps your company in the news. Apple’s AI announcements got a lot of attention. And though many said its news was pedestrian, even boring, what could Apple do but announce AI that does the stuff everyone is already writing about? Sometimes you need to erect a massive billboard that says what everyone else is saying; it’s called marketing. We keep clicking on extravagant AI stories, so it’s working.

Wall Street loves AI, and it rewards companies that make AI announcements, even just partnerships with an AI vendor or the launching of AI tools that only match what competitors are doing. Most of this activity relates to generative AI, and most to personal productivity tools and capabilities, something we can all relate to. With all this AI love floating about, it’s no wonder people think it’s taking over the world.

But sometimes you have to wonder if we’re looking in the right places to gain insight into AI value and progress. I recently read a story about how AI had saved dozens of person-years of time in a few months. But did this make a business case? Truth is, we don’t know. What was the per-worker benefit, and how much did it cost per worker to save that time? Did any of the time savings translate to saving the company money, cutting current positions or eliminating the need to hire into new ones? If your average worker makes twenty dollars an hour, and AI tools save one hour a week, how much does that save the company? Nothing, unless you dock the employee an hour a week or can use the time to eliminate the need to hire someone else. We never seem to hear about these points, but they’re the ones that matter to CFOs.

Everyone likes to save their own labor, to unload repetitive and boring tasks, and so there’s a lot of interest in this sort of AI application. The problem is that this kind of worker assistance doesn’t seem to monetize. I was told by CIOs who audited the use of generative AI in their companies that generative AI tools to help workers with documents, emails, etc. almost never actually saved money. However, the tools their companies were trying were either free or very cheap, and the CIOs said they might save a bit of time or improve results a bit. One CFO said that most of the AI we hear about is what we could fairly call “AI candy.” Or maybe another AI hallucination?

There are good AI projects that save money, though, and CIOs say that these do typically involve the CIO and go through the traditional CFO assessment. For example, almost every company that has tried AI as a customer support chatbot says that it has improved customer service and lowered costs, and CIOs say that they’ve had no difficulty making a business case for AI in that mission. In applications involving network management, just over 80% of CIOs said their projects met ROI goals, and three-quarters of companies that self-hosted AI models for business analysis said their projects met business goals. Why don’t we hear about these?

The problem, say CIOs, is that in the early days of adoption of any technology, it’s the potential of the technology that gets the most attention. Generative AI is easy to play with, so a lot of people do, and their interest gets a lot of attention. The specific challenges of self-hosting AI are of interest to a small number of users, too small to attract as much attention. In addition, the small number of enterprises actually involved in CIO-driven, self-hosted projects means that it takes longer to expose all the real issues to be faced, to answer the questions that will be the most critical for AI adoption.

Going back to those CIOs, over three-quarters say that they don’t have staff skills needed to launch an AI project and bring it to completion. A few admit to having hired for such a project and then found it couldn’t make the business case, pass a compliance audit, or both. Apparently, all this dabbling with generative AI in personal productivity applications isn’t educating the workforce in a way that supports staffing those real-world, money-saving, projects. One CIO told me that there were 20 training sessions offered on writing generative AI prompts for every one course about hosting business analysis applications in a way that could pass compliance muster.

The biggest problem may not be compliance muster, but financial muster. If AI is consuming hundreds of thousands of GPUs per year, requiring that those running AI data centers canvas frantically in search of the power needed to drive these GPUs and to cool them, somebody is paying to build AI, and paying a lot. Users report that the great majority of the AI tools they use are free. Let me try to grasp this; AI providers are spending big to…give stuff away? That’s an interesting business model, one I personally wish was more broadly accepted. But let’s be realistic. Vendors may be willing to pay today for AI candy, but at some point AI has to earn its place in the wallets of both supplier and user CFOs, not just in their hearts. We have AI projects that have done that, but most CIOs and CFOs aren’t hearing about them, and that’s making it harder to develop the applications that would truly make the AI business case.

So the reality of AI is buried in hype? It sure sounds like AI is more hallucination than reality, but there’s a qualifier. Millions of workers are using AI, and while what they’re currently doing with it isn’t making a real business case, that’s a lot of activity. Is it possible that someone in this mass-market AI group, will uncover something that creates enough real value to pay all those AI bills and earn a profit? Is it possible that just having millions demand AI would eventually compel companies to accept paying for the same applications CIOs and CFOs scoff at today? If that’s what happens, then today AI isn’t a hallucination after all, and it might prove that those CIOs and CFOs, and maybe even me, are the ones hallucinating…but I wouldn’t bet on it.