Unraveling AI’s Prediction Potential

artificial intelligence making a prediction via crystal ball
Danielle Dyer Photo

8 minutes

Why credit unions should welcome our artificially intelligent overlords with a healthy dose of skepticism

People are wary of artificial intelligence. They’re wondering if AI will turn out to be just another trend or buzzword that fades in time. They’re simultaneously worried that AI will take away their jobs. (In fact, 77% of people are apprehensive about AI-related job loss, according to Forbes.)

“The press is apoplectic about it,” said Joshua Gans, sharing such dramatic headlines as “How AI Could Change Computing, Culture and the Course of History” (from The Economist) and “Why I Welcome Our Future AI Overlords” (from Politico). Gans is a professor of strategic management and Jeffrey S. Skoll Chair of Technical Innovation and Entrepreneurship at the Rotman School of Management, University of Toronto, and presented “From Point Solutions to System-Wide Integration: Unraveling AI's Hidden Potential in Business” at CUES’ 2023 Directors Conference in Maui.

Reactions to AI range from enthusiasm to great fear, Gans noted; there’s so much concern and attention in general that his team has been researching it and the “problem” of AI adoption in business.

Quashing the Hype

In Gans’ own words, his Directors Conference presentation was the “desexification of AI”—he took an interesting topic and made it boring while stripping away the hype.

“Hype leads to large investments, and large investitures in this kind of market lead to a lot of waste,” he explained.

Gans shared an excerpt from a 2020 Forbes article: “In a survey of more than 3,000 company managers about their AI spend, only 10% reported significant financial benefits from their investment so far.” That’s after spending more than $50 billion on AI systems in 2020. “By 2024, investment is expected to reach $110 billion,” the article continues. That’s a lot of investment for only 10% of companies to see real ROI. 

To cut through the hype, Gans discussed four keyints based on his research with Rotman colleagues Ajay Agrawal and Avi Goldfarb:

  1. Today’s AI is prediction technology.
  2. The transformational opportunities for AI come from developing system-level solutions, not point solutions.
  3. System-level solutions take time to emerge.
  4. When system-level solutions do emerge, they will disrupt.


Simply (boringly?) put, AI is an advance in computational statistics.

“It's just prediction,” said Gans. AI uses information that you do have to generate information that you don’t have. So the real effect of the rise of AI is a drop in the cost of prediction, which can take a lot of time, effort and money when done manually by humans.

Do cheap predictions lead to innovation?

Well, yes—but maybe not the kind we’ve been dreaming about. Here’s where the business problem comes in: We need to manage our expectations of AI. “Expecting perfection is over-expecting,” Gans explained. AI can do a good job of predicting—better than a human—but it won’t be perfect. Perfection is a barrier to applying AI in some of the life-altering ways we expected to be reality by now.

For instance, we still don’t have self-driving cars (or not many*). “After spending $2.5 billion over five years, Uber is still far from delivering its self-driving vehicles,” says the aforementioned Forbes article. In fall of 2020, “engineers at the ride-hailing company concluded that its self-driving cars couldn’t drive more than half a mile before encountering a problem.”

Why? “Driving is a prediction problem,” Gans said.

“When you're teaching a teenager to drive, the teenager has a basic grasp of what to do—stop if there's something stopped in front. Go if it’s clear. When you scream at the kid, ‘Didn't you see the pedestrian in the road?’ you're not yelling about their basic ability to drive but their inability to predict the behavior of the surroundings.

“There is always going to be some randomness in driving,” he noted. “There's always going to be a car stopped on the side of the road. There are millions of cars on the road.” That randomness, or the inability to predict, is enough to stop us from allowing AI to drive our cars. Pretty good isn’t good enough.

(*A tragic case in point is the recent news about a fatal crash involving a Tesla driver who was reportedly using Autopilot.)

On the other hand, reducing the cost of arithmetic through the use of semiconductors in calculators and computers led to digital music and video games, Gans pointed out.

If quick, cheap math can revolutionize the creation and distribution of art and entertainment, what can fast, high-volume predictive capability revolutionize? Perhaps we still need to manage expectations. For now, let’s pose the question: “What can fast, high-volume predictive capability make better?” That’s easier to answer in the financial services industry.

Managing fraud is an excellent example. Gans illustrated by recounting his experience with an annual fraud-detection false positive. Each year, he would take a trip to Florida. Each year during that trip, he would buy a new pair of running shoes. “It’d be flagged as fraud every single time,” he said. “Why? … Because they noticed that I don’t seem to buy a lot of shoes. I would literally wait there for the [fraud alert] text.”

This issue has been resolved, Gans said, now that his credit card company’s fraud-detection system has enough information to identify a regular pattern. He’s been taking this trip long enough, and AI can sift through his history quickly enough, to determine the purchase is predictable behavior and allow it to go through.

Human resources is another area where AI is introducing significant efficiencies. But Gans noted that the idea of using AI in HR often is met with pushback. “‘This doesn’t apply to me because I deal with people.’ I have news for them: HR is one of the areas where prediction is the job.”

Joshua Gans
Professor of Strategic Management, Jeffrey S. Skoll Chair of Technical Innovation and Entrepreneurship
Rotman School of Management, University of Toronto
Expecting perfection is over-expecting.

The Revolution Is System Solutions

Improved fraud detection and HR automation are great benefits. But “the transformational opportunities for AI are from developing system-level solutions, not point solutions,” Gans said.

Let’s get back to Uber—or rather, pre-Uber—to explain the difference.

In London, taxi drivers had to spend two to three years studying to pass “The Knowledge” exam, memorizing the entire map of metro London and learning the shortest path between two points at a given time.

Enter navigational AI. “Now people have The Knowledge in their pocket,” Gans said. “We could drop you into the middle of London with a car and an iPhone, and you could drive people around.”

Navigational AI was originally developed as a point solution to help professional drivers, he noted, not for you and me and our ride-share drivers. The business case was for AI to help taxi drivers to predict surges in activity, so the drivers could be in the right place at the right time, increasing efficiency and fares.

As it turns out, navigational AI had almost zero impact on professional, highly skilled drivers. Instead, it primarily helped unskilled drivers. The AI solution enabled people without skills to act as if they had skills, Gans explained.

Thus, a system solution was born. Uber’s founders recognized the business application of substituting AI for skill. A new system of transportation—ride sharing—was created “because navigational AI allowed a much larger pool of people to navigate successfully,” he said.

Today, we have various point solutions for AI, such as fraud detection. We also have such application, or platform, solutions as Occulus, ChatGPT and other generative AI platforms—foundational tools that can be used to create new workflows and point solutions. What we’re currently lacking are system solutions based on those tools. “Without that, transformation will not occur,” Gans observed.

The Time Before Disruption

Don’t despair, AI enthusiasts. “System-level solutions take time to emerge,” Gans said.

We’re currently in the between times for AI—“the time between the demonstration of the technology’s capability and the realization of its promise reflected in widespread adoption.”

After electricity was first introduced, Gans explained, the “turning point was the discovery that electricity could be distributed.” In manufacturing, it was initially used to increase fuel efficiency and cost savings in factories. As adoption grew and factories were freed from their power source, electricity was used in more and more applications, and we began to see innovation in factory design. This led to optimized configuration, more efficient use of materials and modularization. (You may be familiar with Ford Motor Co.’s 1913 assembly line and the genesis of mass-produced automobiles.)

AI is the new electricity. It’s just in its early-1900s phase. As electricity allowed factories to be decoupled from their power source, AI allows judgment to be decoupled from the predictions those judgments are based on. Applying AI is “not just a question of machines making decisions, but different people getting to make decisions,” Gans said. Previously, a single human or group of communicating humans was responsible for both prediction and making a judgment based on that prediction. Now, since AI can take on the burden of prediction, different humans—perhaps those with less predictive capability but better judgment-making capability—can take on the burden of making decisions.

“ChatGPT requires judgment,” Gans reminded attendees. When you write a letter with ChatGPT, you must first decide how to prompt it, and later decide whether or not to send the result. It’s not (yet) writing and sending itself.

As adoption of decision-enhancing solutions increases, it seems likely we’ll draw nearer to a real AI-driven system-level solution. But it’s important to remember point No. 4: When system-level solutions do emerge, they will disrupt. And they might disrupt an existing system or shiny new point solution in which the industry has already heavily invested. cues icon

Danielle Dyer is an editor for CUES.

Attend a Related Learning Event

Learn more from Rotman faculty about artificial intelligence, blockchain and other potential disruptors at CUES Governance Leadership Institute™ II: Emerging Technologies, June 12-14, 2024, in Toronto, Ontario.
Register Now
Compass Subscription