Below is an excerpt from Yoav’s contribution to Google Cloud’s ‘Future of AI: Perspectives for Startups 2025’ report.

Frankly, I hate the term AGI, or Artificial General Intelligence.
It’s not a thing. I’ve been around the block enough to know that intelligence is multifaceted. Machines will undoubtedly be able to automate more and more functions, but there’s a false sense of there being a discrete point at which that mythical AGI will have been reached. I believe this loose thinking and hype around AGI is a distraction. Instead, let’s talk concretely about AI technology, its strengths and weaknesses.

Why businesses are experimenting madly with AI, but are cautious about deploying it at scale

You see this in the enterprise. While consumer adoption of AI has set a record pace, business has been slower to adopt it. Certainly, CEOs and boards everywhere are presenting their company as being “AI first” (or planning to become that), and are experimenting heavily, sometimes with hundreds of use cases. But for all of the mass experimentation going on in enterprise, only a fraction of AI projects actually reach deployment. This boils down to two key challenges:

  • Cost: The high cost of running LLMs challenges the economics of business software. 
  • LLMs produce wonderful output alongside utter nonsense:  Imagine writing an investment memo or responding to a customer, and your AI is brilliant 95% of the time but produces garbage the other 5%. That is a showstopper in the enterprise.

The first challenge is due to the inherent cost of serving (let alone training) LLMs, and will be dealt with by a combination of two methods. One is using smaller LLMs (the term Small Language Models, or SMLs, is making the rounds), those “tiny” sub-7B and even sub-3B parameter models. The other method is using different, more efficient architectures than the standard transformer architecture; AI21’s Jamba family of models is an example.

The second challenge is particularly acute, and more challenging. The phenomenon is inherent to the probabilistic nature of LLMs, and it’s delusional wishful thinking that this will go away, no matter how much effort is placed on things like “alignment”, “guardrails”, and such. I believe that we’re going to wean ourselves from what I call our current “prompt and pray” modus operandi. The industry will realize that LLMs are an element of a more comprehensive AI system, which can seamlessly integrate and leverage the strengths of various AI technologies, including LLMs, retrieval, tools and other traditional code. AI systems will offer greater control, efficiency, reliability, especially when tackling tasks that require nontrivial reasoning, which most tasks do.

A healthy practice for startups: Use AI systems, but focus on “product algo” fit

AI systems that combine multiple LLMs and other tools offer a compelling solution. These systems allow for better cost and compute management by intelligently routing tasks to the most suitable resources. For example, a smaller LLM could act as a “router,” directing tasks to specialized LLMs or tools for optimal efficiency. These systems can also enhance reliability and quality by incorporating checks and balances during the computation.

But as you apply these AI systems to real-world problems, you should approach this wisely. My common advice to AI startup leaders is to strive for “product-algo” fit. What I mean by that is while AI systems will be a dramatic improvement over barebones LLMs in terms of reliability and efficiency, they will still be imperfect; the underlying uncertainty involved in LLM calls, search and retrieval will not completely go away. So as an entrepreneur creating a new product, understand the strengths and weaknesses of the technology, and craft that product in a way that leverages their strengths and compensates for their imperfections. This is what I call “product-algo fit”.

To read more from Yoav and other AI leaders, download the ‘Future of AI: Perspectives for Startups 2025’ report from Google Cloud here: goo.gle/42NrjVo