Flag

We stand with Ukraine and our team members from Ukraine. Here are ways you can help

Get exclusive access to thought-provoking articles, bonus podcast content, and cutting-edge whitepapers. Become a member of the UX Magazine community today!

Home ›› AI Agents ›› Orchestrating LLMs, AI Agents, and Other Generative Tools

Orchestrating LLMs, AI Agents, and Other Generative Tools

by Robb Wilson, Josh Tyson
5 min read
Share this post on
Tweet
Share
Post
Share
Email
Print

Save

In the evolving world of conversational AI, orchestrating LLMs, AI agents, and generative tools is key to achieving hyperautomation. This article explores the importance of open, modular systems that allow businesses to innovate and scale while avoiding the pitfalls of relying on a single vendor. Learn how to build an interconnected AI ecosystem that fosters continuous improvement and adapts to the future of artificial general intelligence.

In an ecosystem built for the orchestration of LLMs, AI agents, and other generative tools, conversation is the tissue that connects all the individual nodes at play. A collection of advanced technologies is sequenced in perpetually intelligent ways to create automations of business processes that continue getting smarter. In these ecosystems, machines are communicating with other machines, but there are also conversations between humans and machines. Inside truly optimized ecosystems, humans are training their digital counterparts to complete new tasks through conversational interfaces — they’re telling them how to contextualize and solve problems.

These innovations, algorithms, and systems that get sewn together start to build what’s referred to as artificial general intelligence (AGI). Building on the idea of providing machines a balance of objectives and instructions, and a sort of system that’s achieved AGI, will only need an objective in order to complete a task. This leads to the more imminent organizational AGI we’ve been talking so much about. Josh wrote about this connection in an article last year for Observer:

There’s the immediate and tangible benefit of people eliminating tedious tasks from their lives. Then there’s the long term benefit of a burgeoning ecosystem where employees and customers are interacting with digital teammates that can perform automations leveraging all forms of data across an organization. This is an ecosystem that starts to take the form of a digital twin.

McKinsey describes a digital twin as “a virtual replica of a physical object, person, or process that can be used to simulate its behavior to better understand how it works in real life.” They describe these twins inhabiting ecosystems similar to what we’re describing here, that they call an “enterprise metaverse … a digital and often immersive environment that replicates and connects every aspect of an organization to optimize simulations, scenario planning, and decision making.”

Something as vast as an enterprise metaverse won’t materialize inside a closed system where the tools have to be supplied exclusively by Google or IBM. If you’re handcuffed to a specific LLM, NLP, or NLU vendor, your development cycles will be limited by their schedule and capabilities. This is actually a common misstep for organizations looking for vendors: it’s easy to think that the processing and contextualization of natural language is artificial intelligence — a faulty notion that ChatGPT in particular set ablaze. But LLMs and NLP/NLU are just individual pieces of technology that make up a much broader ecosystem for creating artificial intelligence. Perhaps more importantly, in terms of keeping an open system, LLMs and NLP/NLU are one of many modular technologies that can be orchestrated within an ecosystem. “Modular” means that, when better functionalities — like improved LLMs — emerge, an open system is ready to accept and use them.

LLMs, a common stumbling block

In the rush to begin hyperautomating, LLMS have quickly proven to be the first stumbling block for many organizations. As they attempt to automate specific aspects of their operations with these tools that seem to know so much (but actually “know” basically nothing), the result is usually a smattering of less-than-impressive chatbots that are likely unreliable and operating in their own closed system. These cloistered AI agents are unable to become part of an orchestrated effort and thus create subpar user experiences.

Think of auto manufacturing. In some ways, it would be easier to manage the supply chain if everything came from one supplier or if the manufacturer supplied its own parts, but production would suffer. Ford — a pioneer of assembly-line efficiency — relies on a supply chain with over 1,400 tier 1 suppliers separated by up to 10 tiers between supply and raw materials, providing significant opportunities to identify and reduce costs and protect against economic shifts. This represents a viable philosophy where hyperautomation is concerned as well. Naturally, it comes with a far more complex set of variables, but relying on one tool or vendor stifles nearly every aspect of the process: innovation, design, user experience — it all suffers.

Strive for openness

“Most of the high-profile successes of AI so far have been in relatively closed sorts of domains,” Dr. Ben Goertzel said in his TEDxBerkeley talk, “Decentralized AI,” pointing to game playing as an example. He describes AI programs playing chess better than any human but reminds us that these applications still “choke a bit when you give them the full chaotic splendor of the everyday world that we live in.” Goertzel has been working in this frontier for years through the OpenCog Foundation, the Artificial General Intelligence Society, and SingularityNET, a decentralized AI platform which lets multiple AI agents cooperate to solve problems in a participatory way without any central controller.

In that same TEDx talk, Goertzel references ideas from Marvin Minsky’s book The Society of Mind: “It may not be one algorithm written by one programmer or one company that gives the breakthrough to general intelligence. …It may be a network of different AIs, each doing different things, specializing in certain kinds of problems.”

Hyperautomating within an organization is much the same: a whole network of elements working together in an evolutionary fashion. As the architects of the ecosystem are able to iterate rapidly, trying out new configurations, the fittest tools, AIs, and algorithms survive. From a business standpoint, these open systems provide the means to understand, analyze, and manage the relationships between all of the moving parts inside your burgeoning ecosystem, which is the only way to craft a feasible strategy for achieving hyperautomation.

Don’t fear the scope, embrace the enormity

Creating an architecture for hyperautomation is a matter of creating an infrastructure, not so much the individual elements that exist within an infrastructure. It’s the roads, electricity, and waterways that you put in place to support houses and buildings, and communities. That’s the problem a lot of organizations have with these efforts. They’re failing to see how vast it is. Simulating human beings and automating tasks are not the same as buying an email marketing tool.

The beauty of an open platform is that you don’t have to get it right. It might be frightening in some regards to step outside a neatly bottled or more familiar ecosystem, but the breadth and complexity of AI are also where its problem-solving powers reside. Following practical wisdom applied to emergent technologies — wait until a clear path forward emerges before buying in — won’t work because once one organization achieves a state of hyperautomation, their competitors won’t be able to catch them. By choosing one flavor or system for all of your conversational AI needs, you’re limiting yourself at a time when you need as many tools as you can get. The only way to know what tools to use is to try them all, and with a truly open system, you have the power to do that.

As you can imagine, this distributed development and deployment of microservices gives your entire organization a massive boost. You can also create multiple applications/skills concurrently, meaning more developers working on the same app, at the same time, resulting in less time spent in development. All of this activity thrives because the open system allows new tools from any vendor to be sequenced at will.

This article was excerpted from Chapter 11 of the forthcoming revised and updated second edition of Age of Invisible Machines, the first bestselling book about conversational AI (Wiley, Apr 22, 2025).

Featured image courtesy: by north.


post authorRobb Wilson

Robb Wilson

Robb Wilson is the CEO and co-founder of OneReach.ai, a leading conversational AI platform powering over 1 billion conversations per year. He also co-authored The Wall Street Journal bestselling business book, Age of Invisible Machines. An experience design pioneer with over 20 years of experience working with artificial intelligence, Robb lives with his family in Berkeley, Calif.

post authorJosh Tyson

Josh Tyson
Josh Tyson is the co-author of the first bestselling book about conversational AI, Age of Invisible Machines. He is also the Director of Creative Content at OneReach.ai and co-host of both the Invisible Machines and N9K podcasts. His writing has appeared in numerous publications over the years, including Chicago Reader, Fast Company, FLAUNT, The New York Times, Observer, SLAP, Stop Smiling, Thrasher, and Westword. 

Tweet
Share
Post
Share
Email
Print
Ideas In Brief
  • The article explores how conversation connects LLMs, AI agents, and generative tools in business automation.
  • It stresses the need for open, modular systems over relying on single vendors for scalability and innovation.
  • The article highlights the drawbacks of isolated LLMs and emphasizes interconnected systems for effective AI-driven workflows.
  • It encourages embracing the complexity of AI ecosystems to enable flexibility, iteration, and hyperautomation.

Related Articles

Are we on the brink of an AI-first revolution? As more products are built entirely around AI engines, designers must adapt. From dynamic interfaces and non-linear journeys to helping users optimize prompts, discover how the next generation of AI-driven products will reshape UX design.

Article by Tom Rowson
AI-First: Designing the Next Generation of AI Products
  • The article introduces “AI-first” products, designed around AI engines to offer more than just chat interfaces and improve over time.
  • It highlights key challenges for designers: creating flexible interfaces, helping users with prompts, and managing AI errors like hallucinations.
  • The piece stresses the need to adapt to non-linear, iterative user journeys as AI-first apps evolve.
Share:AI-First: Designing the Next Generation of AI Products
4 min read

Unlock the power of AI with smarter data strategies. Discover how digital twins, semantic technologies, and AI-ready documentation can transform your business operations and decision-making.

Article by Vlad Radziuk
From Siloed Assets to a Digital Twin: a Business-Focused Guide for Digitizing Your Enterprise
  • The article explores how digital twins can unify enterprise data and improve decision-making and discusses turning documentation into structured, AI-usable assets.
  • The piece highlights semantic technologies for clearer business-IT collaboration and examines decentralized platforms for streamlining business operations.
  • It emphasizes the importance of curating business data for AI access.
Share:From Siloed Assets to a Digital Twin: a Business-Focused Guide for Digitizing Your Enterprise
11 min read

Discover how AI-first design principles let you build beautiful, functional UIs in minutes — without ever opening Figma or writing any code.

Article by Adam Judelson, Ryan Brotman
Making Designs Without a Designer
  • AI-first design turns simple text prompts into fully functional, production-ready UIs — no coding or Figma required.
  • Learn how to develop structured AI prototyping workflows to eliminate bottlenecks, and ensure fast, scalable, and consistent UX across projects.
  • This isn’t just faster design — it’s the future of product design, making high-quality UI creation accessible to everyone.
Share:Making Designs Without a Designer
10 min read

Join the UX Magazine community!

Stay informed with exclusive content on the intersection of UX, AI agents, and agentic automation—essential reading for future-focused professionals.

Hello!

You're officially a member of the UX Magazine Community.
We're excited to have you with us!

Thank you!

To begin viewing member content, please verify your email.

Tell us about you. Enroll in the course.

    This website uses cookies to ensure you get the best experience on our website. Check our privacy policy and