What Agentic Orchestration Actually Means

Picture it. The demo works. People start depending on it. And then one morning you have to explain yesterday's decision to someone who wasn't in the room. A concerned regulator, an angry customer, a court. If you built it as a demo, you can't patch your way out. You have to start over. Welcome back to Invisible Machines. I'm Josh Tyson, joined as always by Rob Wilson, CEO and co-founder of OneReach. Rob and I are co-authors of Age of Invisible Machines, the first best-selling book about agentic orchestration, and today we are talking about just that. Agentic orchestration. Two words the market glued together and then started treating as a contradiction. Agentic has come to mean autonomous, probabilistic, experimental. And in any domain that cannot afford a wrong answer, orchestration means the opposite. Consistency. An envelope the system has to stay inside of. A decision that you can explain to that person who wasn't in the room. You'll hear vendors describe orchestration as a workflow versus agent split or a wiring diagram or an execution order. Analysts are trying to name a layer above those vendors. Gardner isn't a standards body, they don't certify an orchestration the way aviation certifies a flight envelope. What they did is name a layer they call an agent management platform. A useful map, but still not the full picture. The truth is that agentic orchestration is how you meet an objective with a hybrid of code and probabilistic systems, inside an envelope that stays consistent, even when the path through it is not on Rails. The problem is not that agentic systems cannot be mission critical, it's that most of the market optimized for the demo. We're going to start with a surprisingly helpful smoothie analogy from Jonathan Frankel, the chief AI scientist at Databricks. And by the end of this episode, you'll understand why agentic orchestration is the difference between a one-off smoothie in your kitchen and the consistency at scale we expect from a Jumba Juice or a 787 Dreamliner

With these models, we've built nuclear fusion. We've built this incredible unlimited source of energy, just like we built this factory for intelligence. It's not perfect — there's a lot we need to do to improve it — but it's really powerful. And we forgot to build power lines, and put electricity in people's homes, and figure out what an outlet should look like, and build some electrical appliances, and some blenders and, you know, TVs and other stuff. Those are the power lines and the substations and the transformers and, like, the outlets and the electrical system in your home and the blender. And the end thing that the user comes and wants is, "I just want to make a smoothie."

So that that was Jonathan Frankel, the chief AI scientist at Databricks. essentially what he's talking about is orchestration, right? This this idea that we have this intelligence, but we don't really know how to put it to work. So if you're if you're thinking about making a smoothie, you know, you have the electricity, the the appliances, the produce, and then it comes together into this smoothie. But if you're the power company, To you, orchestration is providing the power.

Yeah, I think it's it's like all the things that have to go right. For you to get a smoothie. and and knowing that each one of those systems, all the way from the power company to the power in your house to the appliances in each appliance, depends on who you ask, right? Each one's gonna have a their definition of what orchestration is. at the top level you sort of wanna define it somewhere between mission critical orchestration versus non mission critical. you know Gartner. calls mission critical engineered agents. and that makes sense to us because it's very different requirements and I think the number one difference is in consistency of execution. Meaning a one-off smoothie doesn't have to taste exactly the same one day to the next. but an airplane needs to fly exactly the same way, no matter what pilot's in the seat, every day. consistently. you don't want variation. So we're gonna focus on mission critical orchestration. because non mission critical orchestration becomes mission critical almost invisibly. it's sort of that saying like it's it's not mission critical till it is. that idea that it drifts into mission critical because we start depending on it. And in this particular case, if you built it as a demo, non mission critical like a smoothie, and then it becomes something people depend on. and it stops working, it's a problem and you've got to rebuild it from scratch.

so yeah, Gartner has this higher level view of orchestration. And it part of that is because they're they're working with companies who are trying to orchestrate tools from different vendors, right? And so if you look at each of those vendors, they tend to have their own version of what orchestration is, and unfortunately they tend to be kind of narrow. So if you like look at Anthropic To them, orchestration is is it a workflow or is it an agent? And you know, that that makes sense given what they're providing. They're providing the model. They want to know where autonomy starts and stops.

They'll they'll sort of say workflows are mission critical and agents are not. and I think the issue I have with that is it confuses things because we all know that there's some percentage of each. So when when is a workflow an agent? When is an agent a workflow? What percent of it needs to be deterministic and probabilistic and I think that just gets very confusing when in the field doing this for years, everything is a hybrid and it changes over time. I struggle with that one. their lens is the LLM lens, right? They're they're sort of in the experimental phase of agentic workflows. So making smoothies and We're trying to fly airplanes.

Yeah, yeah. And in reality, orchestration isn't binary in that way. if you look at like Langchain, they're more focused, let's say, on like the connective tissue. sub agents and skills and handoffs and routing. to them, orchestration might look more like a wiring diagram.

where the appliances and the grid and the power generation is out of scope.

Yeah, and that is part of orchestration, but again, it's not it's not the whole thing. and so then if we look at like Google ADK, to them orchestration is what actually gets executed and in what order.

Yeah. What's interesting about them is they're similar to Anthropic, but using different words to describe different parts and sometimes the same words to describe different parts. So it's very confusing because they'll use words that an Anthropic uses but to describe different things. you really go cross eyed when you try to compare them and how they talk about it.

what's consistent is is that it's it's too narrow a view of what orchestration really is. Gartner has this higher level view of it, which helps bring it somewhat into focus, but I think if we take it a step beyond that, what you've been focusing on for a decade at least is is this idea of mission critical deployments.

Yeah, I like to describe orchestration in the context of mission critical, which I think comes down to just creating consistency, which is necessary for controlled environments that are regulated, that are mission critical, like flying an airplane, because that's where our platform was born, from the idea of Every airplane has to account for multiple pilots. Every flight needs to be consistent. we're a smoothie factory, not a one-off smoothie that can taste different from day to day. It has to be the same smoothie that tastes the same. And probabilistic systems are just not designed to produce the same thing. You can ask an LLM five, ten times the same exact question and and get Completely different answers. So trying to make those systems be consistent is very hard and and and it's very much so a hybrid approach. we're not trying to create one-off art here. We're trying to create consistent systems that aren't as brittle as our old systems, because our old systems Are very predictive in nature. You had to predict everything that was going to happen. You had all those if-then statements. So the beauty of LLMs in regulated environments is that you get what we would call in the Dreamliner space, the execution envelope. And I think it's important to understand that.

so if we were to define orchestration as like objectives met by a combination of code probabilistic systems, how that differs from this brittle software you're describing is, with brittle software, there's like one way through, right? But we're talking about experiences where there might be multiple roads Through it, but they all lead to the same place.

Yeah. A a good way to think about it I think is like if our old software was a railroad tracks where the train can only stay on the tracks, and now our new software is, kind of a hovercraft that can go anywhere the hybrid is like cars on the road where restricted on where you can go. You must stay on the road. and it's better than a train and it's in a lot of ways better than a hovercraft because I can't even imagine the chaos that would go on if we were in hovercraft. but we kind of think about this as a hybrid system that that delivers consistency across executions.

if you take your objective and you're thinking about consistency, if you want the same outcome every time, then the list it gets long, right? You need you need to know what context an agent has, who's managing that context. like how do we know that it's using canonical knowledge and not old old copy, right?

Yes. Yeah. Yeah. What if it's searching the web to decide what recipe the smoothie should be? Each time finds a different recipe, right? We're like, no, it has to be this recipe. every agent must use the same recipe. And by the way, who's gonna manage keeping that recipe up to date as ingredients change, right? just very, very different problems. I one one way I like to think about it is like, mission critical agentic systems require an execution envelope and guardrails. And once a car has hit the guardrails, it's too late. Right?

Yeah. Mm-hmm.

Like you need something to keep it from the guardrails. so in an airplane scenario, the execution envelope is The limited space that an airplane can operate in that's safe and consistent so that the passengers and everybody have a consistent and good experience. if that fails, then the pilot jumps in, right? They're the guardrail of that system.

if a pilot's flying a little prop engine plane or something and pulls the yoke back as hard as they can, the mechanics just kick in and it it shoots up and maybe that's safe, maybe it's not. But in the seven eight seven, if a pilot pulls back as hard as they can, the system's then deciding like how how big of an incline can I make safely? And that's that's the actual response. It's not the direct physical result of the pilot pulling on the yoke.

guardrails are when the execution envelope fails, right? just to kind of underscore this, like you know, for our language, we're gonna say that an agent is a hybrid of a workflow and an agent. we're gonna say to us it's neurosymbolic, it's a combination of both. And it's always gonna be some variant combination of somewhat probabilistic to a lot. versus deterministic. And the use case is what we use to determine the ratio. Meaning are we flying a plane or are we making a smoothie on an airplane? Right. To bring Jonathan's analogy into this.

Who doesn't want a smoothie on an airplane?

Yeah, I think it's funny because Google's own docs currently disagree with themselves. Their workflow agent page calls these specialized agents. Their top level agents page says, you know, any agent application that has more than one agent or executable node is considered a workflow. I can't imagine trying to follow all this. Anyway, nobody's arguing for a pure LLM stack and nobody's arguing to go back to pure code.

Yeah, so maybe it's important too, since we are talking a bit about like these these brittle systems that we don't totally want to return to, it's still worth understanding how they emerge and how we how we got here, so back in the day, you know, we had to write everything from scratch, And now it's obviously a blend. LLMs are really good at writing code, but that doesn't mean that people have stopped writing code and they certainly haven't stopped evaluating code.

Or that LLMs don't write code in advance and then execute the code in runtime, right? Which is, I think, one of the pieces a lot of people miss, you know, it's not, it doesn't have to be one or the other. It doesn't have to be that human writes code, deploys it, LLM executes and writes code on the fly, and executes it on the fly. There's this LLMs write code, humans review it, deploy it. and it runs consistently. and I think that that third one is underappreciated. I can't help but going back as developers and realize like in the day we wrote code, you know, very much so from scratch. You started with a blank screen and as time has gone on, we're using NPM packages, we're using APIs. Really like the developer is sort of orchestrating in a sense, right? you think about what know, the way I think Gartner sort of defines this. it's just this idea that d development became about creating all the pieces and parts to just calling and triggering the pieces and parts in a certain order and then determining what order they should run on the fly is what agents are, right? Where that isn't predetermined. And so I sometimes I think understanding it through the the coding world arc is easier because we see that we've been on a long trajectory of orchestration, and that's how to write code faster. And the reason we use NPM packages, the reason we use APIs is because A, yes, it's faster, but B, we're using pre-created systems that are hardened and trustworthy. And it's important to remember that you know, that doesn't go away. The need for pre-created, pretested, trusted pieces and parts is absolutely essential, even if we're orchestrating those in a ad hoc way using LLMs.

you mentioned that you know if you could if you can automate a seven eight seven, you can automate everything. And part of what makes that true, I think, is the complexity of automating a seven eight seven, right? can you talk a little bit about just how complex that was?

Yeah. so the challenge of that was is you're changing the materials that the plane's made out of. You're changing the aerodynamics of the plane. and essentially you can't fly this thing manually. There is no manual option and this this idea of centralization becomes absolutely essential. even though there was some centralized orchestration going on in prior airplanes like the 777 and others, this is where it became absolutely essential and that you couldn't fly the plane without centralized orchestration. This is where the the execution envelope comes into play. Like understanding that if you can automate these systems. reliably, consistently, effectively, then you needed something more than guardrails, right? This wasn't about the plane finding the limits of its aerodynamics in a nosedive and saying, you know, it will stall out and something stopping you from stalling out. This is about the execution envelope being like what is a comfortable way to operate the airplane for the passengers, call that the execution envelope, and then call the limitations of the airplane the guardrails. with this execution envelope, This is the thing that makes every plane experience for every passenger feel consistent. The guardrails is where the thing it where where the thing doesn't crash. And so what we understand is by the time the guardrails are triggered, you've already had a failure in the execution envelope.

you know, within the 787, I think there were like seventy applications. And and it's not like these applications were were generated by Boeing, right? They came from twenty plus different suppliers. So there's there's that wrinkle as well, where it's it's figuring out how to get all these somewhat disparate systems to become part of something centralized.

Yeah, that's why a an open standard was created in that space, you know, so that everybody used the same standard and all of the suppliers could and and that is a important distinction that when we talk about executing in this agentic world, there is no standard. Everybody's trying to come up with their own of version of a standard and and many of the companies wanna own the standard, right? So And think as you look at how Gartner's viewing it, they're saying, like, look, you know, at the top level of orchestration, it's how do you orchestrate a bunch of agents that have all been built using different standards? and I agree with them because the idea is, as much as all of these companies want you to just use their tools, that is just not realistic in this world. This agentic world that everybody's just going to use Google's system or Anthropic's system. it never happens. I mean, the percentage of companies that use both multiple messaging platforms like Slack and Teams, you know, something that makes the least amount of sense is greater than the numbers that only use one, right? in this world, it's just a It's a reality that's a dream for most of these vendors that's just not gonna happen. So I think, you know, this idea that Gartner has of, you know, a a true orchestration platform is going to be an umbrella over multiple systems that themselves ha have their own standards, is probably where this is going. until there becomes an open standard that everybody decides to to settle on, which you know, we'll see.

Well yeah, like part of part of the value prop for this tech is that, the standard should be the organization's standard, not the vendor's standard. Because that's that's what it partially unlocks, is that you can set the standard because you want it to be personalized to your organization and your goals.

Yeah. I mean it kinda comes down to like you cannot ask the company that that has the tools to orchestrate to do the orchestration because they're only going to s prioritize the tools they offer. You can't trust the orchestratee to be the orchestrator. and when when I say like mission critical, just to be clear here, I don't just mean important, it means a decision that has to be explainable and reproducible after the fact by someone who was not there. It's the bar of certification, that is set. It's the agentic systems cannot currently clear this, right? This is something aviation requires. You have to be able to have the telemetry in the system to explain not just how the system made that decision, but how you controlled and planned up front, what did you put in place to ensure that the system made the right decisions? There are obligations not just to explainability of how it made this decision that it made, but also explainability on what you did to ensure the system made the right one.

even if you're a company that's maybe operating a little more reliably under under the kind of umbrella that Gartner's describing, there's still a ton of opportunities for liability, right? And that's reminds me of something that Jeff McMillan described to us. Jeff McMillan was the former firmwide head of AI at Morgan Stanley. He currently has his own consultancy called McMillan AI. let me see if I can find sort of Jeff's breakdown of of why this stuff is so important.

you go to your doctor who accesses your agentic medical profile, who then uses her agent to diagnose your cough, who then uses another agent to prescribe a medicine to you, which then is received by your agentic pharmacy, then an agentic drone picks it up from your pharmacy and flies it to you and drops it on your front door. And you take that medicine and it kills you. Whose fault is it? Like, we are living in this world now, which we are not prepared for that problem,

Man, yeah, that is exactly it. and and I'd say that depends whose lens, you the LLM providers, like they're living in a world where they're not prepared for that problem. I'd say Boeing and what we had to orchestrate with the 787, knows that problem. It was two billion dollars R&D to address that that very problem, so now it's just like can we borrow from those systems? Right? Why why reinvent the wheel here? Let's borrow from those systems and what's already been solved and the two billion dollars already spent. And and the crashes we had to endure from the Dreamliner, right? I don't wanna overlook the fact that that there were some issues. like let's learn from that and and use that as the launching pad for for other mission critical systems.

Yeah, yeah, it makes a lot of sense 'cause very few organizations have two billion dollars to spend on R and D. And and if a lot of this work has already been outlined for you, especially in terms of its complexity and scope, then you'd be wise to check those lessons.

And it's an open standard in the in the airline world. all of this has been figured out when the standard for ARINC was released. there's a number of of things it addresses like partitioning, not sharing, it's not a free-for-all, we're having like fixed schedules. execution envelope, health monitoring. There's like so many a telemetry, there's so many things that that make that system reliable. and so anyway, like why reinvent the wheel? Why don't we borrow from that, which is what, you know, we've done over the years.

Yeah, and it's it's well worn, right? Aeronautical Radio Incorporated was founded in nineteen twenty nine, so

Yep. It's it's a path well walked. what what is that expression? It's a path well worn?

Maybe it's the path well flown.

Well fun. Let's do that. it's funny because like you know, I I'd like to say like why does mission critical agentic orchestration sound like an oxymoron? 'Cause I think it if you're not asking that question, then you're probably building a demo. So I think the market has trained us to hear like agentic as autonomous, probabilistic and experimental. there's a lot of support for that. but it's it's sort of fine until it's not fine, right? problem's not that agentic systems cannot be mission critical. The problem is that most of the market is optimized for the demo. Demos reward flexibility, production rewards, knowing where f flexibility's safe. So let's talk about the I guess five decision patterns. know the LLM generates deterministic logic, which is the model writes the code and the rules and the state machine, but the result determines the system at runtime. So let's call that pattern number one. pattern number two is the LLM makes the decision on the fly. So the model reasons in the moment because the decision Can't be fully specified in advance. Pattern number three is humans make the decision. So human in the loop, the judgment stays with a person, system executes and instruments it. So this is what you were talking about, the execution envelope in practice, where the pilot pulls as hard as he can on the yoke, but the plane only goes up as comfortable as is for the passengers. And then then we have like pattern number four, which is a human writes the deterministic decision logic, which is where we've been for a lot of the automations we have, traditional software. and then pattern five, which I think is the most important one, the hybrid. most real production systems combine the first four into one.

Yeah, this is reminding me actually of something that we heard from Sudhir Hasbe who's the president and chief product officer over at Neo4J. Let just try and find it.

There are deterministic answers and there are non deterministic answers where you want to do a lot of reasoning, send more context and do more free flow problem solving. So I think figure out what is it that is governed already known answer. So you can reduce the token to whatever really least context you can provide to get the answer. 80 plus percent AI projects still fail in enterprises because their data is not structured in the right way for agentic systems

so the question is then how do you choose, you know, which path to take? for for each use case and each decision, you have to ask, like, how predictable are the subtasks? can the rules be completely specified? you know, what is the cost of being wrong? What is the latency requirements? does the decision need to be auditable? If you if you understand like this doesn't need to be done at a system-wide level, this has to be done a use case by use case level. and the one that matters the most is you know, can you explain the decision to a regulator, a customer, an auditor, or a court? more

Yeah, especially that last one probably, right?

can you explain The you went into or The the things you did to ensure that the system operated as intended. It's not enough just explain how the thing made a decision. It's more important to explain how you ensured the right decision was made. Yeah, every orchestration c system already contains these five patterns if they're gonna work. Or has to contain these, I should say. They don't. The question isn't like how they work. It's who's allowed to decide. and once you answer that, the other problems sort of appear. How do you build a system around those decisions? in a purely deterministic system, it wasn't the system that decided it was a human. In a purely probabilistic system, it was the system that decided. Now you ask If a human must take responsibility for critical decisions, how could any system be purely probabilistic? And I guess we get into episode two, which will be do we trust it?

Yeah. Yeah. How do you build a system those decisions and make it trustworthy? That's our look at what agentic orchestration actually is. The next question: how can you trust it? Stay tuned as we explore trustworthy agentic orchestration, what it takes for a hybrid system to stay inside an envelope once people start depending on it. If this was useful to you, like and subscribe so you don't miss that episode. And thank you to everyone who works behind the scenes to make this podcast what it is. We could not do it without your help. Until next time.