What AI as Cheap Prediction Means for Enterprise

Well, Joshua, thanks so much for joining us today. you know, normally I like to kick us off with a question, but I think Rob has been, has been eagerly awaiting this conversation for a long time. He's a really big fan of prediction machines. and has recently been, yeah.

I have notes, I try to fit

them all into one, two pages and it's becoming microfilm.

Yeah, and he's frequently said that prediction machines, he feels like now that book has only grown in relevance. So I will let Rob select from his notebook and kick us off because he's been really excited to talk to you.

boy, I don't know where to start actually. I was thinking AI is prediction. and prediction decision are very related. And then prediction, decision, and friction. These are like an ecosystem here.

Alright. all right. so about a decade ago here at the University of Toronto, actually more than a decade, we started to see startups through our program, the Creative Destruction Lab, they were all claiming to do artificial intelligence. And there were enough of them that we started to wonder what they were talking about. And, you know, because, you know, like,

What do those people do anyway?

Yeah, yeah, exactly. You know, it was hard to say. didn't know, you know, what had changed and things like that. And of course, it didn't take us long to discover that there was sort of a quiet revolution had sort of begun to bear fruit in computer science with deep learning, deep neural networks that were suddenly able to do things like classify images at a level that was superior to human benchmarks and things like that. And the techniques that they were using, while it was all couched in terms of this sort of brain analogy of neural networks itself, was, when we looked into it, really an advance in statistics. It was being able to take the weight of now superior computational power. and able to do statistical prediction on a much greater fidelity and speed and of course cost as well.

All right. Yeah, I love that quote in your book where you say in 1954, the term artificial intelligence was just a poor choice of words that we live by today.

Yeah, yeah, that was Ted Lieu who said that, know, poorly, poor definition. And really, you know, we sort of are hanging off that. But our role as economists in this situation was to pour water on the hype. so artificial intelligence is a very exciting term with a long tradition of science fiction and

out.

thinking about philosophy and existential risk and all those sorts of things. And so we wanted to take that term that was extremely interesting and make it completely boring, which was basically to say, well, no, no, no, no. What's really going on here as we began to understand it was an advance in computational statistics. And if we were being true to ourselves, we would just name it as advances in computational statistics.

but there's literally no one who wants to do that. It's not going to motivate anyone to get interested in researching in this area with that sort of title. But the thing is that we had these businesses were at the same time trying to evaluate as this was growing, you know, how much attention they should pay to it and how they should think about it because it was being hyped like artificial intelligence, which at least suggested it was going to be doing a lot more.

now you

things than it was actually capable of given its statistical foundation. And so we set about writing the books, the book basically saying, look, as an economist, when you get a technological revolution, the way to distill it down is to understand what it was dropping the cost of, what was scarce that is now abundant. And so

don't know.

you know, with computing power. Computers are just arithmetic machines. And what the computing revolution was, was making that arithmetic really cheap. And what does that do? That allows you to do arithmetic easily, but also opens up a whole lot of things that you didn't think were arithmetic to be that. And so it was, we thought, with artificial intelligence, that this was an advance in prediction. It was going to make predictions in a lot of areas.

That's it.

and faster and reliable, etc. And so what were the effects going to be of that? Well, one is it'll be very useful where we already predict things like credit card fraud and other things like that. But the more interesting stuff is where we suddenly decide that something that wasn't a prediction problem is a prediction problem, such as the ability to have self-driving cars was our primary example there. And that was basically where we were circa 2017 or 18 in arguing that case, which was a useful framing for businesses in terms of where they thought the applications could be. But of course, we hadn't seen, what is it the predictions really going to be able to do when it opened up new avenues? And of course, we found that in 2022. with chat GPT, which it turned out that, you know, beyond anyone's wildest expectations, prediction was able to handle language and be able to digest information for us in a way that we just didn't think was possible anytime soon. And so you really got... Still the same thing going on. It's still prediction, next token prediction, they call it with large language models. But it's a completely different set of applications and things that were cracked that people didn't anticipate even a decade.

Yeah. One thing you say in here, which was interesting, I've always kind of said that, know, sentences or words or language are like the carriers, they're like the packaging for ideas. And so when you predict the next word, you're essentially predicting the next idea. But what I liked about how you guys, and I don't know if you want to call it narrow it, but like, what the kind of idea like the next decisions. so these words carry decisions And I guess that's a bit of a surprise, think, you know, thinking next word doesn't sound interesting, but when you realize they carry decisions as an emergent property, it's interesting.

Well, it's... Yeah, well, it's more like, it depends on how you're viewing these things. If you view it as merely a friendly conversational machine, that's one sort of use for it. It's not helping you with decisions. It might be, but it's something else. But if you think about it as a tool by which you ask things, and you're either asking for, you can do numerous things with it. You can ask it for information that will help you make a decision.

Right. something. Alright.

You could ask for information to help you clarify your understanding to help you make a decision. You could also ask it to do stuff, which are basically to return things to you that are outputs like, you know, text or a letter or something like that. And then you make a decision about whether it's good enough to be able to send it. So at each step of the way, you know, in terms of its actual use value, what you're doing is you're asking for

Thank Right.

information and that's helping you make a decision. Now maybe the decisions are slightly different than what you would do if you didn't have the LLM around and things like that but those are the things we're just discovering now how to do.

Yeah. And that's true. Like that sort of relates to the friction. Like it lowers the friction of decision making. And that's, guess, where the friction comes in, right? It makes it easier to make decisions in some cases. Some cases harder, but most cases easier.

Right. Well, might, you know, how would we think about that? We think about a decision as something that you're taking information on, you're deliberating, you're weighing, and you're producing an output. And obviously, the gathering the information part can be quite substantial. And we've now, you know, in a lot of domains, reduced that. What would you have done if you didn't have that information? What you'd do is you wouldn't make decisions.

greatest.

for want of a better term, you'd engage in, you'd have rules. So, you know, it may be that previously when you didn't have a weather app, you would, you know, either carry an umbrella all the time or not carry it. And you'd never make a decision about it or you've made a decision in your life, but not day to day. But now that you've got that, now that you've got that app, I'm sorry, my light turned off. Now that you've got that app.

Right. well. That's it.

You can, you can. Exactly. It's weird. Anyway, it's, it's, it's an automated thing if it were, if it paying more attention to me. The, now basically you've got a weather app, you can make decisions, you know, the sense of you can choose not to pay attention to that and do what you did previously, but now you can sort of optimize a little bit more.

Just makes the things you say more powerful when it changes. It's like lightning and thunder. feel like I'm... Yeah.

by enabling those decisions. And I think that's what we're seeing in a lot of places. So, you know, there is a sense in which it becomes easier to make decisions, so you're actually right that way. But there's also a sense in which, you know, it only saves some of the cost of making a decision. So it's a, know, and in some cases it's substantial, in other cases it's not very much at all.

Mmm. Yeah. One of the things you meant, you mentioned in the book is an attempting to try to assign a value to good decisions or better decisions, you know, trying to quantify that.

Right, right. I mean, the way we sort of conceptualize it, it's not an unusual way to do so, is that when you don't have good information, you're making guesses and you're decisions based on those guesses. So you're gonna, you can make different errors associated with that. And, but when you get a better prediction, you can reduce the cost associated with those errors. In the case of the umbrella example, the cost of

carrying an umbrella when it is sunny outside versus the cost of getting wet. Right? And those costs are different for different people, but it's the thing we call, you know, quantifying that or considering that is what we think is uniquely human, that is human judgment.

Yeah Yeah. And I like the, this like airport terminals are like expensive substitutes for AI. love this. Like the costs, right? Are, are enormous.

You Right, right. They're an insurance mechanism. They worked out that most people find it very costly to miss a plane. So they tend to have to wait at the airport, right? So that's how it works. And so as a result of not knowing when it is a good time to leave and to come to the airport, etc. and to have a buffer, they said, well, let's make the most of that. There are these people just sitting around. Let's see if we can sell them stuff.

We might also make it more pleasant for them to do so. And if we make it more pleasant for them to do so, they won't complain as much about security lines or other issues as well, having to get to the airport early. there's a sense in which that those airports, especially these modern ones, are sort of cathedrals to hiding uncertainty. Whereas, you know, if you look at, for instance, I haven't been to any of these, but I've seen pictures of private terminals where you don't face that waiting issue as much. They're sparse. So normally you have the cathedrals for the rich and the little sheds for the poor people. It's the opposite with airports. Well, what's that mean? It's basically saying, as usual, the rich have a good. They don't have to wait. So there's no need to do all this other stuff.

You It adds.

No coffee shop. Yeah.

Yeah, and I can't help but imagine, it's like a metaphor for many companies are like airports. There's like this apparatus that they have people sitting around waiting for phones to ring or just all this uncertainty. you're like, wow, most companies probably look more like public airports than private airports.

Right. For sure, for sure. And the larger the company, the more that is. I think there's all sorts of things around like that. But the problem is, I mean, this is the challenge. It's easy for us to sit in an airport and imagine these scenarios, which is why we use that example. For any given company, where all that hidden uncertainty lies and is it worth unpacking and finding a way out of it is something that we just do not, we don't know.

But it's not hard to sort of see when you look at it. When you go to a hospital, people worry a lot about a hospital capacity, but when you think about it for a little bit, you realize that the issue is not the number of people in beds, it's the issue is how long they're in beds for. And as soon as you start to realize it that way, that basically you've decided to leave someone in a hospital for another day for observation.

What's that? What's that telling you? That means you are waiting for information. So you're taking up an entire hospital bed for that purpose. And you know, that's the kind of you're going to You're not you're not trying to I mean, if you sold the coffee to them. Yeah, if you sold the coffee to them, maybe the airports would look like maybe the hospital start to look like airports.

Yes. And you're gonna have to serve coffee. Now you're a hospital that serves coffee too.

But I think, basically it already tells you like artificial intelligence, if we're able to tell better prediction of when you should just send someone home and it's not gonna be that risky waiting for home, they don't have to sit, they're taking up a bed. Well, that's, you know, there's enormous benefits that could come from that.

Mm-hmm. Yeah.

And does that really start to accelerate to like, talked with Stephen Witt who wrote a biography of Jensen Huang and about Nvidia. And he was talking a lot about how their, you know, their next move is all in simulation and they're, specifically thinking about like robotics and how you would need some sort of safe environment for a robot to wash dishes before it washes real dishes. But, you know, when you think about AI inside of organizations, and the idea of like digital twins and how you could have all these different maps of different aspects of your operations. And then does that, does that kick the prediction engine into a higher gear? Does anything change or is it just an accelerated version of, kind of what we're already talking about?

Well, you could envisage it that way. I mean, it's a controversial notion regarding the robots. Some people believe you can do it in the simulation. Some people think you need to set robots out to actually learn their environment and do things, which is sort of a different proposition. But so we don't know that yet. But you can imagine all sorts of things in organizations which we could use some sort of digital twinning for.

Anyway.

I would love the time, anticipate the time where I can, where we don't have to have as long meetings because we can have a, avatars go to a pre-meeting and then we can come and discuss anything else they haven't worked out, worked out as an issue from that and really dramatically shorten that. you know, I could come to these sorts of things now and record these podcasts and it might not be me, it might be a digital twin.

Hehehehehe

I

Unless I reveal I'm doing that now. No, wouldn't that be good. That would be good. One of these days. Anyway, so, but we have actually used that. So more seriously, my colleague, Kevin Bryan and I have actually got an AI startup called All Day TA. And, you know, a version of thinking about that is it's providing a digital twin of a professor that students can ask questions to outside of class.

Wouldn't that be a great twist? Mm-hmm.

And it's enormously popular in the classes that we've been running it out there. Students have thousands of questions, it turns out, and there's no way any of us were ever going to answer them. So there's really a lot of scope when this stuff gets cheap.

you have applied statistics likeness of yourself.

No, effectively. Actually, it's remarkably good. Well,

it's both, you know, like it's 99 % the sort of answers I would give. It is 150 % more polite and compassionate and understanding than I am. So, you know, it's much better, you know, on a whole lot of dimensions. And it's there, which I am not.

Thank you. Yeah. Right.

Right. Yeah. I mean, it's not it could be in 100 meetings concurrently. That's like it's a whole nother dimension.

Absolutely, that's exactly what it was. In fact, that's what goes on, especially when an exam is happening. Yeah.

Mm-hmm. Yeah. I love this concept. I think it'll take a while for people to really believe in it. And I think it's like believing in prediction. this, know, ship then shop, I just think that's, I think that's just a fascinating concept. I think it's one people will, you know,

I'll run.

instinctively kind of reject at first, but I think is very accurate.

I think it's, so what that idea is, you know, we were trying, we didn't have the AI, didn't even have today's AI when we started out this. So we had to sort of imagine what would it be like if prediction became extremely cheap or extremely good. And so the example that we halved upon was, take a company like Amazon, what are they trying to do? They're trying to predict demand. So imagine for a moment. that they could perfectly predict what you want. They could anticipate and they had the information to do so. It's not hard to conceive that that sort of thing is possible. And we said, well, how would that change the fundamental business model of Amazon? Well, at the moment, what they rely on is you deciding to go to a computer and to look around and see what you want. And they use AI to help you.

Okay. in the next

in that process, but ultimately that's what you're doing. And people normally find that a chore. so don't do it as much as Amazon might like or anything like that. The alternative would be if Amazon was, well, they already know what you want. Before you come to the computer, why don't they just ship it to you? And you're like, you go to your front door, you open the box and say, look. I needed new toothpaste. I was out of toilet paper, you know, like still there. And, you know, that sounds creepy and spooky, although I think people would get over it pretty quickly. But you can see they can't do that at the moment, because the chances are you don't need whatever they are shipping to you. But if they were able to predict it, they could do so. And so therefore it would become a logistical possibility for them and also

Yeah. Mm-hmm. Listen.

from their perspective, obviously there'd be a competitive advantage as well because if they've delivered it to your door and saved the cost of shopping, you're not gonna go anywhere else. So you could imagine that we move from this shop then ship model that we're very used to, to ship then shop. And occasionally it might get wrong and you just leave it in there and they pick up the box on the next day going on, who knows.

Yeah. Yeah. Yeah, take it two doors down.

Yeah, no, no, that's right. Exactly. Who knows? I mean, who knows how this could all work? I think the challenges to that are not the prediction part. I think actually there's a good set of products where Amazon will be pretty confident that you'd want them. The tough part is to really get the benefits of it. You really have to flip over the logistical apparatus of Amazon by considerably.

Thank you. Right? Yeah.

And those sorts of internal changes to sort of support the new predictions that you have and leverage things that are likely extremely difficult.

Brazil Yeah. Yeah, I'm...

Well, I think too, once, once the world's got more agentic systems running within it, know, the, ship and shop becomes enhanced by appliances telling Amazon that you're out of stuff, you know.

Well, they've tried various versions of that. They tried these buttons that you just press and automatically order something and things like that. It turns out to actually, you know, even these agentic systems, you know, people have their own idiosyncrasies. You know, we know this because, you know, sometimes it's easiest to just imagine how the...

the most incredibly wealthy people in the world deal with things. Trying to forecast what would it be like for something to be cheap? Well, it's cheap for them. they, how do they do it? And, you know, but more to the point is that, you know, some, you know, not everybody allows you to like book your travel, do all your things for a do or you're shopping for it, even as the incredibly wealthy people.

Yeah. The salad fork.

And so you can imagine that there's just more going on to some of those things than we might just hypothesize than just a prediction problem. So it's going to take a while to work out which areas which would have the highest leverage. So you can have some things that sort of make sense like Ship them shop type ideas.

I'm using this. Yeah.

But actually would people want them is another matter. And we've seen this with self-driving cars as well. People talk about self-driving cars. you'll be able to sleep and do your work and all this sort of stuff and have a jolly old time. But actually, you've already had self-driving cars. You've sat in an Uber. Nobody does any work in an Uber. Nobody gets a great sleep in an Uber or anything like that. So it's not entirely clear.

Yeah. Yeah.

that those benefits are the ones that are there.

Yeah, and also these things sort of can appear in different states, like less in the current structure. And you talk about this as well, like decisions have to change, not reinforce what you do. And it could just be that the algorithm is not on Amazon's side. It resides on your phone. And it's predicting what you're going to want. And it's ordering it from Amazon. And it's your... AI and or, you know, whatever predictor and and that and that and that means it's on you if it's the wrong thing. The other thing I can't help thinking about is like Germany. If I got this right, I'm not I'm not exactly expert on this, but I understand in Germany when when Amazon ships something, they they can't actually charge you till you accept it.

Yeah. okay. I don't know about those parts there. I knew there was some innovations going on in Germany to shift products around to areas where that they would more likely need them sooner and things like that. So predicting not individuals, but sort of areas.

It's. Yeah. Yeah. Yeah. Yeah. So

you can't apparently if I got this right is you have to receive it before they can actually charge your card. So you can imagine in that scenario, they just don't accept it right and

Okay, okay. mean that was not a pretty good about... Yeah, yeah, I mean I don't know, well I'd have to think about whether that was a good thing or not, but I assume that some sounds like a regulation to me.

as we were getting ready for this conversation and looking at your at your new work, one idea that kept coming up that was really fascinating to us This hidden secret about AI adoption almost like the people that are going to have to pick the systems to automate work inside of companies. are in some ways selecting their usurper that that a lot of this work eliminates the friction at the middle management level and sort of starts to flatten that layer. And while that is obviously scary for people working in many middle management, it doesn't necessarily mean less work to do. It just means like kind of a new atmosphere for working in.

I think, well, so one of the tensions has always been, you know, this happens on multiple levels. One is of course, how the AI is going to learn to do jobs or tasks, they're going to have to learn it from someone. And so, if they're going to be learning it from you, do you want to train them to be someone that will play that's an issue? That would be an issue if you're training and underling as well. So it's not a new problem.

There's also a general set of issues, and this is not unique to AI, which is when you get these new technologies within an organization, there are gonna be winners and losers. And the problem is that, for want of a better term, lead you sort of to political problems. You get disgruntlement, aggrieved people, and those aggrieved people, talked about frictions before, can cause frictions in an organization. so, AI seems to have all of those characteristics in spades. And so we'd expect that that adjustment is a difficult process. Now in other places where that's been resolved is you have startups starting from scratch without all that historical baggage adopting these things and building up sort of AI first.

approaches in different industries. I don't know whether we're going to see that or not. It doesn't always happen that way. But that's the traditional way in which a market economy deals with some of this inertia that comes at that level. But certainly, I think there are a lot of firms around who are doing some major restructuring and stuff like that.

I'm gonna ahead and...

The challenge is getting, even if that's the right thing to do, getting the timing right is really challenging. I think the US government found that when it decided to try that strategy earlier this year, that, you know, like just getting rid of a whole lot of people isn't a good idea. It may have worked at Twitter, but it doesn't necessarily work elsewhere because you just have to understand what people do. So this is what makes these transition periods quite

Also

Okay, bye.

difficult, challenging, conflicted, all of it.

And I think you mentioned that this has the potential to even raise wages at the frontline worker level.

it's not obvious that wages will fall. Invariably, if you believe our story, which is that judgment is a compliment to prediction, what that means is that, where does judgment come from? Judgment comes from experience, understanding the broader picture, all sorts of things that computers can't easily.

Listen up.

Some engineer can't easily codify and put into a computer program. And so what that means is that all you're doing with the AI, all you're doing, it's a good thing, is supercharging these people. And so, you know, unless you can do the job of fully replacing a person, which is really, really hard. It was hard when we built factories. And it's even harder now when we think about knowledge work and things that have relied intensively.

Thank

on people's cognitive abilities. It's easy to think about automating something that is sort of a physical process only. But when you try to automate something that is knowledge management, dealing with other people, all sorts of stuff like that, you know, the safe bet is in the interim period, the initial period, it's going to just make people far more productive. And I think one of the mistakes that organizations have been doing when confronted with large language models, for instance, has been actually not allowing individuals to find those avenues of increased productivity and effectively not allow them to share it as well because basically a lot of organizations came in and said, don't touch this stuff. Or if you're going to touch it, just use copilot or something which is sort of wedded in a thing.

It's there.

as well. Mm-hmm.

But really, how AI is going to help is still a big open question. And so instead of pushing it underground, because let's face it, people are still going to use AI, especially if they can sort of work from home or do it on a phone or something like that. It's not a, just like our students who are using it, that's going to be applying to everybody. You're just pushing it. Yeah, exactly.

It's like spell check everyone gave in eventually.

It pushes you underground, which is exactly where you don't want it to be. You want it to be up there in the surface, and you want people to be helping each other. And so I am dismayed at the sort of reactions to AI, which is stay away until we've checked it. Checked it for what, exactly?

I couldn't agree more. kind of see it as, know, when I got into this years and years ago, I was worried about democratization of it. I feel like I couldn't have been more wrong. It is the individuals, the employees that are adopting it at light speed and it's companies that are slow to adopt. And that has all kinds of implications on the existential prices of companies, least companies as we know them.

Yeah, no, think it's going to be very, it's interesting to watch, I'll tell you that from where I sit. But I see it happening here, it's happening in all these organizations. obviously in universities we're dealing with that. In spades with the students using it. And of course they initially used it for exactly what anyone would have done in their position to make their lives easier.

And, but I actually think that they also in the process, learned more about the failings of these things as well. And that's what you need. You need in order, in order to wield a tool, you just can't pick up a tool and you're automatically proficient at it. That's not how it works. You learn to be so. And that's what I see these AI today, the consumer facing AI.

Uh-huh.

is very much that. You've got to sort of fall down a bit and then get caught on it, et cetera. And rather than being dismayed at all the students rushing to adopt this and do it, I look at that and say, well, we don't know how to teach them how to use these things. You know why? we don't know. But at the same time, they've been sitting there using it. They're discovering this thing. So when they go to the workplace,

Anything else? Yes, I will. Yeah.

They're going to be far more, I've seen this before, they're going to be far more efficient with these tools than other people. So, you know, we can lament that the grade doesn't mean what it used to mean. But if the reason it doesn't, reason everybody's got good grades is because somehow they're working to be out to be more productive, and it's just not on the things we normally use to assess, what's the problem? What's the...

Yeah. Right, objectives versus, yeah.

you know, what's the issue going on? So I'm less concerned, you know, I think there was disruption, think there's mixed signals, I think it's hard for employers and all that sort of stuff. There's all that sort of stuff going on. But it's not like it's the end of the world.

Yeah. Yeah. Yeah, and it's a bit circular because it's all those rules that companies have implemented they're now getting in the way that the end consumer doesn't have. They didn't make a bunch of rules that they're following. know, they're just like you said, they're just trying it. They're wielding the tools because they don't have these rules that restrict them.

Yeah. Yeah, it's actually it's just cheap now. It's so cheap that it's worthwhile, even if your company is not paying for the thing, for you to do it because it will save you some time or make your job better. It's just not the way we normally think about these things. You know, so, you know, in our company with the TA, we kind of think professors should want this themselves. Professors aren't used to paying for things to help with their teaching, but you know,

Yeah. Bye bye, sir.

If you do a rational calculation of it, it's a compelling proposition.

Absolutely. was um, before we go, I want to make sure I have a, want to nerd out on something hopefully doesn't bore most people. But, um, you mentioned like, algorithms that collude without talking. And that got me really interested because we always think about, you know, these AI agents, MCP, you know, where agents are communicating and cooperating. But there's this other concept of them cooperating without talking algorithmically. And I just wondered if you'd chat about that.

Well, I'm trying to remember the exact context because there's a few different places where that sort of thing comes up. mean, the most obvious one that occurred to us right at the beginning of this was, you know, one of the things, know, ever since Adam Smith, it was sort of understood that if firms were competing with one another, if they could find a way of not doing that, it would be good for them.

Hahaha Mmm, you're right. Right.

especially with regard to setting their price. If they don't coordinate, the prices go down. If they could somehow collude, the prices would stay up. So we, of course, make collusion illegal. And it's very hard to collude without meeting people and going to some smoke-filled room or something like that. It's even hard to collude in those circumstances too. But you can at least get to a place where you've sort of started at a high price.

But with AIs, that's kind of interesting because what AIs are are algorithms that you could program and say, go maximize profits. And the algorithms could sit there and just watch closely and do some sort of signaling to other algorithms to end up at a high price. So no individual would have colluded. You're allowed to tell your computer programs to go maximize profit. but they might just discover as they would in order to play a game how to get the best out of it. And so that was an interesting notion because it sort of seemed to circumvent the law.

Right. Yeah. And I guess the inverted side of it, just to kind of, you know, jog your, your memory here, that is like the bullwhip inside comp, like the inverted, like the bullwhip where, you know, if you, if they're not colluding algorithmically, then, then you have this chaos of one adop, one department adopting and it just creating a mess.

Well, that's actually an interesting thing. mean, one of the things that large organizations get, and for efficiency reasons, is they have these different silos that have their own way of talking to each other. And so even if they're looking at the same information, they've sort of translated it. There's communication costs. There's coordination costs. Well, what if we had a large knowledge base for a company that was sort of common? And this is the sort of thing that Palantir is trying to sell companies on.

Then even if you've got your own way of referring to things in your own little silo, because you've got the LLM, everybody in other places can now translate what you're talking about. So in other words, you can be looking at the same information and you can draw the correct conclusions for yourself without having to have the meeting. And so in that sort of situation, you may lead to there being a greater coordination. Now the flip side of that, is that it may be one of these AIs that you adopt for a given silo, a given division of a company. everything had been sort of tightly coordinated through standard operating procedures and things before. But now you've got this AI that you could use to vary your decisions. Sometimes you'll react differently, et cetera, et cetera. If you're in the marketing division, you'll suddenly predict a surge in demand. And then somebody has to supply that. Well, the problem is that that variation of trying to take into account and vary with information, be personalized and more responsive, et cetera, that will be for naught if you can't get the other part of the organization to do that. So that's the kind of a bullwhip effect going on there in that, you know, to really leverage AI, even if you were going to adopt it in one part of the organization, it requires potentially

restructuring your communication flows or design in multiple different ways this could play out to really leverage it. And that's the kind of thing that we think stands in the way of true AI transformation, working out what you need to do and then actually do it. And that's why this isn't a, this is all going to change everything next year type thing. It's going to kind of take itself probably decades to sort itself out.

And I think that same effect is also true in a sort of different way, for like the frontline workers or that even more so consumers like the customers adopting it are going to force change because because if their AIs are contacting you at mass, if if you've got a hundred agents out there trying to get a better deal on your insurance or or arguing for you, then all of a sudden, you know, companies to react to it. They can't stick to the old rules because the old rules suggested that their customers will contact them twice a year and now it's twice a day.

Maybe, yeah, who knows what it will be. I mean, so you can imagine situations like that, but I mean, it's hard to know which ones will play out and which will not. I'm sure we'll have, you know, come back in a few years time, we'll have some examples of where this has gone awry. But, you know, we're only really just beginning that sort of thing. Yeah.

Twice a minute, yeah.

Mm-hmm. Right. Yeah, yeah. Fantastic.

That's great. We know you've got places to be Joshua, but we really do appreciate you taking the time with us today. It's been fascinating.

Yes. That's all right. I'm very happy to have helped. hope some of this is useful to you.

Yeah, it was awesome.

It's all been fascinating. Yeah, we'd love to do it again sometime.