How Do You Like Them Apples?

I remember stepping through the theater doors out into the frigid January air and turning to my friend who sighed, “I wish every time I saw that movie was the first time.” It was 1998 and we’d just finished our second viewing of Good Will Hunting in as many weeks. So many scenes punched directly in the heart with a hurt good enough that I, too, wanted the spot to remain forever tender. Having recently watched it again, I found something else had left a mark: Will’s gift for regurgitating anything he’s read echoes through this era of agentic AI.

During his first visit with a new therapist, Dr. Sean Maguire (played with furry gravitas by Robin Williams), Will uses his intellect to eviscerate him. For their second session, Sean takes the boy genius to a park bench and calls his bluff.

Scene cited  ·  Good Will Hunting  ·  Open on YouTube ↗

“Michelangelo, you know a lot about him,” Sean says. “But you can’t tell me what it smells like in the Sistine Chapel. You never actually stood there, looked up at that beautiful ceiling.”

Sean runs through several more instances where having absorbed copious amounts of information doesn’t grant you experience or wisdom. Though he can quote everything he’s read about nearly any topic, Will doesn’t know the real shit about love, emotional maturity, intimacy, or personal sacrifice. “No one could possibly understand the depths of you,” Sean says, later adding, “but I can’t learn anything from you I can’t read in some fucking book.”

With LLMs chattering away all around us, we’re now surrounded by Good Will Huntings (GWHs) galore—regurgitating summaries of all the human knowledge that’s been collected online over the past 30 or so years. GWHs packed with so much training data that even their creators marvel at their depths, unable to say for sure what’s happening deep inside. Still, you can’t learn anything from them that can’t be read in some fucking book, or on some fucking webpage.

To truly go out and learn about the world, Good Will Hunting had to first revisit childhood trauma with the help of an empathetic therapist. True to form, giving GWHs a foundational education is one of the most difficult challenges organizations are now facing. Despite their limited understanding of how your actual business works, GWHs are extraordinarily confident. They will make up stories, like Will Hunting telling his new girlfriend that he has 12 older brothers (Marky, Ricky, Danny, Terry, Mikey, Davey, Timmy, Tommy, Joey, Robby, Johnny, and Brian). She believes him and we tend to believe GWHs because they seem confident and appear interested in our queries.

This spells danger for organizations trying to integrate the technology, and for a near future where our own GWHs are out negotiating with everyone else’s. If any instance of AI might be trying to manipulate you, the most sensible way through is an agent you’ve trained and trust—and what makes it trustworthy isn’t its cleverness. It’s the knowledge underneath it.

Any AI agent that can actually do stuff reliably—give you verified answers, pull the levers on your existing software, talk across all your channels—isn’t one clever mind. It’s a team. A network of agents standing on a shared, trustworthy set of facts about your world, whether that world is a small business, an enterprise, or your own life. Take away the shared source of truth and you don’t have an assistant. You have a very confident stranger with twelve imaginary brothers.

A GWH can tell you a great many things about Michelangelo, but it doesn’t know that you are running a startup devoted to recreating and completing the Tomb of Julius II using 3D-printed marble, seeking funding from his living ancestors. It lacks the context that—in humans—comes from experiencing the physical world and interacting with its inhabitants. Humans working with GWHs need to give them that context, specifically as it relates to whatever it is they are trying to get the thing to do for them.

My Age of Invisible Machines co-author Robb Wilson (CEO and co-founder of OneReach) likes to remind people that you don’t send a new employee off to work without giving them foundational information about your company and how it operates. “You’re not like, ‘Hi, this is the first day of brain surgery and here’s your scalpel’,” Robb says. “‘Start carving away and next month we’ll take a class on brain surgery.’ Knowledge is not the documents that you have in your document repository. It’s not the data you have in your database. Knowledge is the abstraction layer on top of that. And it has to be canonically correct.”

The bad news is that creating a canonically correct knowledge system is complex, tedious, and boring to investors. The good news? No organization taking AI seriously can avoid doing it.

When we talked to Jeff McMillan, Former Head of Firmwide AI at Morgan Stanley, he pointed out that building a source-of-truth for their advisors was far more complex than just dumping data and feeding it to an LLM. Working closely with OpenAI before the release of ChatGPT, his team had to assign time-to-live to documents and attach a human owner to every piece of information who could provide oversight when answers drifted.

Related episode  ·  Jeff McMillan  ·  S7E12  ·  Open on YouTube ↗

So how to begin? McMillan would say: not at the top. Everyone wants to jump to models and apps—the visible layer—but “the heavy duty lifting and the non-sexy stuff is [the] bottom two layers.” And even when you’ve done that work, you may still be missing what matters most. Ask a senior person how they actually run a process, he says, and “most high end sophisticated businesses have a very difficult time articulating that at a degree of specificity you would [expect] from a consulting firm … It is very hard for you to apply artificial intelligence in a precise, high quality output way unless you know those core processes.”

The GWH in your stack can quote Michelangelo. It still can’t tell you what it smells like in your Sistine Chapel. How do you like them apples?

Josh Tyson is a contributing editor at UX Magazine and host of Invisible Machines with Robb Wilson. This member essay is part of the Knowing Before Doing ideation cluster with Sudhir Hasbe (S7E13).