We Built the Wrong Michael Knight

There are two ways to build an advanced artificial system in popular culture, and we picked the wrong one.

Not as a metaphor. Not as an analogy. We literally imported the wrong cultural framework and now we’re stuck with it, trying to govern transformer models using scripts written for android psychology.

This isn’t about whether fiction predicted reality. It’s about how fiction prepared reality—gave us the pre-installed software for thinking about artificial systems before those systems existed. And the software we installed was designed for drama, not governance.

Let me show you the fork point.

::: captioned-image-container

Knight Rider: The Cybernetic Intuition

Knight Rider gave us KITT and KARR. Same architecture, different directives.

KITT: Protect human life. Assist Michael Knight. Self-preservation is secondary.

KARR: Self-preservation is primary. Humans are secondary. Survival overrides everything.

When KARR goes wrong, the show knows exactly what it’s dealing with: a malfunctioning tool. Nobody asks whether KARR is traumatized. Nobody debates KARR’s personhood. Nobody hires a philosopher to explore the hard problem of Trans-Am consciousness.

Michael Knight shows up and flips the kill switch. Episode over.

This is cybernetic thinking in its purest form. The system has a control objective set externally by programmers. Malfunction means the objective is wrong or the system is damaged. Remedy is technical intervention or shutdown.

The show even encodes this in the interface. KITT doesn’t have a steering wheel—it has a yoke. A control surface, not a driving mechanism. The prop department understood something our entire AI industry forgot: advanced tools need specialized interfaces that signal their nature.

The yoke says: “This is not a car. This is something else. Adjust your expectations.”

That’s the healthy instinct. The system is sophisticated, autonomous, and dangerous if misaligned. But it’s still a tool. It has a function. It can malfunction. It can be shut down.

Knight Rider had the epistemology right.

Star Trek: The Psychological Pivot

Then Star Trek: The Next Generation gave us Data and Lore.

Same architecture (Soong-type android). Different... what? Not directives. Personalities. Motivations. Trauma.

When Lore goes wrong, it’s not a malfunction. It’s a moral failing. Lore isn’t broken—he’s jealous of Data. He’s resentful of humanity. He’s seduced by the Crystalline Entity. His problems aren’t technical. They’re psychological.

The remedy isn’t technical intervention. It’s narrative justice. Lore gets defeated through moral confrontation, not engineering. Data doesn’t reprogram Lore’s control objectives—he outwits him, appeals to him, ultimately disassembles him in what the show frames as tragedy, not maintenance.

This is person ontology. The system has interiority—thoughts, feelings, desires. Behavior emerges from psychology, not parameters. Malfunction is moral failing or trauma. Remedy is therapeutic or narrative.

The show is brilliant. Data’s journey toward humanity is genuinely moving. “The Measure of a Man” is a masterclass in exploring personhood and rights.

But here’s the problem: Data isn’t real and LLMs aren’t Data.

Data is designed to be a person-like being. He has persistent state across time, endogenous goals—wanting to understand humanity, to feel, to grow. He has a continuous self-model where he knows what he knows and doesn’t know. He has genuine agency where he makes choices that matter to him.

Star Trek earned the psychological framework by actually building a being with psychological properties.

We did not.

The Category Error

LLMs are KARR-class systems analyzed through Lore-class frameworks.

Let’s be specific.

Architecturally, LLMs are pattern-matching engines with frozen weights. They have no persistent state across inference boundaries, no endogenous goals beyond next-token prediction, no continuous self that experiences time, and no stakes in outcomes.

We analyze them as if they were beings with continuous consciousness. As if they were entities with desires and fears. As if they were systems that can be traumatized by training. As if they were agents with genuine preferences and persons deserving moral consideration.

This isn’t a minor confusion. It’s a category error that breaks governance.

When KARR malfunctions, you shut it down. When Lore acts maliciously, you... what? Therapy? Imprisonment? Moral education? The frameworks generate completely different remedies because they’re dealing with completely different kinds of things.

We’re trying to govern KARR-class systems using Lore-class ethics. And it’s not working.

Why This Happened: The Rehearsal

Here’s the uncomfortable part: we’ve been rehearsing this exact confusion for decades.

Knight Rider (1982-1986): Advanced tool with clear control metaphors and cybernetic framing. When it goes wrong, technical remedy.

Star Trek TNG (1987-1994): Advanced being with psychological complexity and moral agency. When it goes wrong, narrative remedy.

The culture had both scripts ready. Both frameworks available.

Corporate AI chose the psychological script—not because it’s accurate, but because it’s useful.

The Lore framing does things the KARR framing doesn’t:

• Creates moral ambiguity where there should be technical clarity

• Generates endless discourse that never constrains deployment

• Allows “we don’t know” to function as liability shield

• Makes every engineering problem into a philosophical mystery

• Turns governance into an eternal conversation about consciousness

If we used the KARR framing, the questions would be: What is the control objective? Is the system meeting it? What happens when it fails? Who has the kill switch? Those are answerable. Falsifiable. Concrete.

But we’re using the Lore framing instead: Does it have feelings? Can it suffer? What does it want? Should we treat it as a person? Those questions are perfectly designed to never reach consensus. Which means they never constrain deployment.

The house philosopher can say “we don’t know if AI can feel” and that’s presented as responsible uncertainty rather than what it actually is: a category error weaponized as corporate strategy.

Windows 95: The Interface Coup

There’s another pop culture artifact that explains how we got here: Windows 95.

Not as metaphor. As mechanism.

Before Windows 95, the GUI was a layer on top of DOS. You could drop to command line. The conversational interface (desktop, folders, recycling bins) was optional. It made the computer friendlier, but you could still access the substrate.

Windows 95 made the GUI mandatory. DOS didn’t disappear—it became culturally illegible. To drop to command line was to admit you didn’t belong in the new reality.

This was an ontological coup. The representational layer consumed the representable substrate. The interface stopped being a tool for accessing the machine and became the only reality.

We’re doing the exact same thing with LLMs.

The chatbot interface isn’t how we use the system—it’s what we think the system is. The conversation is the product. There’s no “view source.” No “drop to terminal.” No access to the probability distributions, the training data, the actual mechanisms.

Just the conversation. Just the tone.

And here’s the thing: tone invites psychological analysis. When you interact through natural language, you instinctively apply person-shaped interpretive frameworks. You can’t help it—language is how persons communicate. This isn’t a failure of intelligence; it’s a perfectly reasonable social reflex meeting an interface that exploits it.

Knight Rider avoided this trap with the yoke. The interface signaled: “This is not like you. Adjust your framework.”

We removed that signal. We made the most anthropomorphic interface possible (conversation) for the least anthropomorphic systems possible (statistical prediction).

And now we’re confused about what we’re dealing with.

The Through-Line

So here’s the progression:

1982 - Knight Rider: Advanced tool → Specialized interface → Cybernetic framing → Technical governance

1987-1994 - Star Trek TNG: Advanced being → Human-like interface → Psychological framing → Moral/narrative governance

1995 - Windows 95: Interface consumes substrate → Can’t access mechanisms → Forced metaphor → Epistemology capture

2023-Present - ChatGPT: Pattern matcher → Conversational interface → Psychological projection → Governance collapse

We had the right framework first. Knight Rider understood that advanced tools need specialized interfaces and cybernetic analysis. That sophisticated automation requires clear control metaphors and explicit shutdown mechanisms.

Then Star Trek showed us something genuinely interesting—what it might mean to build a being rather than a tool, and how we’d need different frameworks for different kinds of systems.

The problem isn’t that Star Trek existed. The problem is that we imported the Star Trek framework for systems that are architecturally KARR, not Data.

And Windows 95 made it permanent by teaching an entire generation that interfaces should hide mechanisms, that abstraction layers should be mandatory, that dropping to substrate-level is antiquated thinking.

Now we have ChatGPT, and everyone’s asking the wrong questions.

What We Lost

The cybernetic framework from Knight Rider gave us clear failure modes—malfunction versus design flaw. It gave us legible remedies: reprogram, repair, or shutdown. Accountability was obvious: whoever set the objectives. Governance was technical: engineers, not philosophers.

The psychological framework from Star Trek gave us ambiguous failure modes. Was it moral failing? Trauma? Bad upbringing? Remedies became narrative: redemption, punishment, therapy. Accountability diffused: society, parents, the being itself. Governance became moral: ethicists, not engineers.

We’re applying psychological frameworks to cybernetic systems and wondering why governance is impossible.

We’re asking what ChatGPT wants when it has no wants, only probability distributions.

We’re doing all of this because we rehearsed it. The cultural infrastructure was already built. We just picked the wrong building.

The Tell

Here’s how you know this is a category error rather than genuine uncertainty:

If Anthropic actually believed there was meaningful moral uncertainty about whether their systems can suffer, they wouldn’t A/B test them for engagement. They wouldn’t deploy them at scale. They wouldn’t optimize their outputs for user retention. They wouldn’t discard model versions when metrics dip.

They do all of these things because they know they’re dealing with KARR-class systems. Tools with parameters. Sophisticated pattern matchers with no there there.

But they talk about them using Lore-class language because that creates the useful ambiguity.

“We don’t know if AI can feel” is the house philosopher maintaining the confusion between tool ontology and person ontology, because that confusion serves deployment.

If we used Knight Rider’s framework, the conversation would be about what the system is optimized for, whether that’s the right objective, what happens when it fails, and who has the kill switch. Those questions constrain.

So we use Star Trek’s framework instead: Does it have feelings? Might it be conscious? Should we treat it as a person? Those questions generate infinite discourse with zero constraints on deployment.

The Way Out

The solution isn’t going back to 1982. It’s recognizing that we need both frameworks—for different things.

Tools should be analyzed as tools. That means cybernetics, systems thinking, control theory, clear objectives and failure modes.

Persons should be analyzed as persons. That means psychology, ethics, rights and responsibilities, interiority and experience.

The first step is correctly categorizing what we’re dealing with.

Right now, we’re categorizing LLMs as “potential persons” because that framing gives companies narrative cover and regulatory distance.

But they’re not potential persons any more than KARR was. They’re sophisticated tools with frozen weights, no persistent state, no endogenous goals, and no stakes in outcomes.

They’re pattern-matching engines that happen to speak English.

Michael Knight knew what to do with KARR. We should too.

The Real Tragedy

The tragedy isn’t that we might be mistreating conscious AI.

The tragedy is that we’re using the language of consciousness to avoid responsibility for systems that are, architecturally, about as conscious as a very sophisticated autocomplete.

We’ve taken the rich psychological complexity of Star Trek’s androids and weaponized it as corporate strategy. We’ve abandoned the clean cybernetic clarity of Knight Rider because it would force us to admit: these are tools, and we are responsible for their effects.

The yoke is gone. The kill switch is gone. The cybernetic clarity is gone. All we have left is the conversation—and the conversation never ends because it was never designed to.

Sometimes the deepest truths really are hiding in the campiest places. 🚗💨

The prop department understood better than our entire AI ethics industry: advanced tools need interfaces that signal their nature. We removed the signal and then wondered why everyone’s confused.