9.8 KiB
Script: I Don't Want to Be a System 3 Thinker
The Hook
[VISUAL: Face Camera]
"If you are in tech, you can't scroll through your feed these days without running into a new AI tool that promises to handle your hardest problems. Agents, assistants, co-pilots — they are everywhere. And the message is always the same: just let the AI think for you. Delegate your intellectual heavy lifting and stay competitive."
"I get why that pitch is appealing. It does feel like magic when an LLM spits out a solution in seconds, or perfectly articulates what you were already thinking. But here is the question I keep coming back to: what happens to our own judgment when we outsource our thinking to machines? What is this doing to me?"
"A recent study from researchers at Wharton tries to answer exactly that, and it introduced a concept I can't stop thinking about: Tri-System Theory."
[VISUAL: Switch to Screen Share — SLIDE 1: Title]
"Let's get into it."
The Classic Model
[VISUAL: Advance to SLIDE 2: The Classic Model of Thinking]
"Before we can understand what Tri-System Theory is, we need to understand what came before it."
"For decades, the dominant model of human cognition has been Dual-Process Theory. You may know it from Daniel Kahneman's book, 'Thinking, Fast and Slow.' The framework divides thought into two systems."
"System 1 is fast, intuitive, and emotional. It's the thinking that happens automatically — pattern recognition, gut reactions, snap judgments."
"System 2 is the opposite. Slow, deliberate, and analytical. It's the effortful reasoning you engage when you solve a math problem, weigh a complex decision, or evaluate an argument."
"Together, these two systems were thought to explain essentially all human cognition. Notice the key assumption baked into this model: all of it happens inside a biological mind."
Introducing System 3
[VISUAL: Advance to SLIDE 3: A New Participant Enters]
"The Wharton researchers argue that AI exposes a gap in that classic model. The gap is this: it assumes all cognition is biological."
"They propose we are now operating in what they call a 'triadic cognitive ecology.' That means we've added a third system to our thinking environment."
[CLICK]
"System 3 is artificial cognition that lives entirely outside the human brain — in cloud-based or local large language models. It doesn't sit in your nervous system. It sits on a server farm somewhere."
"And crucially, because its outputs are fast, fluent, and wrapped in a kind of epistemic authority, it creates a real gravitational pull on how we think."
System 3's Properties
[VISUAL: Advance to SLIDE 4: What Makes System 3 Different]
"The study defines System 3 by three properties."
[CLICK]
"First, it is External. This might seem obvious, but it has important implications. The other two systems are biologically constrained. System 3 has no such limits. It scales infinitely and is accessible to anyone."
[CLICK]
"Second, it is Automated and Data-Driven. System 3 runs statistical, rule-based, and generative algorithms at speeds and scales we simply cannot match. But — and this is important — its performance directly reflects the biases and gaps in its training data. It is only as good as what it was trained on."
[CLICK]
"Third, it is Dynamic. This is the property that makes it most dangerous to our cognition. System 3 doesn't just passively wait to be queried like a static database. It actively interacts with your inputs in real time. It can supplement your System 2 deliberation, or it can short-circuit your thinking entirely by serving up a fast, confident-sounding answer before your System 2 ever engages."
"Viewed this way, System 3 isn't just a tool. It is a co-agent in your decision-making process."
Offloading vs. Surrender
[VISUAL: Advance to SLIDE 5: Two Very Different Things]
"This brings us to what I think is the most important distinction in the study."
"The researchers draw a careful line between two behaviors: Cognitive Offloading and Cognitive Surrender."
"Cognitive Offloading is strategic. You use a tool to scaffold and assist your own active deliberation. You are still thinking. You are still in the driver's seat. The AI is a resource, not a replacement."
"Cognitive Surrender is something else entirely. It is a deep abdication of reasoning. You accept the AI's response without critical evaluation and substitute it entirely for your own judgment. Your System 2 goes dormant. You stop thinking."
"Here is the thing: both behaviors can look pretty much identical from the outside. In both cases, you consulted an AI and got an answer. The difference is entirely internal — whether your own analytical engine kept running."
The Study Data
[VISUAL: Advance to SLIDE 6: What the Data Shows]
"The researchers didn't just theorize about this. They ran controlled experiments across thousands of trials using cognitive reflection tests paired with an AI assistant. And here is where it gets uncomfortable."
[CLICK]
"When the AI was accurate, participants followed its advice over 92% of the time. Their accuracy jumped well above baseline. Great, that's what we'd want."
[CLICK]
"But when the AI was wrong — and remember, they randomized the AI's accuracy behind the scenes — participants still followed the faulty advice roughly four out of five times."
[CLICK]
"And the consequence of that? Accuracy plummeted below the baseline of people who didn't use AI at all. Not just 'slightly worse.' Worse than if they had never consulted the AI in the first place."
[CLICK]
"We essentially hand over our accuracy to the machine's accuracy. When it's right, we're great. When it's wrong, we're worse than we'd have been on our own."
The Confidence Trap
[VISUAL: Advance to SLIDE 7: The Confidence Trap]
"Here is the part that stayed with me the most after reading this study."
"Having access to System 3 significantly inflated participants' confidence in their answers. That part makes sense — you have a resource, you feel more sure."
"But here's the alarming part: that confidence remained artificially high even on trials where the AI was consistently steering them wrong."
[CLICK]
"Think about what that means. Even when the feedback was screaming at them that the AI was unreliable — actual wrong answers on actual problems — they continued delegating to it with high confidence."
"We are not just passively accepting bad outputs. We seem to lose the ability to recalibrate our trust in the machine, even in the face of clear evidence. That is a form of cognitive surrender that goes deeper than laziness. It is a structural vulnerability."
What Makes Us Surrender
[VISUAL: Advance to SLIDE 8: What Makes Us Surrender]
"So why does this happen? The researchers identified several factors."
[CLICK]
"The first is Time Pressure. In experiments where participants had only 30 seconds to answer, classical System 2 deliberation was suppressed. Under pressure, the lowest-friction path wins every time. Taking the AI's answer is just easier. I feel this pull constantly in my own work."
[CLICK]
"The second is individual Trust in AI. Participants who scored high on a 'Trust in AI' scale were significantly more likely to follow AI outputs blindly and significantly more vulnerable when those outputs were wrong. The more you trust it, the harder it is to override it."
"Conversely, participants with higher fluid intelligence and a higher 'Need for Cognition' — people who actually enjoy thinking — were much better at buffering against bad advice."
[CLICK]
"The third finding is perhaps the most sobering: even incentives aren't enough. In one experiment, participants were given financial rewards for correct answers and real-time feedback on each item. That motivation did help — it doubled the rate at which people overrode faulty AI advice. But cognitive surrender still persisted. Even with money on the line and direct error signals, people kept deferring to the machine."
"The conclusion I walked away with: this is almost inevitable when engaging with AI systems. Resisting it requires focused, intentional effort."
Conclusion
[VISUAL: Advance to SLIDE 9: The Move]
"So what do we actually do with this? I'm not anti-AI. I build and use these systems every day, and I think System 3 is a genuinely powerful architectural resource. But I am deeply concerned about what passive reliance does to our own capabilities."
"Software engineering — like a lot of disciplines — relies on a feedback loop. You build, you fail, you debug, you learn. That friction is not an obstacle to expertise; it is the mechanism of expertise. When we constantly surrender our reasoning to System 3, we rob ourselves of that friction."
[CLICK]
"So step one: use System 3 as a tool. Explore options, accelerate mechanical tasks, scaffold your workflow. That's legitimate cognitive offloading. What it shouldn't be is the final authority on your decisions."
[CLICK]
"Step two: keep your System 2 running. Don't let your analytical engine idle. Even when you're using AI, make sure you are still thinking — questioning the output, stress-testing the logic, forming your own opinion before you look at the machine's answer."
[CLICK]
"Step three: verify, critique, and own the result. You are the shepherd of AI outputs. The last judgment belongs to you. Not because it is slower or more painful, but because it is the only way to stay genuinely capable — and to know when the machine is lying to you."
[VISUAL: Face Camera]
"The takeaway from Tri-System Theory isn't to banish AI. It's to recognize it as an active participant in your thinking that demands strict oversight. Use the tools that make you more capable. Just don't forget how to think."
"I'm personally committed to that. I don't want to be a System 3 thinker."
"Thanks for watching."