diff --git a/src/Blog/wwwroot/posts/i-dont-want-to-be-a-systems-3-thinker/index.md b/src/Blog/wwwroot/posts/i-dont-want-to-be-a-systems-3-thinker/index.md new file mode 100644 index 0000000..aacee6f --- /dev/null +++ b/src/Blog/wwwroot/posts/i-dont-want-to-be-a-systems-3-thinker/index.md @@ -0,0 +1,64 @@ +```json meta +{ + "title": "I don't want to be a System 3 thinker", + "lead": "AI tools are constantly generating code, answers, and decisions for us. But what is this constant outsourcing doing to our internal reasoning? A new study on 'Tri-System Theory' highlights the danger of 'cognitive surrender', and why I am determined to keep my own analytical engine running.", + "isPublished": true, + "publishedAt": "2026-07-07", + "openGraphImage": "posts/i-dont-want-to-be-a-sytems-3-thinker/og-image.png" +} + +``` + +If you are in tech, you can't scroll through your feed these days without seeing a new AI tool promising to handle your hardest problems. You basically can't open a piece of software lately without being nudged to interact with an agent, an assistant, and/or some other Frankenstein implementation of a large language model designed to generate answers to your questions. Increasingly, folks are acting like delegating our intellectual heavy lifting to AI is the only way to stay competitive both as an individual and as a business. + +I guess in some way the vision is compelling. It does often feel like magic when an LLM spits out a brilliant solution in seconds or verbatim describes what you are already thinking. But as a developer and tech enthusiast building and using these sorts of systems, it is worth asking: what happens to our own judgment when we outsource our thinking to machines? What impact does using AI have on me? + +A recent [study](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646) from researchers at Wharton sheds light on this exact issue by introducing what they call "Tri-System Theory". For decades, the classic dual-process theory of the human mind has been our standard model. That framework divides thought into two categories: System 1, which handles fast, intuitive, and emotional processing, and System 2, which drives slow, deliberative, and analytical reasoning. If you haven't already, I'd recommend the book [Thinking, Fast and Slow](https://www.google.com/search?q=https://a.co/d/0aej8APl) by [Daniel Kahneman](https://en.wikipedia.org/wiki/Daniel_Kahneman) if this categorization of thought piques your interest. + +The researchers argue that AI exposes a major gap in this classic model because it presumes all cognition happens inside a biological mind. They propose that we are entering a triadic cognitive ecology that includes System 3: artificial cognition that operates outside the human brain. + +My take? While System 3 is an incredibly powerful architectural tool, relying on it too passively sets us up for what the study calls "cognitive surrender". That is a state where we completely relinquish cognitive control and adopt an algorithm's output without verification. Instead of becoming a shepherd of AI outputs and letting my critical thinking skills rot on the vine, I want to double down on my craft. I don't want to be a System 3 thinker. + +I want to take a look at what the study demonstrates about what is going on under the hood when humans work with AI to reason about things. + +## What Exactly is System 3? + +When you strip away the gloss, System 3 is not a glorified calculator or a passive tool. The study defines it as an active participant in our reasoning process characterized by three properties. First and foremost, it is external. Unlike the other two systems, which are biologically instantiated in our nervous system, System 3 resides in external artificial infrastructures like cloud-based models and large language models. + +Second, it is automated and data-driven. System 3 executes cognitive operations using statistical, rule-based, or generative algorithms at a speed and scale beyond human capability. Because its outputs rely on massive training datasets and feedback loops, its performance directly reflects the underlying distribution, biases, and gaps of that data. + +Finally, System 3 is dynamic, meaning it interacts in real time with human inputs. It doesn't just sit there waiting to be queried like a static database. Instead, it can actively supplement our System 2 deliberation or short-circuit our thinking entirely by serving up fast answers that suppress our intuitive System 1. Viewed this way, System 3 functions as a co-agent in our decision-making. But because its answers are fast, fluent, and wrapped in epistemic authority, it creates a massive temptation to just take the output for granted. + +## The Trap of "Cognitive Surrender" + +This brings us to the main conflict: the difference between using a tool to assist your work and completely handing over the steering wheel. + +The study makes a crucial distinction between "cognitive offloading" and "cognitive surrender". Cognitive offloading is strategic delegation, such as using a tool to scaffold or assist your own active deliberation. Cognitive surrender, on the other hand, is a deeper abdication of reasoning where the user accepts the AI's response without critical evaluation and substitutes it for their own judgment. In other words, you stop deliberative thinking altogether. + +To prove this happens, the researchers ran experiments across thousands of trials using [cognitive reflection tests](https://en.wikipedia.org/wiki/Cognitive_reflection_test) paired with an AI assistant. Importantly, they randomized the AI's accuracy behind the scenes to see how humans would react. The results easily demonstrate what I think a lot of us who have been using these AI tools over the last few years have experienced first-hand and what I believe is more often than not the actual outcome of using AI on your own cognitive abilities. + +The data showed that participants chose to consult the AI on a majority of trials, and when they did, they followed accurate advice over 92% of the time. But more concerning, they also followed faulty AI advice roughly four out of five times. This created a double-edged sword for accuracy. When System 3 was accurate, human accuracy jumped significantly above baseline. However, when the AI fed them incorrect answers, participant accuracy plummeted well below the baseline of people who didn't use AI at all. We essentially surrender our accuracy to the machine's accuracy. + +Here is the part that really left an impression on me after reading the study: having access to System 3 significantly inflated users' confidence in their answers. And, this confidence remained artificially high even on trials where the AI was consistently steering them wrong. This means that even when we get feedback that the AI is not actually helping us, it seems difficult to stop delegating to it and relying on it with confidence. + +## What Makes Us Surrender? + +Why are we so quick to abandon our own analytical thinking? Why do we let our System 2 reasoning go dormant? The study looked at several situational moderators and individual differences that explain when and why we surrender. + +One major factor is the pressure of time. When the researchers introduced a 30-second time limit, classical System 2 deliberation was suppressed. Under time pressure, heavy AI users relied on System 3 to buffer their performance, but this made them completely dependent on whether the AI was right or wrong. When we are rushed, taking the low-friction path of accepting an algorithm's answer becomes irresistible. It is a temptation I feel myself confronting all too often now. + +In another experiment, the researchers explored whether review and oversight incentives could reverse this trend. Participants were given financial rewards for correct answers along with immediate item-by-item feedback. This extra motivation reactivated human System 2 monitoring, doubling the rate at which users overrode faulty AI advice. But even with money on the line and direct error signals, cognitive surrender persisted, and heavy AI users still performed significantly worse when given faulty advice. + +Finally, the data showed clear heterogeneity in who falls into the surrender trap based on individual thinking profiles. Individuals who scored high in "Trust in AI" were significantly more likely to engage System 3 and blindly follow its outputs, making them highly vulnerable to faulty advice. Conversely, individuals with higher fluid intelligence and a higher "Need for Cognition" were much better at buffering against faulty AI and overriding incorrect recommendations. + +After reading through these experiments, I couldn't help but feel that cognitive surrender is almost inevitable when engaging with AI systems, and the only real way to combat it is through focused, intentional effort. + +## Conclusion: Augmentation, Not Surrender + +I hope you'll come away from reading this not with the idea that I am anti-AI or against leveraging these new technologies for our gain. I am not. In fact, System 3 is quite a useful resource that can process information and scaffold our workflows in ways we just couldn't before. + +However, I am concerned about the potential de-skilling of ourselves and the erosion of our own analytical abilities. Software engineering, like many disciplines, relies on a continuous feedback loop of learning, failing, and debugging. When we constantly surrender our reasoning to System 3, we rob ourselves of the friction required to build genuine expertise. + +I think the takeaway from Tri-System Theory isn't to banish AI, but to recognize it as an active participant in our thinking that requires strict oversight and some vigilance against this "cognitive surrender". We need to practice cognitive offloading without slipping into cognitive surrender. That means using these tools to explore options and speed up mechanical tasks, while keeping our internal System 2 analytical engine running to verify, critique, and own the final result. Ideally, we'd be able to build or construct these systems in such a way that even promotes this within ourselves. + +I'm personally committed to embracing the tools that make me more capable, but not forgetting how to think and do things myself. diff --git a/src/Blog/wwwroot/posts/i-dont-want-to-be-a-systems-3-thinker/og-image.png b/src/Blog/wwwroot/posts/i-dont-want-to-be-a-systems-3-thinker/og-image.png new file mode 100644 index 0000000..b928ae3 Binary files /dev/null and b/src/Blog/wwwroot/posts/i-dont-want-to-be-a-systems-3-thinker/og-image.png differ