The Productive Brain
The human brain is a powerful resource. It is the source of our creativity. We use it to learn new things and record our life experience. Thus, it is the cradle of our skills. We have all had the fundamental human experience of growing our skills through learning and practice. We have also had the experience of atrophy when we have declined to put in that effort.
AI marketing seems to consistently tout productivity gains, yet companies repeatedly focus on how developers feel, rather than how productive they really are. In reality, we allow our own creativity to atrophy the moment we let the AI do a task for us. Who (or what) is putting in that effort?
How do we know what impact AI really has on us? Recent objective studies have been conducted to measure just that! It is worth noting that these studies are under peer review at the time of writing this blog article. However, at Amplify, we feel this is important enough to discuss right now.
What can we learn from these studies?
Software Development Productivity
Joel Becker, Nate Rush, Elizabeth Barnes, and David Rein at METR conducted a randomized controlled trial measuring software developer productivity. In this trial, they employed 16 experienced open source developers to work on their own repositories using real issues that provided real world value to the repositories. These issues were randomly assigned as AI-enabled or AI-free and given to the developers to work on while recording their screens.
Before the exercise began, experts in the field and the participants all estimated that AI would improve productivity. Once finished, the participants reflected on their experience and gave another estimate on the impact of AI during the study. All provided estimations (pre- and post-experiment) claimed a productivity boost.
What do you think METR found?
“When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts. This gap between perception and reality is striking: developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%.”
These results are easier to see in their own chart on their website:
The contrast between how we feel about productivity with AI vs empirical results showing a reduction of productivity with AI.
Interesting! We have objective data to highlight a hole in the AI marketing hype! Subjective self-reports are hiding the objective truth. No wonder these companies don’t want to measure productivity objectively.
Neural Connectivity in Essay Writing
Nataliya Kosmyna at MIT conducted a study with 54 students who were given tasks of essay writing on high-level concepts. They all wore EEG caps to measure how brain activity differed between the brain-only group (control) and the two tech-supported groups: one with a search engine, another with ChatGPT.
Kosmyna made a critical observation which is directly applicable to all of us humans:
“Brain connectivity systematically scaled down with the amount of external support: the Brain‑only group exhibited the strongest, widest‑ranging networks, Search Engine group showed intermediate engagement, and LLM assistance elicited the weakest overall coupling.”
How does AI affect us?
“The use of LLM had a measurable impact on our participants, and while the benefits were initially apparent, as we demonstrated over the course of 4 sessions, which took place over 4 months, the LLM group’s participants performed worse than their counterparts in the Brain-only group at all levels: neural, linguistic, scoring.”
It would behoove us to pay special attention to that last finding. AI presents an immediate benefit coupled with long-term degradation. Effort stimulates the brain. Avoiding effort makes us dumber.
What Do You Want For Your Brain?
The results of these studies are verifiable in our own lives! Those fundamental human experiences of growth and atrophy are seen in these studies. If your brain is important to you like ours is to us, we need only to adopt one mindset to change our lives.
In Mindset, Dr Carol Dweck identified two mindsets: the fixed mindset and the growth mindset. A fixed mindset individual generally shies away from challenge and effort while feeling a potential threat against identity. Thus, they find their growth slowing down to a crawl. On the other hand, a growth mindset individual takes on ever-harder challenges while feeling a thrill to test one’s own limits. Thus, they experience the joy of continual growth!
We have all seen the deleterious effects of curation AI in social media: doom scrolling, increased anxiety, reduced attention spans, worsening mental health, and many more issues. How will generative AI affect human brains after 5 years of consistent use? How about after 10 years? What happens to the brains of our children in a world where generative AI is normal?
At Amplify, we choose to use our brains. Even if AI eventually overtakes humans in every real world use case, we will still be human. We will still need practice to grow our skills. Without practice, skills atrophy. With practice, skills grow. We believe the only sustainable path to ongoing creativity is to practice what we learn.
Humans created hardware. Then we created software. No AI was present when that happened, meaning software is a true FUBU idea: For Us, By Us. We author software as developers. We use software as customers. We are the purpose.
