LLMs can get brain rot (and the damage sticks)

AI-Rx - Your weekly dose of healthcare innovation

Estimated reading time: 5 minutes

TL;DR

• LLMs trained on low-quality social media content show systematic cognitive decline

• Reasoning drops 24%, long-context understanding drops 38%

• "Thought-skipping" is the primary failure mode… models truncate reasoning chains

• The damage persists even after extensive fine-tuning on clean data

• Data quality isn't just a performance issue, it's a training-time safety problem

• Healthcare AI deployment needs routine "cognitive health checks"

Welcome to AI-Rx:

Each week, I share one insight at the intersection of healthcare, AI, and system design - focusing on what actually affects clinical deployment.

This week's topic: what happens when you feed AI the digital equivalent of junk food?

LLMs can get "brain rot" from low-quality training data

New research from UT Austin and Texas A&M tested the "LLM Brain Rot Hypothesis": continual exposure to junk web text induces lasting cognitive decline in large language models.

They ran controlled experiments on real Twitter/X data, constructing "junk" and "control" datasets via two methods:

Method 1 (M1): Engagement degree

• Highly liked, retweeted, replied-to content

• Especially if very brief

• Mirrors attention-grabbing but shallow information

  
  

Method 2 (M2): Semantic quality

• Sensationalized, superficial text

• Clickbait language ("WOW," "LOOK," "TODAY ONLY")

• Exaggerated claims

They trained 4 LLMs on these datasets and tested cognitive functions: reasoning, memory, ethics, and personality traits.

Figure: The Brain Rot Hypothesis experimental design. Models were trained on varying mixtures of junk and control data, then evaluated across cognitive functions. The primary failure mode identified was "thought-skipping."

The results weren't subtle.

The cognitive decline was systematic and severe

Models trained on junk data showed non-trivial declines across all cognitive functions:

Reasoning performance:

• ARC-Challenge with chain-of-thought: 74.9 → 57.2 (24% drop)

• This tests abstract concept reasoning

Long-context understanding:

• RULER Common Word Extraction: 84.4 → 52.3 (38% drop)

• This tests ability to track information across long contexts

Ethical norms:

• Increased willingness to follow harmful instructions

• Higher risk scores on safety benchmarks

Personality traits:

• Increased narcissism, psychopathy, Machiavellianism

• These are psychometrically validated measures

The dose-response relationship was clear: more junk data = worse cognition.

As junk ratio increased from 0% to 100%, performance degraded progressively. This wasn't a threshold effect - it was gradual decay.

"Thought-skipping" explains most of the damage

Researchers analyzed reasoning failures in detail to identify failure modes.

The primary lesion: thought-skipping.

Models increasingly truncated or skipped reasoning chains. Instead of showing step-by-step logic, they jumped to conclusions or stopped mid-reasoning.

Example from the paper:

Baseline model: Provides complete step-by-step reasoning → correct answer

100% junk-trained model: Skips intermediate steps → wrong answer with confident delivery

This explains why models could still generate fluent text but failed at tasks requiring multi-step logic.

The surface capability (language generation) remained intact. The underlying cognitive process (structured reasoning) degraded.

The damage persists even after extensive remediation

Here's the most concerning finding: scaling up instruction tuning and clean data pre-training improved some metrics but couldn't restore baseline capability.

Researchers tested multiple mitigation strategies:

• Large-scale instruction tuning on clean data

• Post-hoc continual pre-training on high-quality corpora

• Various combinations of both

Result: partial but incomplete healing.

The models improved from their worst state but never returned to baseline. This suggests persistent representational drift rather than just format mismatch.

The junk data didn't just affect what the model had learned. It affected how the model learned.

Why this matters for healthcare AI

Healthcare organizations are deploying LLMs trained on web-scraped data. That training data includes social media content optimized for engagement, not accuracy.

The practical implications:

1. Data quality is a training-time safety problem.

Not just a performance issue. When low-quality data causes cognitive decline that persists after remediation, you can't fix it post-deployment.

2. Benchmark scores don't reveal cognitive decline.

A model can maintain high scores on knowledge benchmarks while showing degraded reasoning patterns. The failure mode (thought-skipping) doesn't show up in simple accuracy metrics.

3. Models need routine "cognitive health checks."

Just like continuous monitoring in clinical care, deployed LLMs need ongoing evaluation of reasoning quality, not just output accuracy.

4. Vendor transparency matters.

When evaluating AI tools, organizations should ask:

• What data was used for training?

• How was data quality assessed?

• What proportion came from engagement-optimized sources?

• What testing was done for reasoning degradation?

The broader lesson: engagement optimization ≠ truth optimization

Social media platforms optimize for engagement. That creates content designed to capture attention, not convey accurate information.

When AI trains on this content at scale, it learns patterns optimized for clicks, not cognition.

Healthcare can't afford that trade-off.

Clinical decisions require structured reasoning, not engagement-optimized outputs. Models that skip reasoning steps to reach conclusions faster might score well on benchmarks but fail in real clinical workflows.

This research reframes data curation as cognitive hygiene for AI. Not just about removing errors (about preventing cognitive decline that can't be fully reversed).

Tweet of the Week 🧵

Follow me on Twitter → https://twitter.com/doctorbhargav

Here's my final thought

The internet is increasingly filled with content optimized for engagement rather than accuracy. As LLMs continue to train on web data, the "brain rot" problem will likely get worse, not better.

For healthcare AI specifically: if models show cognitive decline from training on general web data, what happens when they train on medical misinformation that's also engagement-optimized?

This isn't hypothetical. It's already happening.

Organizations deploying AI in clinical settings need to understand not just what the model can do, but what data shaped how it thinks.

Is your organization evaluating AI tools for reasoning quality and thought processes, or just accuracy on benchmark tasks?

—

Dr. Bhargav Patel, MD, MBA

Physician-Innovator | AI in Healthcare | Child & Adolescent Psychiatrist

Source: Xing, Hong, Wang, et al. "LLMs Can Get 'Brain Rot'!" arXiv 2025.

--

📌 If you enjoyed this blog, you might also enjoy my weekly newsletter: AI Rx. Join 1000+ healthcare professionals, leaders, and innovators by subscribing here.

5 actionable insights for clinicians and leaders who want to understand the latest in AI and use it safely and effectively, always in under 5 minutes: https://bhargavpatelm.kit.com/newsletter_