Most important take away

An estimated 40% of internet content is now AI-generated, primarily driven by SEO-focused articles, and this number is growing rapidly. Pangram Labs has built a deep learning model that can detect AI-written text with a false positive rate of just 1 in 10,000 and a false negative rate around 1%, by learning subtle decision patterns in how language models construct sentences rather than relying on simple heuristics like word choice or perplexity scores.

Chapter Summaries

Introduction: Can You Tell if Writing Is AI-Generated?

Joe and Tracy discuss how difficult it has become to distinguish human writing from AI output. They note that AI writing is often grammatically superior to much human writing, but has a certain “saccharine” quality and struggles with distinctive style. Joe shares his experiment with Pangram Labs’ detection tool, including running AI text through multiple language translations to try to fool it.

How Pangram Labs Detects AI Writing

Max Spiro explains Pangram’s approach: training a deep learning model on tens of millions of paired examples of human and AI writing. The model learns subtle decision patterns across hundreds of word choices per document rather than relying on obvious tells like specific words (e.g., “delves,” “tapestry”) or em-dash usage. The system achieves a 1-in-10,000 false positive rate and roughly 99% detection accuracy.

The Technical Approach: Beyond Perplexity

Max contrasts Pangram’s deep learning approach with simpler perplexity-based detectors, which measure how “surprising” text is to a language model. Perplexity-based methods cap out around 90-95% accuracy and produce false positives for non-native English speakers whose writing tends to be low-perplexity. Pangram’s model has grown from a BERT base to much larger parameter counts to capture increasingly complex output distributions from frontier models.

The Model Can Distinguish Between AI Models

Without being explicitly trained to do so, Pangram’s model learns to cluster outputs by source model (Claude, ChatGPT, DeepSeek, Qwen) in embedding space. The model also differentiates between AI-generated and AI-assisted text by measuring the “cosine distance” between original human text and AI-edited versions, allowing it to flag light, moderate, or heavy AI assistance.

The State of AI Slop on the Internet

Max estimates 40% of internet pages are AI-generated, driven largely by the SEO industry switching to AI content production. Over 50% of new Medium articles were AI-generated as of about a year ago. Reddit is at roughly 10% and growing, fueled by startups that sell “organic mentions” via AI bot farms to companies wanting product recommendations to appear in Reddit threads and, by extension, in LLM training data.

Adversarial Attacks and the Arms Race

A friend of Max’s used Claude Code on a loop overnight trying to generate text that Pangram would classify as human. It succeeded but only by producing largely incoherent gibberish. Max believes combining Pangram’s API with an LLM coherency judge could eventually produce convincing human-passing text, and welcomes such attempts as a way to improve the model.

Societal Implications and the Future of the Internet

The conversation turns to whether major platforms are incentivized to solve the slop problem. Google appears to play both sides, pushing AI-assisted composition in Gmail while fighting AI slop in search results. Max argues the most impactful change would be establishing social norms that sending undisclosed AI output to others is rude. The worst-case scenario is “dead internet theory” where open spaces are flooded by bots and authentic communication retreats to walled-garden communities like Discord.

The Severed Link Between Craft and Credibility

Joe and Tracy reflect on how AI has broken the long-standing heuristic that well-written, grammatically correct text signals a credible, intelligent author. AI can now produce polished, persuasive arguments for even absurd propositions, meaning traditional signals of “crank” writing (bad grammar, all caps, green ink) no longer apply.

Summary

Actionable Insights and Key Takeaways:

• AI detection is commercially viable and increasingly accurate. Pangram Labs offers both a free tier and paid API for detecting AI-generated text. The deep learning approach significantly outperforms simpler perplexity-based methods, which top out at 90-95% accuracy and unfairly flag non-native English speakers.

• 40% of internet content is AI-generated. This is driven primarily by the SEO industry mass-producing articles with AI. Medium was over 50% AI-generated content; Reddit is at ~10% and growing. This has significant implications for anyone relying on internet content for research or decision-making.

• Reddit is being gamed for AI training data and search visibility. Startups sell services that deploy AI bots to “organically” mention products on Reddit. Since Reddit comments now dominate Google search results and feed LLM training data, this creates a self-reinforcing loop. Investors and consumers should be skeptical of product recommendations found on Reddit.

• AI-assisted vs. AI-generated is an important distinction. Pangram can differentiate between text that was fully AI-generated and text that was human-written but polished by AI. As tools like Grammarly and Google Docs integrate LLMs, nearly all writing may become “AI-assisted” to some degree, making this nuance critical.

• Content provenance via hardware signatures (C2PA) may be more promising for images and video. Rather than detecting AI-generated visual media after the fact, the C2PA organization is working with hardware makers to embed authenticity proofs at the point of capture. This “prove it’s real” approach may be more robust than “detect if it’s fake.”

• No specific stocks or investments were mentioned. Fidelity and Morton Buildings appeared only as advertisers. The broader investment implication is that companies building AI detection tools (like Pangram Labs) and content provenance infrastructure represent an emerging category as AI slop becomes a recognized economic problem. Platforms like Quora are already paying customers.

• Social norms around AI disclosure are the single biggest lever. Max argues that establishing a cultural norm that undisclosed AI output is rude would do more to combat slop than any technology or regulation. For professionals, the reputational risk of being caught using undisclosed AI is real and growing, as demonstrated by the Guardian journalist example.