54% of Readers Preferred AI Writing Over Published Authors
TL;DR
54% of 86,000+ New York Times quiz-takers preferred AI passages — in a blind test matching AI-generated snippets against published authors like Cormac McCarthy and Ursula K. Le Guin, readers not only struggled to identify the human text but often liked the AI version more.
The real fight is over what the quiz measured — critics said the Times reduced writing to decontextualized sentence fragments, which strips away the things many writers believe actually matter: voice, structure, sustained argument, and ideas developed over thousands of words.
Paul Roetzer argues the core question isn’t whether AI can write well, but when humans should use it — his framework runs from Level 0 “all human” work like essays and keynote speeches to Level 4 autonomous AI content like FAQs, product descriptions, and repetitive SEO copy.
“The process is the purpose” is the line that anchors the episode — for personal writing like LinkedIn posts, podcast commentary, and his newsletter, Paul says he avoids AI because writing is how he thinks, learns, and creates authentic connection.
Mike Kaput’s warning to writers is blunt: if you’re offended because you think AI can’t do this, update your priors — he says denying the quality gap has closed is less useful than accepting reality and figuring out how creative work, training, and meaning evolve from here.
They end on a surprisingly optimistic note for writers and journalists — both hosts argue that strong writers remain valuable because writing signals critical thinking, and Roetzer says as a CEO he’s actively trying to find roles for former journalists and storytellers inside AI-native organizations.
The Breakdown
The New York Times quiz that hit a nerve
The episode opens with the now-viral stat: more than 86,000 people took a New York Times interactive quiz, and 54% said they preferred the AI-written passages over the human originals. Readers were shown short pairs of passages, asked to guess which was AI and which they liked better, and they consistently struggled to tell the difference.
Why literary people were furious
The hosts note that the source material included major authors like Ursula K. Le Guin, which made the whole exercise feel especially loaded given her long opposition to the commodification of art. Critics pounced on the format itself: a few isolated sentences are not a novel, essay, or story, and reducing writing to fragments warps what “good writing” actually means.
Paul reads one matchup: Claude vs. Cormac McCarthy
Paul walks through one example from the quiz’s literary fiction category: a sparse, atmospheric church scene versus a famous passage from Blood Meridian. He clicked the first one, learned it was written by Anthropic’s Claude Opus 4.5, and saw that 50% of readers chose it; in another example, he says 67% preferred the AI version. His broader takeaway is almost resigned: across text, images, audio, and video, it’s only getting harder to know what a human made.
The real question: not “can it write?” but “when should we use it?”
That leads Paul to the point he actually cares about, which he says framed his AI for Writers Summit keynote: the issue is not whether people prefer AI writing, but when humans should let AI write at all. He quotes his own line, “For me, writing is thinking,” and explains that for some work, saving time is irrelevant because the act of writing is how he processes ideas and creates meaning.
His five-level human-to-machine writing scale
Paul then lays out the framework: Level 0 is all human, where unique voice is essential; Level 1 is AI-assisted research and editing; Level 2 is co-writing; Level 3 is AI-led content with human review; Level 4 is mostly autonomous AI output. He maps examples onto each level, from investigative journalism and manifestos on the human side to email templates, FAQs, product descriptions, and SEO content on the machine-heavy side.
Authenticity is the dividing line
He sharpens the rule of thumb: more human matters when the writing is ethically sensitive, emotionally nuanced, central to personal or brand identity, strategically important, or explicitly valued for the author’s voice. More machine works when the content is factual, standardized, high-volume, repetitive, and useful mainly for information rather than self-expression — like generating a thousand product descriptions.
The book dilemma makes it personal
Paul gets candid about his own tension here: he has book ideas he believes matter, but writing a business book takes 300 to 500 hours, and as a CEO he doesn’t have that time. He says he could probably get one to market in 30 days with AI, whereas doing it entirely himself means it may not happen this year — so he’s actively wrestling with whether being the “shepherd” of ideas is enough.
Mike’s message to writers: accept reality, then adapt
Mike says the backlash is understandable emotionally, especially because this hits creatives differently than coding, but he adds a hard truth: if you’re upset because you believe AI simply can’t write well, you need to update your priors. Still, both hosts end with optimism — writing remains a proxy for critical thinking, writers are often excellent at working with AI, and Paul says he’s actively trying to build roles for former journalists and storytellers inside his company because he sees them as deeply valuable.