Judgment is the scarce resource

One serious study finds AI-assisted customer-service workers faster; another finds experienced developers slower. Episode 7 uses that tension to challenge the idea that productivity is a property of the model. Berk Bayri looks at expertise, tacit context, meaningful human oversight, creative abundance and the danger of polished output arriving before the thinking is mature. The episode reframes “human in the loop” as an actual design problem: which human, with what authority, information and consequence? Featuring a Brian Eno excerpt on “premature sheen” and why finish can arrive before an idea is ready.

────────────────────────────────────────

References

Erik Brynjolfsson, Danielle Li and Lindsey R. Raymond, Generative AI at Work, Quarterly Journal of Economics 140(2), 2025. AI assistance increased customer-service productivity by 15% in the studied firm; gains were heterogeneous and stronger for less-experienced workers.

METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025. RCT with 16 experienced developers and 246 real tasks; AI access increased task completion time by 19% in this specific setting.

METR, We are Changing our Developer Productivity Experiment Design, February 2026. METR says later tools likely provide larger productivity benefits, but selection effects prevent a reliable estimate from its follow-up experiment.

Harvard Business School Institute for Business in Global Society, AI Won’t Make the Call: Why Human Judgment Still Drives Innovation, September 2025. Summarizes research on the continuing role of experience and judgment in evaluating ideas and strategy.

Anne S. R. Marx et al., Do We Have the Knowledge We Need? Rethinking Human-AI Decision-Making in Corporations (2026 position paper). Discusses fragmented organizational knowledge and proposes allocating human/AI agency based on task attributes and knowledge availability.