economics Paradigm Challenge

Unlike regular government programs that get messy as they grow, AI-run projects actually work way better the bigger they get.

SSRN · March 18, 2026 · 6425002

Keshav Agrawal, Susan Carleton Athey, Ayush Kanodia, Shanjukta Nath, Emil Palikot

The Takeaway

Most social policies suffer from a 'voltage drop' when moving from a small test to the real world. However, because algorithms improve with more data, scaling an educational AI to a larger population made it 2.5 times more effective than the initial controlled trial results indicated.

From the abstract

Can personalized recommendations improve engagement in educational technology? We design, test, and scale a collaborative filtering system for Freadom, an English-learning app for Indian children. A randomized controlled trial (RCT) with 7,750 students shows that personalization, deployed in a single content section, increases engagement by 60% in that section and by 14% app-wide. We then exploit an eligibility threshold in a regression discontinuity design (RDD) to track effects over five month