For more than a decade, Stack Overflow has been the go-to forum for developers seeking answers to coding questions. At its peak in the mid-2010s, the site saw more than 200,000 new questions each month. Those days are gone. Since the arrival of AI assistants such as ChatGPT, Stack Overflow’s usage has fallen off a cliff. In fact, from March 2023 to March 2024, the volume of new questions dropped from ~87,000 to ~58,800, a 32% decline in just one year. By the end of 2024, monthly questions were down 40% year over year, hitting levels not seen since 2009. Developers, who previously relied on community validation for accurate answers, now increasingly trust an algorithm’s replies, which are immediate and confident (yet not always correct).
This shift creates an ironic problem: Those large language models (LLMs) owe their impressive knowledge bases to human-generated content from platforms like Stack Overflow, creating their own cannibal’s dilemma of sorts. I’ve written at length about this problem, but the real question isn’t about how to recapture what we’ve lost, but how to move into the future. As these communities fade, what will LLMs use for future training, and how will developers ensure the accuracy and reliability of the answers these tools provide?
Paradoxically, the AI assistants displacing Stack Overflow are themselves products of Stack Overflow’s success. LLMs such as ChatGPT became “smart” by training on vast troves of human-generated content—much of it scraped from sites like Stack Overflow. Every well-explained answer, every debated best practice, and every upvoted code snippet on these forums became grist for the AI mill. In a very real sense, the data that flows from community Q&A platforms is the bedrock on which future iterations of LLMs are built. Early models were trained on data sets that included millions of Stack Overflow posts capturing the nuances of programming problems and their solutions.
