Harvard researchers found that as models become more accurate, they become less diverse. The information is in there. Standard decoding just never finds it. Queenie Luo, Gary King, Michael Puett, and Michael D. Smith propose a fix that requires no retraining.
Every major AI lab is racing to improve benchmark scores. Models that answer factual questions more accurately and reason more reliably are winning on leaderboards and attracting users. What gets less attention is what is sacrificed in the process. A new working paper from Harvard identifies a specific casualty: the ability to generate genuinely diverse, creative responses over extended use.
As models are trained to converge on correct answers, their probability distributions become more peaked around the most common outputs. More information gets pushed into statistical tails where standard decoding methods never look. Newer, more accurate models generate narrower and more repetitive outputs than older ones when asked open-ended questions.
The researchers call the most affected category the "search quest": any prolonged, exploratory effort to find an answer that does not yet exist in a fixed form. Choosing a wedding dress, finding a research topic, naming a startup, designing a product. Unlike a factual query with a correct answer, a search quest requires learning the space of possibilities. You do not know what you want until you have seen enough options to understand what is out there. Current AI tools produce a useful first list and then start repeating themselves.
The mechanics of why this happens are grounded in how models generate text. At each step, a model computes a probability distribution over all possible next tokens. Standard decoding methods select from the highest-probability tokens, keeping output fluent but ensuring the vast majority of information encoded in the model's parameters is never accessed. The researchers illustrate this: when prompted to brainstorm 18th century world history book topics, a model's top tokens lead to European history. Move to the 300th through 2000th positions and you find tokens like "Asia" and "African," topics the model clearly knows but never volunteers.
The Harvard team's solution, called Recoding-Decoding, introduces two forms of randomness at generation time without retraining. A random priming phrase, constructed from the 2,000 most common English nouns, is embedded in the prompt as "Related to NOUN." A random diverting token is drawn from three-letter starting stems of the 5,000 most common English words and placed at the start of each new sentence. Both exploit positional bias, a documented property of language models: they attend more heavily to tokens at the beginning and end of input sequences.
The battlefield test is the most striking result. Standard GPT-5.1, queried about interesting world history battlefields, produced 19 unique locations across 1,000 runs, all in Europe or North America. RD produced 1,307 unique battlefields across a globally distributed range, including East Asia, South Asia, the Middle East, Africa, and Australia. Standard decoding identified zero battlefields that RD did not also find.
Across 50 brainstorming topics, RD consistently outperformed all ordinary decoding variants on diversity while maintaining relevance scores of 0.99, essentially matching standard decoding's 0.99 to 1.00. On five public datasets spanning 500 prompts, RD increased diversity by 161 percent on GPT-5.1 and 140 percent on Gemini-3, with relevance scores within one percentage point of standard decoding.
One finding cuts against a common assumption: that newer, more capable models are better for creative tasks. The data shows the opposite. As model accuracy improves, the gap between standard decoding and RD widens. Better benchmark performance comes at a measurable cost to output variety when the model is used for open-ended exploration. GPT-5.1 under standard decoding scores lower on diversity than older GPT-3.5.
The paper also addresses collective effects. Research published in Science Advances found that generative AI enhances individual creativity while reducing collective diversity: different people's outputs converge on the same ideas. The researchers replicated this pattern when they discovered that students in a university course submitted nearly identical essays without communicating. RD addresses this by ensuring different users asking similar questions receive genuinely different suggestions.
The method requires either a completion API or a simulated version using the chat completion API. As frontier labs increasingly restrict raw API access in favor of structured chat interfaces, the practical ceiling for this approach depends on whether providers make completion endpoints available. The research is a reminder that what AI labs optimize for determines what they get and sometimes what they sacrifice.
