Study finds LLMs can give wrong climate answers, especially under user pushback
The test covered 33 models and asked whether assistants keep accurate climate information when a user objects.
A study led by researchers at University College London and UNESCO set out to measure how large language models answer climate questions, a gap in existing research given that chatbot sessions are private and cannot be audited directly. The team combined basic factual benchmarks with more involved conversations, including cases where a user pushes back on an answer. They tested 33 models, among them several versions of the leading proprietary and open-weight systems, so the findings are not tied to a single product. The work matters to developers and companies deploying these tools because it asks whether an assistant holds to accurate climate information under pressure rather than simply yielding to a user's objection.
Why it matters
Chatbot sessions are private, so outside researchers have no direct view of what people are told about climate. This study tests the models directly, using both basic factual questions and longer conversations where a user disputes an answer. Because it covers 33 models, including several versions of major proprietary and open-weight systems, the results point to a broad pattern rather than a flaw in one product. For readers who rely on these tools for climate information, the open question is whether an assistant will hold to accurate answers or give way when a user pushes back.
Matters or hype?
Does this story matter, or is it hype? Vote before you see what everyone else said.