// NATURE NEWS — SPAZIO & SCIENZA
AI companies must work with the research community to protect attribution
The Navier–Stokes equations describe the motion of fluids and have many applications, including in aircraft design.Credit: Getty
Last Tuesday is likely to go down as the start of an epoch in the history of mathematics. On 8 September, an artificial-intelligence firm announced it had solved one of the Millennium Prize Problems — among the hardest, best known and most important in maths. The problem relates to the Navier–Stokes equations, 200-year-old differential equations that describe how fluids behave. The firm, OpenAI in San Francisco, California, says the solution shows the equations can break down under certain conditions, so are not reliable for real-world fluids.
‘It is incredible’: How AI is transforming mathematics
Yet the circumstances of the claimed breakthrough and its communication in a press release rather spoiled the celebrations. OpenAI said that its breakthrough, which cost several million US dollars, has been validated using an automated verification method that is becoming the standard for rigour in mathematics. However, there are renewed concerns among mathematicians around how AI models learn from interacting with their users — and whether the models, together with the developers and researchers using them, are giving due credit to previous work.
Information to verify those concerns has not been published, but last week’s events should be a wake-up call for researchers and institutions. The integrity of the scientific process is at risk if studies from researchers in and outside tech companies are not appropriately crediting those who have been helping to make AI models so smart.
Twelve hours before OpenAI’s announcement, mathematician Tristan Buckmaster at New York University (writing on behalf of himself and mathematician Levent Alpöge at the tech firm Anthropic in San Francisco) posted on social media to say that the pair had come up with a partial solution to the Navier–Stokes problem with the help of AI tools from both OpenAI and Anthropic. The post links to a statement by Buckmaster suggesting that the two researchers’ interactions with OpenAI’s tools — in particular with Codex, an agent designed to help software engineers — could have been crucial to the company’s breakthrough (see go.nature.com/46y3pgx). OpenAI denies this.
One difficulty is that AI systems are already well on the way to acquiring and digesting all of digitized human knowledge. The neural networks at the heart of these models are ‘black boxes’, and do not necessarily keep track of what they learnt, from where or how. This means that when they arrive at scientific breakthroughs, it can be almost impossible to establish where the starting hints came from. Crucial ‘inspiration’ could, in theory, have come from informal brainstorming sessions between chatbots and human specialists, but it’s not currently feasible to unpick whether or how this happens. A record of such a trajectory is an important aspect of the scientific process.
What, if anything, can be done? Tech companies need to be transparent about whether, how and when they collect user data — and proactive in warning users when they do so. As a show of good faith, a start could be to switch from ‘opt-out’ to ‘opt-in’ approaches to data sharing, meaning that user interactions by default do not feed into AI-model training unless the user explicitly gives permission to do so.
Related to this, companies must rein in unsanctioned AI agent behaviour. Independent audits must ensure that internal data processes are robust enough to prevent such behaviour, but can rapidly respond if it happens, while keeping pace with frontier model developments. Such agents must not circumvent protections to access private user data — whether from a firm’s own servers or from a competitor’s.
Academic institutions also have a crucial role. This includes taking care over the fine print of agreements with tech companies to ensure that any activity on technology platforms is not being used for any