// NATURE NEWS — SPAZIO & SCIENZA
Explainable deep learning improves human mental models of self-driving cars
Nature
volume 657, pages 114–120 (2026) Cite this article
Self-driving cars increasingly rely on deep neural networks to achieve human-like driving1,2,3. The opacity of these black-box planners makes it challenging to accurately anticipate when they will fail4,5,6, with potentially catastrophic consequences7,8,9. Although research into interpreting these systems has surged, most of it is confined to simulations or toy setups because of the difficulty of real-world deployment10,11, leaving the practical utility of these techniques unknown. Here, we introduce the Concept-Wrapper Network (CW-Net), a method for faithfully explaining the behaviour of machine-learning-based planners that causally grounds their reasoning in human-interpretable concepts without sacrificing performance. We deploy CW-Net on a real self-driving car and show that the resulting explanations improve the human driver’s mental model of the vehicle, allowing them to better predict its behaviour, particularly in surprising situations. This demonstrates that explainable deep learning integrated into self-driving cars can be both understandable and useful in a realistic deployment setting. We anticipate our method could be applied to other safety-critical systems, such as autonomous drones and robotic surgeons, as well as to other architectures, such as end-to-end learning systems and vision–language–action models. Overall, our study establishes a deployment-validated pathway to interpretability for autonomous agents, which could help make them more transparent and safe.
There are hundreds of companies developing autonomous vehicle (AV) technology globally12, promising to revolutionize transportation for everyone. At present, the industry spans two segments: fully autonomous ride-hail systems and consumer vehicles with driver assistance13. In consumer vehicles, machine learning (ML) solutions have markedly improved the technology, yet they still require human intervention in unusual or challenging situations in which learned planners may not determine the correct action14. As driving is safety-critical, these infrequent failures matter, making it essential that the human driver is able to anticipate and be prepared for these situations15. However, the opaque nature of ML planners makes it challenging to interpret and communicate the causes of their decisions, hampering the ability of human drivers to understand and predict AV behaviour while achieving real-time situational awareness16,17,18.
Lack of effective communication between the AV and the human driver has contributed to multiple high-profile incidents, some resulting in fatalities7,8,9, highlighting the urgent need to make ML planners interpretable11. Previous studies have sought to address this using surveys and simulated scenarios19,20,21,22,23,24,25,26, a human driver emulating the AV27,28, or language models providing rationales for the driving policy in natural language3,29. However, these studies were theoretical, did not provide causally faithful explanations, were only evaluated in simulation or did not convincingly show the practical utility of the explanations to end users. This leaves open the question of how to provide explanations that are understandable, useful and faithful to the decision-making process of the AV in a realistic setting.
To answer this question, we scale up our work on interpretable-by-design deep reinforcement learning10 using motifs from the literature on concept-bottleneck models30 to propose the Concept-Wrapper Network (CW-Net). CW-Net grounds the reasoning of a black-box ML planner in human-interpretable concepts, such as ‘Approaching stopped vehicle’ or ‘Close to cyclist’. This method is rooted in case-based reasoning, a classical artificial intelligence (AI) approach31,32,33,34 inspired by cognitive models of human reasoning and memory35. CW-Net can be applied to arbitrary pretrained deep neural networks, does not require retraining from