According to a recent survey on autonomous vehicles, around 66% of US drivers express fear about fully autonomous vehicles and another 25% express uncertainty. And yet despite this, interest in advanced driver assistance systems remains high.

Well-publicised incidents involving autonomous vehicles means that people remain apprehensive about their safety because they feel driverless car don’t handle situations on the road in the same way that they would. Fundamentally, they don’t believe that the driverless car is as good at driving as they are.

The reality is that driverless cars are engineered and designed by human minds, but they cannot replicate how human minds work as they lack the human attributes of common sense and nuance.

So the way a human driver would automatically handle an unexpected adverse situation on the road can be a real challenge for a driverless car.

But are recent advances in AI set to change things?

AI systems increasingly have language capability – think of ChatGPT – a form of generative AI that uses natural language processing to create dialogue and respond to questions. It uses reinforcement learning through human feedback which is augmented with machine learning using specialised algorithms to find patterns within data sequences to improve responses. The more tailored the data, the more tailored the response – technically, ad infinitum.

So, could systems with language capabilities enable driverless cars reason and behave similarly to human drivers?

In the late 2010s, the development and arrival of deep neural networks (DNNs) enabled data to be processed in a way that was similar to the human brain. In terms of autonomous driving, this meant that critical elements such as obstacles and pedestrians could be identified in traffic scenarios.

Position, size and situation of the obstacles could be represented through spatial properties which included speed and distance relative to the autonomous vehicle.  A ‘sense-think-act’ engineering approach ensued by which sensor data was used to predict obstacle trajectories and then plan the vehicle’s subsequent actions.

The problem was, this didn’t correlate with human thought processes or the brain mechanisms at work when someone is driving. And of course, when there is so much about the brain that we still don’t know, trying to apply human intuition to a driverless vehicle is problematic.

So, research involving psychology, cognitive science, and neuroscience is being applied to autonomous driving. The theory that sensing something and acting upon that sense are not so much sequential as correlative processes suggests that humans may perceive their environment in terms of their capacity to act on it as an interrelated process rather than sensing the environment and then acting upon it as a linear process.

For example, a human’s driving actions may focus on their intended outcome whereas an automated ‘sense-think-act’ approach processes the driving scenario in distinct steps without any sense of intention. This means that the autonomous vehicle’s data driven approach will struggle to generalise and respond to unseen situations – which is where human drivers excel. And although training datasets can become larger and larger, it still feels beyond the ability of an automated car to have responses to all possibilities.

Put simply, humans can assess a scenario with the addition of common sense through the knowledge, reasoning, and intuition gained simply by being human. Even though the situation may be novel, one human can interpret the actions of another human to make judgements and take appropriate action.

More recently however, advances in AI using large language models (LLMs) are starting to offer a solution. Because they are proficient at understanding and generating human language through their training on huge amounts of information, they can develop something akin to human common sense. This enables them to comprehend complicated and unseen situations, vocalise explanations, and recommend actions.

In driverless vehicles, research is focusing on using multimodal models to provide commentary and explanations of driving decisions. However, with this potential comes additional problems regarding the evaluation of safety and reliability. Inevitably this will be more complicated than with a sequential ‘sense-think-act’ approach. New testing approaches will be required in addition to the increased demands on computer resources and the need for real time responses that conflict with the speed with which responses can be generated.

Nevertheless, if LLMs are to be the future of autonomous vehicles that are able to apply common sense, then chatbot language capabilities of AI models could be the technology to provide it. We may yet have driverless cars that can think like humans.