Image by Rainy Wong

Safety In Autonomous Vehicles Industry

Juan Carlos Aguilera

This article will be focused on the United States regulations and active programs due to the available documentation, with this I’ll try to illustrate the type of measurement that is being considered, similar strategies might be in process on other countries or continents like Europe. 


As we move forward with the development of Autonomous vehicles there are several safety measurements that need to be certified,  Is still a long way to go until we have a level five autonomous vehicle on the streets, but as we move forward to it, some tests are required, and an evaluation is needed. 


The United States is one of the few countries that are able to run some AV tests on the streets, Europe has not been able to come to an agreement for regulations and homologations for AV tests. The US relies mainly on the National Highway Traffic Safety Administration (NHTSA) to regulate, certify and approve the companies to perform tests on public roads. The NHTSA AV Tests Initiative allows the manufacturers to register and if some specific regulations and guidelines are followed to get an AV license. Currently, this initiative has its largest test density in Michigan, California, Arizona, and Florida but there are several other states that have intensive testing sites as well.  


The main objective of the NHTSA as well as most if not all of the manufacturers is to improve safety, backed by the fact that 94% of the critical crashes are due to human error, under this thought is expected that Automated or Autonomous vehicles have the potential to reduce or almost eliminate the critical and life-threatening crashes. In order for a vehicle to be classified as safe, it must comply with the Federal Motor Vehicle Safety Standards and certify that the vehicle is free of safety risks. 


To get to know in more detail how each of the manufacturers is working to achieve the safety levels requested by the NHTSA, you can reference the Safety Reports, most autonomous vehicles companies have released at least once a safety report where they talk about how their technology works, what’s their approach to achieving safety, and how their vehicles are going to interact safely with the public, this reports allow us as general public to better understand their goals and strategies toward a safety vehicle. Furthermore, there are other approaches that also help to determine the safety standards that each company is achieving during their public tests, that is the case of the disengagement reports published by the Department of Motor Vehicles or DMV.

Most of the safety reports are going to address the following topics in one way or the other, how the system behaves safely in public environments, how it functions in a safe way, what kind of safety procedures are in place for crashes as well as the type of tests that had been done to validate the crash safety, proactive or preventive safety (how to avoid a collision), safe interaction between the user and the vehicle, and what they are doing on cybersecurity a topic that has raised a lot of concern because a fully autonomous vehicle could be highly exposed to life-threatening situations if there is any cybersecurity attack. 

If you want to take a look at the different safety reports, I’ll leave the links at the bottom of the article. 


The disengagement report has been a really hot topic in recent days because it has been just released for the 2020 results. The disengagement report as its name tells us is a report that shows the amount of disengagement and total drove miles by each company. A disengagement is any event that causes the vehicle under test to exit its “autonomous” state to a “manual” state, this event could have been triggered either by the vehicle or the test operator who is in the vehicle, but why this is important? A disengagement is relevant because it means that the vehicle faced a specific scenario that it was either not prepared to face or the decision made by the vehicle was not the correct one, which could lead to fatal accidents although most of the cases may just be trivial disengagements. The companies should be doing a deep analysis of the root causes of the disengagement and how that will be avoided in the future. 


Some controversy has been presented recently because a lot of people is arguing how useful is this disengagement report, for a lot of o people and companies this might not provide any relevant information for the general public because the strategies and several other constraints could be drastically affecting the results of a specific company and this might not be an indicator of the company being on the lead or not. For example, maybe a company is performing simple or basic tests on low complexity streets while another company is testing more complex scenarios in a specific part of the city, the first company might have better results on the disengagement report but it can also mean that it has not been able to test yet what the second company is already testing: This types of scenarios could cause a deceiving interpretation of the results and the data. 


The disengagement report can be retrieved from its webpage where you can download the disengagement report and the miles driven report, Forbes has a short article where they talk about the results of 2020 and some general observations of who has driven the most miles as well as a brief talk about the disengagements report as well.

In 2016 RAND corporation published an article written by Nidhi Kalra and Susan M. Paddock where based on the statistical analysis they came to the conclusion that 5 billion miles will be required to be driven by an autonomous car to be able to classify it as safe improving the human behavior with a fleet of 100 vehicles driving 24 hours 365 days at 25 miles per hour will take about 225 years, which is not necessarily feasible for most of the companies, it’s also not taking in consideration some big setbacks that could cause some of the testing to get invalidated and the normal development process, for that reason the autonomous vehicles companies are relying greatly on simulation, where thousands of scenarios can be run in multiple computers and could reduce drastically the training time of the AV’s algorithms.


Figure 2: Miles Needed to Demonstrate with 95% Confidence that the Autonomous Vehicle Failure Rate Is Lower than the Human Driver Failure Rate

Image Source : RAND Corporation - Driving to Safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability