New IIHS Study Confirms What We Suspected About Tesla’s Autopilot And Other Level 2 Driver Assist Systems: People Are Dangerously Confused

Being right about something occasionally is fun, and gives you a little dopamine rush of pleasure, a pleasure that I personally encounter extremely rarely. I feel like I’m right about something today, so mark your calendars, but it’s not something I’m actually pleased to be right about: People both don’t understand and aren’t using Level 2 (L2) semi-automated driving assist systems well or safely. I’m being bold and saying I’m right about my thoughts on this – which I’ve been ranting about for quite a while – because a study from the Insurance Institute for Highway Safety (IIHS) just dropped, and it comes to essentially the same conclusions: Significant percentages of people who have cars with L2 semi-automated driver assist systems overestimate the capabilities of the systems, and as a result use them in potentially dangerous ways. With these systems becoming more and more common, it’s time we addressed this bluntly and honestly.

The study, called Habits, Attitudes, and Expectations of Regular Users of Partial Driving Automation Systems, is based on responses to questions from a pool of 604 respondents who own cars with one of the three most common L2 semi-automated driving systems, broken down like this:  General Motors Super Cruise (200), Nissan/Infiniti ProPILOT Assist (202), and Tesla Autopilot (202).

What Level 2 Driving Automation Systems Actually Do

Now, before I get into the results of this study, let’s just make sure we’re all on the same page regarding exactly what each of these driver-assist semi-automated systems actually do. Fundamentally, they they all control speed and braking, dynamically keeping set distances from the car in front like dynamic cruise control, and they do some steering, keeping centered in a lane and following road curves and so on. Some systems do a good deal more, like Tesla’s Autopilot and GM’s SuperCruise, allowing for lane changes and some degree of navigating on streets other than highways, and so on.

What all these systems have in common is that they are very much not actually self-driving systems, and the person in the driver’s seat must remain alert and ready to take control at any moment, because any of these systems could disengage or make a poor decision at any time, with zero warning. This fact that the driver must remain on constant alert even when the semi-automated systems appear to be doing most of the work associated with the driving task, is at the heart of the problem that this study reveals, and, frustratingly, is something that researchers who study automation have known about since at least 1948.

The Problem

The problem even has a name, taken from N.H. Mackworth’s 1948 study called The Breakdown of Vigilance during Prolonged Visual Search: the Vigilance Problem.

Essentially, the vigilance problem is not a technological one, it’s a human one. Given a system that operates almost independently but requires passive oversight by a human, that human will not be good at paying attention to the system, and as a result will not likely be ready to take action when needed. In the case of these semi-automated driving systems, the situation is actually worse, because the naming and marketing and perception of these systems suggests more capabilities than they actually have, which makes people even less vigilant.

Nissan’s ProPILOT Assist is a bit of an exception to this, as its marketing and implementation do a lot less to suggest capability, and the numbers from this IIHS study back this up, as we’ll see soon.

What Drivers Think They Need To Do When Using Automated Driver Assist Systems

In fact, let’s look at some numbers right now, starting with this really basic question: Are drivers using these systems comfortable with not paying attention to the road while the systems are engaged? Here’s what the study found:

Over half of Super Cruise users (53%), 42% of Autopilot users, and only 12% of ProPILOT Assist users said they were comfortable letting the partial driving automation drive the vehicle without having to watch what was happening on the road.

Holy crap, over half of Super Cruise users didn’t think they needed to “watch what was happening on the road?” It’s worth noting that Super Cruise is the only one of these three systems that requires no physical contact with the steering wheel when engaged, instead relying on a camera-based system to track the driver’s eyes.

Tesla owners at 42% aren’t much better, but Nissan owners actually are, at only 12%.

The study explains a bit:

Over-trusting either hands-free (Schneider et al., 2022) or hands-on-wheel partial automation (Victor et al., 2018) can lead drivers to not intervene even when they see a hazardous situation forming in front of them because they incorrectly believe the system can handle more than it was designed to do. One question this study could not answer was whether hands-free driving capability is more likely to give users the impression that the system is more functionally capable and safer than a hands-on-wheel system. Nonetheless, more than half of Super Cruise users in the current study said they were comfortable letting the system drive itself without having to watch what was happening on the road, compared with approximately 40% of Autopilot users and only 12% of ProPILOT Assist users. Super Cruise users were also far more likely than the other two groups to say that most non-driving-related activities were safe to do while using the system.

What The IIHS Study Reveals Drivers Are Doing Instead Of Driving

So what the hell are these “non-driving-related-activities” that people are doing instead of paying attention to the road? Here’s what the study found:

Some of these seem like no big deal: eating and/or drinking is something we all have done while driving, same with talking to passengers or talking on a cellphone call via Bluetooth or looking at scenery. Most of those people have been doing since before cars even shifted themselves, let alone drove themselves in any sort of way.

Other activities on the chart a lot more alarming: per the survey, 49% of Super Cruise, 44% of Autopilot and 19% of ProPILOT users say they’ve been texting, which isn’t great. But 19% of Tesla Autopilot users have apparently used a laptop or tablet? And 20% watch videos? 18% read a book? What the hell? And 10% of Autopilot users admitted to sleeping? Sleeping!

The fuck is wrong with these people?

That chart notes the difference in what people do with the systems both on and off, and come to the very obvious conclusion that drivers are much more likely to do stupider things when their semi-automated systems are on:

The present study supports the finding of previous research that partial driving automation facilitates engagement in non-driving-related activities (e.g., Dunn et al., 2021; Noble et al., 2021; Reagan et al., 2021); however, the attitudes, expectations, and habits around individual activities and system design safeguards vary among drivers depending on the system they use.

Interestingly, Super Cruise users were the most likely  “to characterize phone and other peripheral device use, watching videos, grooming, reading, and having hands off the wheel as safe to do while using the system,” which I would suspect is related to the fact that Super Cruise does not require contact with the steering wheel. ProPILOT users were the least likely.

Attention Reminders In Level 2 Systems

All these systems to have Attention Reminders, where if the system thinks you’re not paying adequate attention, measured by either steering wheel torque sensors or eye-tracking cameras, it gives the driver a warning, and, if that warning isn’t heeded, the semi-automated driving assist system may be suspended or disabled.

The study surveyed driver’s “perceived annoyance” at these reminders:

Sometimes ignoring these warnings can cause the driver to be locked out of the driver assist systems if they don’t respond to prompts quick enough, and we get some fascinating details of some of these lockout scenarios Super Cruise users noted, maybe with more specificity than you’d expect:

Moreover, seven Super Cruise users mentioned eating as one of the activities that led to their lockout experience—most of them specifically described taking their hands off the wheel to eat—but none of the Autopilot users cited this reason. Illustrating the challenges of eating while driving, two Super Cruise users said that they had dropped their food, in one case a taco and in another case a hamburger, and had to let go of the wheel to retrieve their meals. Fifty-four percent of Super Cruise and Autopilot users who experienced lockouts said they were at least somewhat annoyed these lockout events happened.

Look at that; IIHS has recorded two dropped food incidents, one a taco-class incident and one a burger-class incident. Fascinating.

In the attention reminder context again we see differences between the systems, though for the most part, drivers seem to understand the need for these reminders. Nissan ProPILOT Assist users got the fewest reminders, and the study has some good insight into why (emphasis mine):

A substantial proportion of ProPILOT Assist users reported never having received attention reminders, which raises the possibility that interactions with the system’s lane-centering feature might also contribute to these group differences. ProPILOT Assist’s lane-centering support remains active while the driver steers within the lane; this characteristic is a component of cooperative steering, or shared haptic control. This design philosophy encourages the driver to actively participate in the steering task (Marcano et al., 2021), helping to reinforce the driver’s role in the relationship and improving the driver’s sense of agency and willingness to intervene whenever necessary or desired (Wen et al., 2019).

The Benefit of Cooperation

Unlike Autopilot or Super Cruise, Nissan’s system is designed to be cooperative, and doesn’t disconnect when the driver gives steering, throttle, or brake inputs. It’s not modal, like the other systems, and as such the driver is never in a position where they would “give up” control of the car to the machine. While this seems like a less advanced way of doing things, I think for semi-automated L2 systems it makes so much more sense, because the real problems arise when the driver thinks the machine has more control than it actually does. If you never imply that the car is in complete control, with the driver only monitoring and instead always keep the driver an active participant in the loop, then many of the vigilance problem issues simply won’t happen.

The study notes this difference between ProPILOT Assist and the other two systems:

In contrast, Autopilot’s lane-centering support deactivates whenever the driver exceeds a (relatively small) threshold amount of steering torque. Super Cruise’s lane-centering support temporarily suspends and only automatically reactivates once the driver has stopped steering and the system has regained the necessary information to  position itself within the lane. It is unclear to what extent these design differences influence driver behavior, but the temporary suspension or deactivation of system support might make drivers more reluctant to participate in the driving over time (Banks & Stanton, 2015).

Perhaps counterintuitively, the more work the semi-automation system does, the less it seems to make drivers want to take control when needed, out of fear of the system disengaging. While I don’t think this study has definitively proven this to be the case, it nevertheless is a pretty alarming concept: Driver reluctance to take control so as not to cause the system to disengage has a lot of potential to be dangerous. There should be no hesitation for drivers to take control if they feel uncomfortable in any way.

The Unexpected Effects Of Camera-Based Driver Monitoring Systems

While driver monitoring systems are getting better, even the camera-based ones are not foolproof, and this study also noted Tesla drivers using a water bottle to fool steering wheel torque sensors. Additionally, camera-based driver monitoring systems that don’t require holding the wheel have another disadvantage in that they further the illusion that the system is more capable than it actually is, which permits dangerous behavior like not looking at the road (emphasis mine again);

While steering torque monitoring alone is a poor basis for managing driver behavior (Lin et al., 2018), the current study’s findings indicate that camera-based driver monitoring on its own is not a silver bullet either. The longer a driver looks away from the road the greater their crash risk (Klauer et al., 2006; Yang et al., 2021). Even though Super Cruise uses camera-based monitoring to know where the driver is looking, many Super Cruise users reported being more likely to look away from the road for extended periods while using the system than the drivers in the other two groups. It is unclear why this is the case, but, compared with Autopilot and ProPILOT Assist users, Super Cruise users were also more likely to say that looking away from the road is safe to do while using the system than during unassisted driving and to say that they can do it better and more often with the automation’s support. Clearly, the expectations and attitudes of many of these drivers do not reflect an accurate understanding of the system’s limits.

The study also notes that we’re still in the early stages of adoption of semi-automated systems, and as a result these findings may be skewed to specific “early adopter” behaviors:

As Lin et al. (2018) noted, the scarcity of vehicles equipped with partially automated systems in the registered vehicle fleet means that studies such as this one are presently capturing behavior and perceptions of early adopters. It is unclear how user attitudes and expectations will evolve as system designs change and the technology becomes more widespread.

What I find alarming about the early adopter idea is that of anyone, early adopters should be the most aware of a system’s technical abilities and limitations, shouldn’t they, since they’re so damn interested? Or, could this just mean that the eagerness of an early adopter is enough to make them victims of their own wishful thinking about the capabilities of these systems?

Perception And Marketing Is A Big Deal

Of course, a lot of the issue is with how these systems are marketed and portrayed; names like Autopilot and a heavy emphasis on hands-free driving does lead to a lot of overestimation of these things:

Worryingly, some drivers appear to have a false sense of security about how they are meant to use the technology and what it is designed to do. Misunderstanding the user’s roles and responsibilities corresponds with a higher degree of engagement in nondriving-related activities compared with driving without assistance. This confusion is likely influenced by system design, as some systems are more likely to give drivers the impression that they are more functionally capable than they are.

There’s also some interesting data about the demographics of the groups in the study, none of which is all that surprising: GM’s system, showing up mostly on Cadillacs, skews old. Teslas skew younger and overwhelmingly male, and the Nissans are the group that has demographics closest to just the general population. Again, not shocking.

What is a bit shocking – well, actually, it’s not shocking, because what’s happening is exactly what we’ve known happens with systems that do most of the work, and still somehow try to demand constant vigilance: It doesn’t work. Maybe alarming is a better word, because these systems are already on the roads, and they’re continuing to be built and sold, and they are all, I think, fundamentally, inherently, broken.

The Takeaway

This isn’t something that needs a technological fix, because it’s not a technological problem. It’s a human problem, and it doesn’t matter how many over-the-air updates you send out or what revision the latest Tesla Full Self-Driving Beta is up to, because if any of these systems still demand that a person be ready to take over without warning, they’re doomed.

The better the system is at driving, the worse it will be for keeping people ready and alert, so there’s a bit of a paradox here, too. The more featured the system, the more advanced it is, the more it does, the less it demands from the driver, the worse it actually is. Because if it still may need you to intervene at any moment, the less engaged you are, the worse the result will be if anything should go wrong.

That’s why of all of these, Nissan seems to be on the most workable track: keep the interaction between human and machine cooperative, not one or the other in charge at any given time, and you can still get the safety and reduced-stress benefits of semi-automated assisted driving with less of the inherent dangers caused by vigilance problem-related issues.

Level 2 semi-automated systems are flawed, and I think the IIHS made my point for me, so thanks.