Flashing Lights Can Also Cause Cars To Have ‘Digital Epileptic Seizures’

If you’ve never seen someone have an epileptic seizure then you’ve missed out on one of life’s scary adventures. I distinctly remember sitting in my middle school auditorium during a presentation by a muscle-themed Christian group when a classmate in front of me started seizing. It was terrifying. He was eventually ok and we all learned about the dangers of flashing lights for certain individuals.

Did you know cars can be similarly impacted by flashing lights? Researchers from Ben-Gurion University of the Negev and Japanese tech company Fujitsu found that cars with advanced driver assistance systems (ADAS) like Tesla’s Autopilot or Ford’s BlueCruise can experience “digital epileptic seizures” due to the flashing lights on emergency vehicles.

The researchers were specifically interested in the “16 documented incidents involving Teslas (with autopilot engaged) crashing into parked emergency vehicles (police cars, ambulances, and firetrucks)” and have dubbed this phenomenon “Epilepticar.”

The activation of emergency vehicle lighting creates a phenomenon, which we term the EpileptiCar phenomenon, that causes the confidence score of object detectors regarding a detected object (in our case it is a car) to fluctuate, within a wide score range, with the score dipping below a reasonable detection threshold in some cases.

You can see this happening in video of a flashing police light on a Ford Fiesta parked in the dark:

And this is what it looks like on a graph:

Here are the researchers explaining exactly how the system is set up:

Five ADASs (HP, Pelsee, AZDOME, Imagebon, Rexing) and a smartphone camera (Samsung Galaxy S22 Ultra) were used individually to record a 60- second video recording (the FPS and resolution of the ADAS footage can be found in Table 1, and the Samsung Galaxy footage was recorded at 24 FPS at FHD resolution) of a grey Ford Fiesta equipped with a blue emergency vehicle lighting(purchased on Amazon4).

The experimental setup is presented in Fig. 2. In the first 30 seconds of each recording, the car’s emergency vehicle lighting was off, and in the last 30 seconds, it was on (see Fig. 1). Each recorded video was segmented into frames, and the four object detectors were applied to each frame. The outputs were used to generate confidence score signals, a time series of the detector confidence score as a function of time/frame, for each ADAS/smartphone and object detector regarding car detection.

The most obvious issue here is that cars using hands-free driving systems could get confused by the lights. A human being seeing lights immediately knows what it is, but a computer doesn’t reason or infer in exactly the same way a human does.

“The EpileptiCar phenomenon highlights the important distinction that the ability of autonomous systems to implicitly handle these situations is still far from the level of a human,” said David Doria, director of engineering and automated driving at Magna in response to this research in an Automotive News article on the topic.

What’s more interesting, to me, is that researchers say that used by someone hoping to attack a driverless car. You can imagine someone on the road, approaching a car employing an ADAS system, and then flashing a bunch of random lights at it in order to confuse the vehicle into stopping or slowing down. This could potentially work because, as we know, people often don’t pay attention when using these systems.

Is there a solution here?

The researchers suggest something they call “Caracetamol,” which is a portmanteau involving “car” and the headline drug “paracetamol.” Who says researchers aren’t funny?

It’s basically an identifying protocol of sorts that says “Hey, this is a police car or similar.”

Caracetamol is a software framework designed to enhance the resilience of object detectors against the EpileptiCar phenomenon. It utilizes a multi-layer architecture, including:

1) a classifier to identify the presence of flashing lights

2) a layer of refined object detectors trained on augmented data containing emergency vehicles with activated flashers.

3) a layer that combines the outputs of these refined detectors with the original object detector.

The researchers say they shared their findings with the National Highway Traffic Safety Administration, the ADAS analyzed, and Tesla.  This just reiterates the concept that computers don’t think like humans and edge cases are everywhere.