Background

From 2014 to 2016, more than ten thousand road accidents have occurred in India due to drivers falling asleep behind the wheel, with more than eleven thousand people suffering injuries in the process. The trucking industry in India is dealing with the problem of increasing driver shortages— causing existing truck drivers to clock in abnormally high work hours of more than twelve hours. A survey has observed that truck drivers tend to take small naps of ten to fifteen seconds when their driving hours extend beyond ten hours. A similar scenario can be observed in the public road transport sector. For example, sleep deprivation is a significant problem faced by the Uttar Pradesh State Road Transport Corporation, with bus drivers getting less than four hours of sleep while covering over seven hundred kilometres. While accident preventive campaigns have encouraged drivers to stop their vehicles if they are on the verge of falling asleep, such self-awareness might not be present in slumberous and exhausted human beings.

Solution

We have proposed a wearable device that will continuously monitor for instances of drivers being drowsy. If such a scenario occurs, alert them and/or trigger some safety mechanisms in the vehicle. Unlike conventional computer vision-based sleep detection systems installed in some smart cars, this wearable device employs electrooculography (EOG) and will:

• Detect the onset of drowsiness, which occurs before the driver falls asleep.
• Be independent of ambient conditions (like less illumination during nighttime).
• Have a minimal, lightweight, uni-sized and aesthetically pleasing design.
• Have a simple installation and maintenance procedure.
• Be an economical, standalone portable system.

Moreover, the applications of this wearable could also be extended to private vehicle drivers, crane operators and control room operators in power plants and other industries. Due to its active data logging, this device could also be used by insurance companies to analyze the role of driver drowsiness behind possible accidents and subsequent insurance filings.

Principle

Our eyes exhibit various kinds of movements; some of them are generally performed in combination with other bodily actions. Hence, the measurement of these eye movements could be utilized to detect the corresponding action. They could be either used to gauge the activity and cognition of an individual or to perform some tasks whenever that specific action is performed. Common voluntary eye movements include rightward and upward gaze, while involuntary eye movements include blinks, the sudden closure of eyes due to brightness, etc. One such eye movement includes blinks of durations more than 750 ms. These are termed prolonged blinks (PBs) and are characteristic of a person who is drowsy. The duration of these blinks, along with their inter-arrival time (the time interval between two successive prolonged blinks), could be used to determine if a person is on the verge of falling asleep.

Since ocular muscles control eye movements, the biopotential generated by their action may be contemplated for eye movement detection. Electrooculography (EOG) is a technique for measuring such potentials that exists between the ocular muscles: EOG can be used to measure eye movements and, therefore, PBs as well. Specifically, detecting about two to three PBs occurring within a 10-15 s interval is enough to conclude that the person is drowsy.

Implementation

For picking up the EOG signals from the subject, we are using non-invasive electrodes. These are economical, pre-gelled, self-adhesive, reduce artefacts, and are simple in application. Before designing the wearable, we conducted a study to determine the optimal electrode position for detecting PBs. As per the results obtained, we decided to go with combinations 3-6-8 and 5-4-8 for electrode placement. Here, positions 3, 4, 5 and 6 would be measured, while position 8 would be the reference. Provisions were made to include all these positions in the final version of the wearable.

The EOG signals picked up by the electrode are relayed to a signal conditioning circuit. The EOG potential that appears on the skin is only about ±5-20 μV. It is also often buried under more prominent physiological signals such as EMG and EEG. Hence, the first step was to amplify this signal to ±3V. Additionally, the EOG frequency ranges from 0.1 Hz to 40 Hz. Since the frequency range of EOG is relatively short, it is easily distinguishable from other physiological signals. Using simple bandpass filters and notch filters, we restrict the frequency such that only the desired signal would be passed through. The filtered signal is then scaled to 0-3V before being sent to the microcontroller.

However, the detection of PBs is still not straightforward due to two reasons:

1. While driving, the eyes can exhibit multiple movements. For example, looking at the rearview mirrors, gauging the instrument panel, etc. EOG will pick up all these eye movements and not just PBs. The challenge here is identifying the unique features in the EOG signal corresponding to PBs and tuning the algorithm to scan for the same continuously.
2. Being a physiological signal, EOGs can vary from person to person and even be different for the same person; e.g., the magnitude of these signals can be weaker when the person is not feeling well. With these constraints, a static algorithm cannot be expected to detect PBs in all users with high accuracy. Incorporating popular Artificial Intelligence (AI) algorithms would go a long way in resolving this issue. However, our wearable is prioritized to be an affordable and compact unit. Therefore, the onboard microcontroller will require highly optimized Machine Learning (ML) algorithms to detect PBs accurately and in real-time.

With these considerations in mind, we designed an efficient ML algorithm from scratch. It initially pre-processes the EOG signals received from the signal conditioning circuit. This is done through windowed mean-filtering and subsequently, obtaining the first-order derivative (FOD) of the filtered signal to account for baseline wandering. The resultant signal is then used for training the algorithm and subsequently detecting drowsiness. The entire procedure is executed on a TIVA TM4C123G microcontroller and takes place in real-time on a per-sample basis.

For training, a user is expected to generate ten instances each of upward gazes and PBs. During this period, the algorithm identifies the unique features of the EOG waves generated by that subject and then applies it for detection during the operating period. During operation, an EOG waveform is analyzed for the presence of four states and six prominent features. Instead of a binary classification for each feature, fuzzy logic is employed to give a holistic answer on how close the candidate EOG waveform is to a PB. This is beneficial to account for the variability of PBs even from the same user.

Considering extreme driving schedules, the system is needed to be running continuously for up to 15 hours at a stretch. Since the entire unit consumes about 100 mA on average, the ideal choice is a 2000 mAh Li-ion or Li-po battery. The onboard power supply facilitates the safe charging & discharging of this battery. Using voltage converters, steady power rails of ±3.3V are provided for use by the microcontroller and the alerting and signal conditioning circuitry.

Results

Due to the COVID-19 pandemic, our team faced significant difficulties testing the wearable since it requires physical contact with the subject. As an alternative, we developed a novel simulation technique to simulate EOG signals with controlled variability. These signals have been modelled using EOG signals generated by a human volunteer “A” and were calculated to have a 99% correlation with the real-world counterparts.

Initially, the simulator trains the algorithm by generating instances of PBs and upward gazes as demanded. Then, during the operating period, the simulator generates EOG signals corresponding to various eye movements randomly. The algorithm is expected to use the knowledge base it formulated during the training period to detect all the occurrences of simulated PBs.

The aforementioned test was conducted on two separate occurrences and the following results were obtained:

Additional Resources

1. Paper published on arXiv: A. Pillay, A. Kale, R. Anchan, A. Bhadricha, and S. P. Ram, ‘Real-Time Detection of Drowsiness Among Vehicle Drivers: A Machine Learning Algorithm for Embedded Systems’, arXiv:2111.03177 [eess], Nov. 2021.

2. Paper published on IEEE Xplore: R. Anchan, A. Pillay, A. Kale, A. Bhadricha and S. P. Ram, “Optimal Bipolar Lead Placement in Electrooculography (EOG): A Comparative Study with an Emphasis on Prolonged Blinks,” 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2020, pp. 1-7, doi: 10.1109/ICCCNT49239.2020.9225609.

3. Detailed Project Report

4. Technical video presented for the 2019 IICDC Semifinals: