Table of Contents

1. Problem Statement and Proposed Solution
2. NeedFinding
3. Prototyping
4. Sample Survey
5. Implementation
6. Evaluation
7. Timeline
8. Alternative Solution
9. Group Members

Problem Statement

Problem Statement

         Drowsiness is a safety hazard that is plaguing our country. In fact in 2013, the National Highway Traffic Safety Administration estimated that drowsy driving was responsible for 72,000 crashes, 44,000 injuries, and 800 deaths ^[1] . These numbers are underestimated and up to 6,000 fatal crashes may be caused by drowsy drivers. This is obviously a problem because sleep deprivation and drowsiness are costing lives, and currently there are no  clear-cut methods for police officers to identify drowsy drivers ^[2] . They currently use a criterium to determine if a driver is drowsy. By looking at the symptoms like the driver being alone, blood alcohol level being below the legal limit, no sign of brakes being applied, driving too close to the car in front of them, and more, police officers use those signs as indications that that the driver is sleepy. Notice that the items in this criterium do not include a factual number like one that would be provided in a breathalyzer test. Also, drowsiness is considered a genuine safety hazard in the workplace. For example, in the aviation industry, pilots are mandated to have a minimum of eight hours of rest in any 24-hour period of time ^[3] . However, simply giving them the opportunity to rest cannot guarantee that they are actually well rested to perform their high-risk job.  Because it cannot be guaranteed, detecting drowsiness is an important step in preserving the safety of the workers and the customers.

        Ideally we would love to sample from every working professional, especially those who have high risk jobs. However, because they are not easily available to us and with the time associated in capturing their information, we believe that students are an appropriate sample to draw from. Students are very time-crunched and sleep deprived, which can help us categorically identify drowsiness. And our hopes is to project that sample onto the working population.

Proposed Solution

         Our goal is to alert the user when he or she is feeling drowsy. This is in hopes that it will deter the individual from performing risky activities like driving, operating a forklift, etc. We foreshadow that this research may be of interest to truck drivers or businesses that take risk management very seriously, like the nuclear energy industry. They will most likely be the ones who can afford this pricey headset to begin with; however that is not to say that with over time, economies of scale will be leveraged and the headsets may become cheaper. Since safety is a universally recognized trait, we believe that the students of the University of Rochester will be able to provide opinions that are as insightful as individuals who are in the work force. The user would open a mobile application that we develop. Then the user would put on a headset , and we would monitor his or her brain waves and blinking patterns. We would allow about 10 seconds to pass from the time the user presses the record button before we actually start reading his or her brain waves. This is due in part because we believe that a user would be in a more excited state immediately after he or she turns on the headset; we want to allow a period of time to pass before we analyze his or her reading. Based on our research, we understand that when a person is drowsy, delta and gamma rays are most prominent, while alpha and beta waves decline ^[4] . By using Fast Fourier Transformation algorithm, we will be able to determine the frequency of each time period. Then we will plug those numbers into ratio   , as this ratio has been shown to be significantly higher when in a drowsy state ^[5] . If we determine that the user is drowsy, then we would alert the user through the mobile app that he or she should take caution when performing any activity that requires his or her attention.

Our hypothesis is that when the brain waves correspond to alpha or theta waves, we would then look at his or her blinking patterns. If the blinking pattern is slow and the user’s brain waves are similar to that of alpha or theta waves, t hen it would trigger a warning that he or she is in a drowsy state. Because alerting users it not a trivial task, we will use our needfinding survey as well as feedback from our prototype sessions to find methods that can best and most effectively alert the user. The above ratio has been scientifically proven and makes intuitive sense, but we also plan to reach out to a Brain and Cognitive Science (BCS) professor to verify our hypothesis.

Problem Statement Continued

This diagram depicts our solution at a high level. When the user puts on the headset, he or she will open up our in-house created application. Immediately after opening the app, when the user presses the “Start Recording” button, we will make a call to the headset via Bluetooth to read the current data from the user’s brain. Ideally, we want to keep the application running even in the background, so the user can use his or her phone while wanting to be monitored. If all signs are normal, then the app will simply continue reading passively.  Otherwise, if there are signs of drowsiness, then users will be alerted via screen notification, push notification, phone vibration, and alarm. Note that the colors and features are subject to change based on feedback from users, following user-centered design.

---

NeedFinding

What we are going to use:

        We are going to be interviews and surveys. For our interviews, we will be targeting both experts and fanatic s. For our expert interview, we plan on reaching out to either Professor Robert Jacobs or Professor Ralf Haefner from the BCS department to gather more scientific information, like information regarding the brain and how waves can be measured. One of our group members is taking courses with these professors, so there is already rapport built.

For our fanatic interviews, we would like to interview a public safety administrator and students. In regards to the public safety administrator, we want to reach out to Kenneth Grass, Public Safety Patrol Captain. We feel that since he is directly responsible for the public safety officers on the ground, he would be a good resource to gather the behaviors and attitudes of ground officers. He can give us more insight into what is needed to be successful for his line of work and the dangers behind sleep deprivation. We want to capture their information because they all perform risky behavior such having to be alert for an extended period of time to perform their job successfully or staying up late studying yet continuing to driving while being tired. Our goal from the interviews from the fanatics would be to see if there is an understanding of the consequences of drowsiness and whether they would engage in alternative, less-risky activities. For example, if students are very drowsy or not in a clear state of mind but need to go somewhere, we would want to know if we alert them, would they want to order an Uber or Lyft instead.

        In addition to interviews, our survey will be distributed to students through social media like Facebook. We believe that the students at the University of Rochester (UR) are very diverse, and the UR is known for having a lot of international representation. This means that they bring in many different experiences that constitute risky behavior from their homelands. We hope to gather more perspective and validate our hypothesis that there is a need for our tool. A sample of our survey is attached on the following page.

Sample Survey

Prototyping

Because reaching a state of drowsiness may take a while, the following prototyping will be for the mobile app only, and the evaluation of the headset with the application will be outlined in the Evaluation section. The intent is to streamline the process while the headset is still being built. Users will be given a cap to wear to mimic the headset and the tester playing the computer will react to the user vocalizing when they are drowsy.

We will start with low fidelity prototype and then gradually iterate into higher fidelity prototyping. For iteration 1 and iteration 2, the feedback from the participants will be logged via Google Form. We will be asking quantitative and qualitative questions regarding the topics in the evaluation column.

In the final iteration when users are testing the fully programmed Android App, we will take a more hands-off approach and have them run through the application. We will also perform a Google Form questionnaire.

The testing pool will be as follows:

• 3 Recurring Users ( longitudinal )

• These users will be testing the consistency of the designs throughout the semester
• They will be able to provide valuable suggestions regarding the functionalities and aesthetics because they will know the background of our research from the beginning of our test

• 3 Cross Sectional studies

• These users will be randomly chosen to participate in the iterations.
• Since they will not be familiar with our plan from day one, we expect them to provide feedback on design flaws that no one else would notice because they are too deeply invested the project.

Iteration	Type	Testing Environment	Evaluation
1	Paper Prototype Low fidelity with fantastic breadth (Testing Android App and headset)	One tester and one user. The tester will ask the user to perform a task.	Whether or not the process is easy to understand Which parts of the design well and what can be improved Any other suggestions on improving the application
2	Interactive Wireframe Higher fidelity with greater depth	One tester and one user. The tester will tell the user what task he or she should perform and will observe.	Design aesthetics Presentation of information
3	Coded Android App Finished product, highest fidelity with greatest depth	One tester and one user. The tester will silently observe the users with the functional application.	Overall performance Design aesthetics Interview questions about user experience

Implementation

Doze Alert seems at first like an overly ambitious project to try to implement and succeed. However if we break the development process down to three main steps it ends up being a realistic and attainable goal.

The three steps:

• Accurately and actively record EEG activity from the user in a non-invasive manner.
• Analyze the recorded data in real time to determine whether user is currently alert.
• If the user is not alert, let the user know in a friendly but serious way that the they are drowsy and the task they are performing may no longer be safe.

Recording EEG data in a comfortable and non-invasive manner is probably the most important step in the whole process. If the user needs to get tape/paste out and hook themselves up to Doze Alert for every single use almost no one would want to use it at all. To solve this problem we are planning on using the Ganglion Board  from OpenBCI. OpenBCI (Open Source Brain-Computer Interface) is a company that builds open source biosensing boards that can record EEG, EMG, and accelerometer activity. Along with developing the software necessary to interact and record data from the BCI, they also release the schematics for the headset that physically holds th e boards . With these schematics you can 3d print and assemble the headset yourself, which we are planning on doing. You must also buy the wiring and dry electrodes for the board’s inputs. The Ganglion board is OpenBCI’s cheapest option, with 4 active channels and an on-board accelerometer. It also comes with a Bluetooth module built in, allowing you to connect to the device to your phone/computer wirelessly.

We will be using the OpenBCI Python library to assist in processing the data. The library allows us to stream data in a useable format for analyzation and detection of the different waves occurring. It is not clear yet whether or not we will need to implement any machine learning algorithms for drowsiness prediction, it may be the case where simple algorithms will do the job. From our research so far ^[6]   we will be looking for occurrences of Beta waves (12.5 – 30 Hz) as these are associated with active thinking and wakefulness, Alpha Waves (8 – 12.5 Hz) which are associated with shut eyes and sleep, and for spikes that are triggered by eye blinks. We will also make use of the built in accelerometer so it can be determined if a person’s head is leaned forward or not, or if they’re making active movements (for noise reduction).

        Lastly for the front end we will be building a basic interface using Qt and Python. This will theoretically allow us to target as many platforms as possible (Linux, Android, iOS, etc…) .

Evaluation

        Testing Doze Alert on users poses a significant challenge. It will be simple to test whether or not the app can pick up on whether a person’s eyes are closed, but to actually test for drowsiness requires the participants to actually be drowsy. Therefore we will probably have to test users either early in the morning or late at night, and in the following interview ask if they were feeling tired before the experiment or not. We will walk them through a series of tasks, such as them closing their eyes and opening them, blinking, and trying to relax. We will then allow the users to experiment with the device themselves, so they can get a better feel for the device and its features. From the participation of the user we should be able to gather data on accuracy, delay, and make adjustments for a second iteration.

         After the user is finished we will then follow up with an interview.

Interview Questions (subject to change):

• How was the comfort of the device?
• Will you use Doze Alert while driving?

• Why/why not?

• Would you feel safer knowing the person driving you was wearing this device?
• Did you feel the device was effective at detecting drowsiness?
• Did you feel the device was effective at detecting closed eyes?
• What are some suggestions for improving the comfort of the device?
• What suggestions do you have for improving how the app alerts you?
• In your opinion, what features should be included in the app?
• In your opinion, what features should be removed from the app?
• In your opinion, was the design sleek enough?

Timeline

Date	Goal	Individual(s)
October 12th	1st Proposal	All
October 16th	Interviews Created	Dan
October 16th	Surveys Created	Josh
October 17th	Proposal Edited	All
October 18th	NeedFinding Started	Alex
October 18th	Data Collection on Brainwaves Started	Josh
October 18th	Paper Prototype Completed and Tested	Alex
October 20th	Data Analysis Started	Dan
October 21st	Data Collection Complete	Josh
October 21st	Wireframe Completed and Tested	Alex
October 21st	App Development Started	Josh and Dan
October 28th	Data Analysis Complete	Dan
October 29th	Website Started	Alex
November 2nd	Mid-semester Review of Final Project	All
November 15th	App Complete	Josh and Dan
November 15th	Website Complete	Alex
November 16th	Evaluation	All
November 17th	Presentation Complete	All
November 19th	Presentation Practice	All
November 25th	Video Complete	Josh
November 25th	Poster Complete	Dan
November 28th	Final Presentation	All
December 7th	Final Project Showcase	All

Alternative Solution

Admittedly, there are many possible setbacks to this project. Below are a few and how we plan to mitigate them.

Risk 1 - Efficiency of Testing Group

People may not be incentivized to stay with our longitudinal study.

Here is our plan on mitigating this risk. There is not much about this that we can think of doing. We could offer to buy them Starbucks for their time but as the semester goes on, students get busier. We hope that they do not drop out of our study.

Risk 2 - Project Completely Not Working

We do not have extensive background in brain wave modeling, so despite research from online resources, we cannot guarantee that this will be implemented the way that we picture it in our heads. This is probably our biggest risk. ^[b]  Fortunately, however there is a lot of information online that we have at our disposal. And there are a lot of open source information regarding OpenBCI that we can learn from as well. We know for certain that the boards we ordered from the company can detect brain waves, blink, and tilt, which are all we need for the project’s success.

However, we still want to make sure that we minimize the risk. We first will talk with a BCS professor to gain more insights and knowledge about the brain . Secondly, since the biggest technological risk comes from the headset and calling an API to OpenBCI, we would need to substitute this step out if the headset does not work as planned. We would eliminate the use of the headset and instead opt for vision as our source of data. With a camera in front of a person, we could detect facial features and muscles to make determinations. At that point, features of the application will need to be changed and updated. This aligns with what we are exposed to in lectures so it would be a safer alternative. Lastly, our timeline is a bit rushed but the reason for that is because we want to ensure that we have adequate time to pivot and make necessary changes.

Another alternative that was presented to us is to solely measure blinking patterns to detect drowsiness. Measuring eye blinks via a headset is possible and this could serve as an alternative. We could then measure blink rate as a determinant and alert the user.

Group Members

Picture	Name	Field(s) of Study	Role	Skills
	Alex Mai	Computer Science, Business	UI Design, Research, Development (if needed)	Java, basic Python, HTML, CSS, JavaScript, Swift
	Daniel Barnett	Computer Science	Research, Development, Design	Basic Neuro Computation, Python, Java
	Joshua Churchin	Computer Science	Research, Development, Design	Java, C, C++, HTML, CSS

Note that we have a three-person team, thus there will be quite an overlap between some responsibilities. We did, however, attempt to separate overall responsibilities via the timeline and deliverables page.

---