Prepared by: Zhuo Li Dept. of Electrical and Computer Engineering Utah State University

1. ECE5320 MechatronicsAssignment#01: Literature Survey on Sensors and Actuators Topic: Brain Wave Sensor Prepared by: Zhuo Li Dept. of Electrical and Computer Engineering Utah State University 3/9/2012

2. Outline • Reference list • To probe further • Major applications • Working principle • Typical setup • Major specifications • Advantages • Disadvantages • Products on-shelf • Outlook ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

3. Reference list • [1]. A. S. Hellera, Tom Johnstonec, A. J. Shackmana, S. N. Lighta, M. J. Petersond, G. G. Koldenb, N. H. Kalinb, and R. J. Davidsona, Reduced capacity to sustain positive emotion in major depression reflects diminished maintenance of fronto-striatal brain activation, PNAS, December 29, 2009, vol. 106, no. 52, 22445–22450. • [2]. Yasui Y, A brainwave signal measurement and data processing technique for daily life applications, NeuroSky White Paper, [online], http://www.ncbi.nlm.nih.gov/pubmed/19483376. • [3]. Hamalainen, M., Riitta, H., Ilmoniemi, R., Knuutila, J., and Lounasmaa, O. (1993). Magnetoencephalography—theory, instrumentation, and applications to noninvasive studies of the working human brain. Reviews of Modern Physics, 65(2), 414-497. • [4]. O'Regan, S., Faul, S., and Marnane, W. (2010). Automatic detection of EEG artefacts arising from head movements. Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE, Buenos Aires, Argentina, August 31 - September 4, 2010, 6353-6356. • [5]. Fisher, R., Webber, W., Lesser, R., Arroyo, S., and Uematsu, S. (1992). High-frequency EEG activity at the start of seizures. J Clin Neurophysiol, 9(3): 441-8. • [6]. Pfurtscheller G, Lopes da Silva FH (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol, 110(11): 1842–1857. • [7]. Niedermeyer E. and da Silva F.L. (2004). Electroencephalography: Basic Principles, Clinical Applications, and Related Fields. Lippincot Williams & Wilkins. ISBN 0781751268. • [8]. Yasuno et al (2008). The PET Radioligand [11C]MePPEP Binds Reversibly and with High Specific Signal to Cannabinoid CB1 Receptors in Nonhuman Primate Brain. Neuropsychopharmacology, 33, 259-269. • [9]. Schlögl, A., Slater, M., & Pfurtscheller, G. (2002). Presence research and EEG Properties of EEG recordings. Proceedings of the 5th Annual International Workshop PRESENCE. Porto, Portugal, October 9-11. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

4. To explore further • [i]. Droid’s sensor Blog, A Headset Sensor That Reads Your Brainwaves, http://droidsensors.com/2010/07/a-headset-sensor-that-reads-your-brainwaves/ • [ii]. TED Talk, A Headset Sensor That Reads Your Brainwaves, http://www.ted.com/talks/view/lang/en//id/921 • [iii]. Lab For Affective Neuroscience, Univ. of Wisconsin Madison, http://psyphz.psych.wisc.edu/web/research.html • [iv]. EEG Used to Convict Murder Suspect http://medgadget.com/2008/09/eeg_used_to_convict_murder_suspect.html • [v]. Iyad Obeid, Temple Univ, TEDxPhilly, Mapping the brain, http://www.youtube.com/watch?v=HV-k7EwZVNQ • [vi]. NeuroSky, http://www.neurosky.com/Default.aspx • [vii]. Mind Flex, Duel, http://mindflexgames.com/ • [viii]. Uncle Milton, Star Wars Science, http://unclemilton.com/star_wars_science/ • [ix]. National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov/guide/ • [x]. Brain Waves & Brainwave entrainment, http://www.sleepwarrior.com/brain-waves-brainwave-entrainment ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

5. Major applications • Ordinary life • Smart home • Medical • Assistant devices for handicapped • Clinical examination • Depression, sleep disorder analysis [1] [iii] • Academic research • Exploring the brain • Entertainment • Game console • Security • Convict criminal suspect by Brain Electrical Oscillations Signature test (BEOS) [iii] Fig5. [x] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

6. Introduction • What are brain waves? • “All brain cells communicate via electrical signals. One’s brain contains about 100 billion neurons, well over the number of stars in our galaxy. Each neuron is connected to about 10,000 other neurons, making for about 100 to 500 trillion neuron-to-neuron connections in the brain.” [x] • “Our current technology cannot see what’s happening at the neuron level. They’re just too small, and there’s just too many of them. However, our technology cansee overall patterns of brain activity.” [x] • Neurons firing • Brainwave Entrainment • Also called brainwave synchronization. It purportedly depends upon a "frequency following" response on the assumption that the human brain has a tendency to change its dominant EEG frequency towards the frequency of a dominant external stimulus. [Wikipedia.org] • One study showed that when two people are in the same room, their brainwaves and heartbeats become entrained — that is, their brainwaves become matched up in frequency. [x] • Here is a video of TED talk by Keith Barry demonstrating Brainwave Entrainment http://www.ted.com/talks/keith_barry_does_brain_magic.html ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

7. Introduction • Methods to study brain function • Functional magnetic resonance imaging (fMRI) • Nuclear magnetic resonance spectroscopy (核磁共振) • Electroencephalography (EEG) • Magnetoencephalography • Electrocorticography (ECG) • Brain-computer interface (BMI) often called a mind-machine interface (MMI), is a direct communication pathway between the brain and an external device. BCIs are often directed at assisting, augmenting, or repairing human cognitive or sensory-motor functions. [Wikipedia.rog] • Brain wave sensor Brain wave sensor is the sensor to capture the signals generated by brain activities. Data acquisition device implementing EEG measurement is nothing but a net of voltmeters. Nowadays, the brain wave sensor refers more on the data fusion and processing technology than sensing itself. Most products are turnkey (whole set) involving sensing with applications. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

8. Working principle • Electroencephalography (EEG) An electroencephalogram is a test that measures and records the electrical activity of one’s brain. [Wikipedia.org] • Source of EEG activity The brain's electrical charge is maintained by billions of neurons. Neurons are electrically charged (or "polarized") by membrane transport proteins that pump ions across their membranes. Neurons are constantly exchanging ions with the extracellular milieu, for example to maintain resting potential and to propagate action potentials. Ions of like charge repel each other, and when many ions are pushed out of many neurons at the same time, they can push their neighbors, who push their neighbors, and so on, in a wave. This process is known as volume conduction. When the wave of ions reaches the electrodes on the scalp, they can push or pull electrons on the metal on the electrodes. The difference in push or voltage between any two electrodes can be measured by a voltmeter. Recording these voltages over time gives us the EEG. [Wikipedia.org] Fig7. An EEG recording net. [iii] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

9. Working principle • EEG Wave patterns [Wikipedia.org] • Delta is the frequency range up to 4 Hz. It tends to be the highest in amplitude and the slowest waves. • Theta is the frequency range from 4 Hz to 7 Hz. Theta is seen normally in young children. It may be seen in drowsiness or arousal in older children and adults; it can also be seen in meditation. • Alpha is the frequency range from 8 Hz to 12 Hz. Hans Berger named the first rhythmic EEG activity he saw as the "alpha wave". • Beta is the frequency range from 12 Hz to about 30 Hz. It is seen usually on both sides in symmetrical distribution and is most evident frontally. Beta activity is closely linked to motor behavior and is generally attenuated during active movements. [6] • Gamma is the frequency range approximately 30–100 Hz. Gamma rhythms are thought to represent binding of different populations of neurons together into a network for the purpose of carrying out a certain cognitive or motor function. [7] • Mu ranges 8–13 Hz., and partly overlaps with other frequencies. It reflects the synchronous firing of motor neurons in rest state. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

10. Working principle • EEG Wave patterns Delta waves Sensorimotor rhythm aka mu rhythm. Theta waves Beta waves Alpha waves Gamma waves Fig8. Brain wave patters. [Wikipedia.org] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

11. Working principle • Brainwaves, Not Thoughts • “NeuroSky ThinkGear technology digitizes analog electrical brainwaves to power the user-interface of games, education and investigative applications. A monumental hurdle lies in distinguishing brain signal from the noise that comes from ambient electricity, muscle movement, etc. Such interferences are digitally filtered and eliminated. Raw brain signals are amplified and processed —delivering concise input to the device. Algorithms come from both NeuroSky as well as research institutions and universities, and currently include “attention”, “meditation,” and physical eyeblinks. NeuroSky technology accurately measures brainwaves today, but is busy in labs throughout the world advancing development in new areas of emotional EEG, Electromyogram (EMG), Electro-Oculogram (EOG) and Electrocardiogram (ECG). For a deeper scientific description of our technology.” [vi] • “The NeuroSky MindSet headset is our first manifestation available for researchers. As a turnkey EEG data acquisition tool, it measures raw, power band and algorithm output for use with BCI2000, MATLAB and proprietary applications. New developer kits for capacitive sensors, multi-channel sensors and Steady State Visually Evoked Potentials (SSVEP) are all currently available to preferred partners. For more information on our developer hardware kits.” [vi] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

12. Typical setup • Please refer to the video in [ii] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

13. Advantages • EEG based brain wave sensor • EEG sensors can be used in more places than fMRI, SPECT, PET, MRS, or MEG, as these techniques require bulky and immobile equipment. For example, MEG requires equipment consisting of liquid helium-cooled detectors that can be used only in magnetically shielded rooms, altogether costing upwards of several million dollars[3]; and fMRI requires the use of a 1-ton magnet in, again, shielded room. • EEG has higher temporal resolution - milliseconds, rather than seconds - it can, in fact, take as many as 2000 samples per second [5] Only MEG rivals these speeds.[3] • EEG is relatively tolerant of subject movement, unlike all other neuroimaging techniques. There even exist methods for minimizing, and even eliminating movement artefacts in EEG data [4] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

14. Disadvantages • EEG based brain wave sensor • Significantly lower spatial resolution compared to other measuring approaches. • EEG determines neural activity that occurs below the upper layers of the brain (the cortex) very poorly • Unlike PET and MRS, cannot identify specific locations in the brain at which various neurotransmitters, drugs, etc. can be found. [8] • Often takes a long time to connect a subject to EEG, as it requires precise placement of dozens of electrodes around the head and the use of various gels, saline solutions, and/or pastes to keep them in place. While the length of time differs dependent on the specific EEG device used, as a general rule it takes considerably less time to prepare a subject for MEG, fMRI, MRS, and SPECT. • Signal-to-noise ratio is very poor, so sophisticated data analysis and relatively large numbers of subjects are needed to extract useful information from EEG. [9] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

15. On-shelf Products • NeuroSky [vi] • MindSet Headset • MindWave Headset w/ Education Bundle • MindBand • ThinkGear AM • Brain Athlete Fig13-1. NeuroSkyThinkGear AM. [vi] Fig13-2. NeuroSky Brain Athlete. [vi] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

16. On-shelf Products NeuroSky MindSet Overview [vi] The MindSet was the first mass market EEG headset when it debuted in 2009.  The device consists of earphone, a microphone, and a sensor arm.  The headset’s reference and ground electrodes are on the ear phones and the EEG electrode is on the end of the sensor arm, sitting on the forehead above the eye. Not only can it collect research-grade EEG data, but it features a set of high-quality Bluetooth enabled wireless headphones complete with microphone. The MindSet is versatile, compact, and portable. It is an excellent choice for gaming, research, or public installations.  Measures: •   Raw-Brainwaves  •   EEG power spectrums (Alpha, Beta, etc.) •   eSense meter for Attention and Meditation Physical: •   190g weight •   Length 21cm X width 17.5cm X height 5.0cm Signal and EEG: •   Maximum signal input range: 1mV pk-pk •   Hardware filter range: 3Hz to 100Hz •   MS001: includes 60Hz environmental AC noise filtering •   MS002: includes 50Hz environmental AC noise filtering •   Amplification gain: 2000x  •   ADC resolution: 12 bits (-2048 to 2047)   •   Sampling rate: 512Hz   •   eSense interpretation update rate: 1Hz  •   0% byte loss (ie packet loss)  •   1Hz eSense interpretation rate •   UART Baudrate: 57,600 Baud  •   SPP through put: 9600 Baud  •   S/N ratio: >70dB  •   Class 2 Bluetooth Radio Range: 10 m 6dBm RF max power •   250kbit/s RF data rate Fig. NeuroSkyMindSet Headset ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

17. On-shelf Products • Mind Flex Duel [vii] Brainwave entertainment/toy Fig15. Mind Flex Duel. [vii] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

18. On-shelf Products • Uncle Milton Star Wars Science [viii] Brainwave entertainment/toy Fig16. Uncle Milton Star Wars Science [viii] ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators

19. Outlook • Since brainwaves are weak electric signals, contactless monitoring is difficult. Hence, contactless brainwave sensor appears technically hard but appealing. ECE5320 Mechatronics. Assignment#1 Survey on sensors and actuators