Clinical Systems Neuroscience

By Kenji Kansaku and Niels Birbaumer

The impaired brain has often been difficult to rehabilitate owing to limited knowledge of the brain system. Recently, advanced imaging techniques such as fMRI and MEG have allowed researchers to investigate spatiotemporal dynamics in the living human brain. Consequently, knowledge in systems neuroscience is now rapidly growing. Advanced techniques have found practical application by providing new prosthetics, such as brain–machine interfaces, expanding the range of activities of persons with disabilities, or the elderly. The book’s chapters are authored by researchers from various research fields such as systems neuroscience, rehabilitation, neurology, psychology and engineering. The book explores the latest advancements in neurorehabilitation, plasticity and brain–machine interfaces among others and constitutes a solid foundation for researchers who aim to contribute to the science of brain function disabilities and ultimately to the well-being of patients and the elderly worldwide.

• Language: English
• Publisher: Springer
• Release date: Jan 28, 2015
• ISBN: 9784431550372

Book preview

Part I

Brain–Machine Interfaces and Neurorehabilitation

© Springer Japan 2015

Kenji Kansaku, Leonardo G. Cohen and Niels Birbaumer (eds.)Clinical Systems Neuroscience10.1007/978-4-431-55037-2_1

1. Brain–Machine Interfaces in Stroke Neurorehabilitation

Surjo R. Soekadar¹, ²  , Stefano Silvoni³, Leonardo G. Cohen⁴ and Niels Birbaumer¹, ³, ⁵

(1)

Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Silcherstr. 5, 72076 Tübingen, Germany

(2)

Applied Neurotechnology Lab, Department of Psychiatry and Psychotherapy, University Hospital Tübingen, Calwerstr. 14, 72076 Tübingen, Germany

(3)

Ospedale San Camillo, IRCCS, Venice, Italy

(4)

Human Cortical Physiology and Neurorehabilitation Section (HCPS), NINDS, NIH, Building 10, Room 7D54, 20892 Bethesda, MD, USA

(5)

Institute of Diabetes Research and Metabolic Diseases of the Helmholtz Center Munich at the University of Tübingen, Otfried-Müller-Str. 10, 72076 Tübingen, Germany

Surjo R. Soekadar

Email: [e-mail]

Abstract

Stroke is one of the leading causes for severe adult long-term disability. The number of people who depend on assistance in their daily life activities has drastically increased over the last years and will further accumulate due to demographic factors. Besides impact on cognitive and affective brain function, motor paralysis is the heaviest burden of stroke. While recent studies demonstrated the human brain’s remarkable capacity to reorganize and restore function under effective learning conditions, most rehabilitation strategies require residual movements that, however, are lacking in up to 30–50 % of stroke survivors. For these patients, there is currently no standardized or accepted treatment strategy. Recently it was shown that brain–machine interfaces (BMI) translating electric or metabolic brain signals into control signals of computers or machines provide two strategies that play an increasing role for the recovery of these stroke survivors’ motor function: first, assistive BMIs striving for continuous high-dimensional brain control of robotic devices or functional electric stimulation (FES) to assist in performing daily life activities and, second, rehabilitative BMIs aiming at augmentation of neuroplasticity facilitating recovery of brain function. Recent demonstrations of such assistive and rehabilitative BMI system’s clinical applicability, safety, and efficacy suggest that BMIs will play a substantial role in rehabilitation strategies for severe motor paralysis after stroke.

Keywords

BMIBrain stimulationBrain–machine interfaceNeurorehabilitationStroke

1.1 Introduction

Stroke is a major cause for severe adult long-term disability [1, 2], leaving an increasing number of people dependent on assistance in their daily life activities [3]. While over the last years, effective treatment strategies were developed and clinically tested, e.g., constraint-induced movement therapy (CIMT) or robot-assisted therapy, these treatment strategies cannot be applied in many stroke survivors as they require residual movements not present in up to 30–50 % of the cases [4]. Currently, there is no accepted and standardized rehabilitation strategy for stroke patients with severe motor paralysis and no residual movements.

Driven by advances in neurotechnology, remarkable progress has been made over the last years towards Jacques Vidal’s original idea of direct brain–computer communication envisioning instant translation of brain events into control commands of external computers or machines [5], e.g., neuroprosthetic devices and robots. Such brain–computer or brain–machine interfaces (BCI/BMI) can utilize electric, magnetic, or metabolic brain signals recorded invasively from within the skull or noninvasively using sensors or electrodes placed over or close to the surface of the head to control, e.g., a robotic arm or exoskeleton, allowing to engage in daily life activities. Due to biophysical reasons, the versatility of control using invasive BMIs is usually higher compared to noninvasive BMIs, but the latter are often sufficient for applications that do not require high degree-of-freedom (DoF) control, e.g., simple grasping motions or basic communication.

1.2 Assistive and Rehabilitative BMI in Stroke Neurorehabilitation

Currently, there are two main strategies pursued to restore function after stroke using BMIs [6, 7]. These strategies are independent of the invasiveness of the approach and probably involve the same neural mechanisms for BMI learning and control, mainly operant conditioning [8] and feedback learning. The first strategy aims at bypassing nonfunctional corticospinal pathways to allow for continuous control of robotic devices [9] or functional electric stimulation (FES) of paralyzed muscles [10–14]. Such assistive BMIs have demonstrated versatile motor control in various daily life activities and created notable enthusiasm [9, 15]. The second strategy aims at facilitation of neuroplasticity and motor learning to enhance motor recovery (rehabilitative BMIs) [16, 17].

Thus far, six types of brain signals have been utilized to control noninvasive BMIs: (1) sensorimotor rhythms (SMRs, 8–15 Hz), [18–20], (2) slow cortical potentials (SCPs) [21], (3) event-related potentials (ERPs) [22], (4) steady-state visually or auditory evoked potentials (SSVEP/SSAEP) [23], (5) blood-oxygenation level dependent (BOLD)-contrast imaging using functional MRI [24], and (6) concentration changes of oxy-/deoxyhemoglobin using near-infrared spectroscopy (NIRS) [25, 26]. Invasive BMIs have successfully used local field potentials (LFPs) recorded from inside the cortex [27, 28] or on the surface [29–31] and action potentials (spikes) (e.g., [32–34]) for BMI control.

The first clinically relevant assistive BCI allowed individuals suffering from locked-in syndrome (LIS), a condition in which patients are awake and cognitively aware of their environment but unable to move or to speak, to select letters or words on a screen [21, 35]. For a long time, this was the only clinically relevant BMI application, as the impact of invasive and noninvasive assistive BMIs for restoration of movement was negligible.

Recent demonstrations, though, suggest that assistive BMI will become a realistic option to improve living conditions of individuals with severe paralysis once the associated costs and risks of these systems can be balanced with long-term patient benefits. After Hochberg et al. [15, 36] reported versatile control of a prosthetic limb by people with tetraplegia using intracortical spikes from a 96-channel electrode array placed over the motor cortex, implantation of two 96-channel electrode arrays allowed a tetraplegic woman to control skillful and coordinated reaching and grasping movements of a robotic arm [9]. Still, surgical implantation of electrodes or electrode arrays entails the risk of infection and hemorrhage that many stroke survivors may not be willing to accept. The availability of fully internalized systems requiring only a small burr hole, and thus reducing the perioperative risks, while offering efficient restoration of function, though, may change the risk–benefit ratio of invasive assistive BMIs. Additionally, noninvasive assistive BMI systems merging brain signals with other biosignals, e.g., electrooculograms (EOG) and electromyograms (EMG), termed brain/neural-computer interaction (BNCI) systems have recently provided some remarkable examples of restored motor function outside the lab.

These impressive demonstrations show that invasive and noninvasive assistive BMIs are a realistic option to improve living conditions of patients with paralysis. Once the associated costs and risks of these systems are balanced with long-term patient benefits, it is conceivable that a large portion of stroke survivors with severe paralysis (encompassing 30–50 % of all cases) will make use of assistive and/or rehabilitative BMIs in their daily life.

The theoretical concept of rehabilitative BMIs, also termed biofeedback or restorative BMI, is based on the early work of Barry Sterman [37] who showed that operant conditioning of SMR can reduce frequency of grand mal seizures in severe chronic epilepsy [38]. Later, controlled clinical studies supported the implication of this finding also for other neurological and psychiatric disorders, e.g., attention deficit hyperactivity disorder (ADHD) [39–41] or depression [42]. A case study suggested that learned regulation of ipsilesional SMR can be beneficial after stroke (e.g., [43]). In line with other studies showing that ipsilesional cortical function early after stroke can predict motor recovery [44, 45] and motivated by previous work performed by Basmajian [46, 47], Birbaumer and Cohen [48] developed the first SMR-based BMI for stroke survivors that allowed them to control an orthotic device attached to their paralyzed hand and fingers. By providing immediate sensory feedback contingent upon their ipsilesional brain activity [49], they hypothesized that reestablishing contingency between ipsilesional cortical activity related to planned or attempted execution of finger movements and proprioceptive (haptic) feedback, such BMI will strengthen the ipsilesional sensorimotor loop fostering neuroplasticity that facilitates motor recovery [16, 17, 48]. Other groups have used SMR-based BMI as a method to monitor and train motor imagery [50] previously shown to be beneficial for stroke recovery. The mechanism by which such BMIs impact motor function is less clear, but may link to the concept of bringing brain activity closer to normal [51].

An initial study indicated that the majority of chronic stroke patients can learn to control ipsilesional SMR [49], but a few weeks of training did not result in any significant motor function improvement or generalization of the skill into activities of daily living. However, BMI training coupled with goal-directed behavioral physical therapy over a longer period led to substantial improvements of motor and cognitive capacities of a severely affected chronic stroke survivor [52]. While before the training, the participant was unable to use his/her hand or arm for any relevant daily life activities, the ability to extent the fingers was completely restored after the training. Also, concentration and attentiveness improved significantly. A neuroimaging study indicated increased activation of the ipsilesional hemisphere and a small but significant increase of the fractional anisotropy in the ipsilesional corticospinal tract after the training [53]. Another study that applied combined BMI and FES of paralyzed finger muscles in a chronic stroke survivor reports restored individual finger extension after nine sessions [54, 55].

Based on these findings, a larger controlled clinical trial with 32 chronic stroke survivors without residual movements was conducted and showed that 20 sessions of ipsilesional BMI training combined with goal-directed behavioral physiotherapy led to motor improvements superior to those found in a control group that trained under random BMI feedback [56]. Assessment of the ipsilesional corticospinal tract’s integrity indicated that motor recovery was correlated with the presence of upper limb motor evoked potentials (MEPs) [57]. Likewise, assessment of the ascending sensory pathways’ integrity showed similar relevance for successful BMI control and learning [58]. A more recent clinical study in chronic stroke survivors with less severe paralysis comparing conventional robot-assisted therapy with BMI-controlled robotic training found similar results [59]. Other less controlled studies with smaller samples further corroborate this finding [60, 61].

These studies demonstrate that chronic stroke patients with severe motor deficits can regain motor function under effective learning conditions. In this context, BMI may represent an important and effective rehabilitation tool for stroke patients with severe paralysis for which currently no other therapy exists.

Based on the same principle, but rather for application in basic science than broad clinical application, real-time fMRI (rt-fMRI) neurofeedback has also been used to increase the activity of ipsilesional motor cortical areas in chronic stroke survivors [62]. Providing feedback of deep brain structures, for instance, dopaminergic midbrain regions, rt-fMRI allows studying the role of subcortical brain structures for recovery of function after stroke [63]. Multisite rt-fMRI feedback used to increase the connectivity between functionally associated brain regions may prove particularly useful [64].

While the majority of stroke patients were able to learn BMI control [49, 65], learning was often slower compared to healthy controls [19]. Thus, developing strategies enhancing BMI learning may further increase applicability of BMI in stroke neurorehabilitation. In this context, combination of BMIs with invasive and noninvasive brain stimulation proved particularly powerful [66].

1.3 Combining BMIs with Brain Stimulation

While it was known for centuries that electric currents applied to the brain could modulate mood, cognition, and behavior, the relevance of brain stimulation in both basic and clinical science has substantially increased in the last two decades. This may relate to the recent development of tools that allow systematic investigation of physiological mechanisms underlying brain stimulation effects [67], but it may also relate to the potential specificity and immediacy of the intervention compared to others, e.g., psychopharmacological approaches. Besides invasive stimulation techniques, such as deep brain stimulation (DBS) or motor cortex stimulation (MCS), noninvasive brain stimulation (NIBS) techniques, including transcranial magnetic stimulation (TMS) and transcranial direct current stimulation (tDCS), i.e., the application of weak electric direct currents (DC) of 1–2 mA through saline-soaked sponges or electrodes, are increasingly used [66]. For instance, it was shown that tDCS can improve learning and consolidation throughout different domains [68, 69]. When applied over the ipsilesional motor cortex of chronic stroke patients, reaction time and pinch force of the affected hand increased [70]. Similarly, repetitive TMS (rTMS) applied as facilitatory rTMS to the ipsilesional hemisphere [71] or inhibitory rTMS to the contralesional hemisphere [72] or their combination [73] influenced motor functions after stroke.

A recent study demonstrated that tDCS can enhance learning of SMR-based BMI control [20]. In this study, healthy participants learned regulation of SMR-based BMI immediately after receiving 20 min of anodal or cathodal tDCS over their primary motor cortex (M1). After 1 week of daily training, learning of SMR control was superior in those participants who received anodal tDCS compared to those who received cathodal or sham stimulation. The newly acquired skill remained superior in the group who received anodal tDCS, even 1 month after the end of training.

As several studies indicated that timing of stimulation relative to training can influence the stimulation effect [74, 75], development of new strategies allowing simultaneous or state-dependent stimulation promises to improve applicability and effectiveness of BMI training protocols. Recently, combination of simultaneous tDCS during EEG-based BMI control was successfully demonstrated [76]. While the use of EEG limits the possibility to reconstruct brain activity of regions immediately underneath the stimulation electrode, another strategy recently introduced by the same group allows for in vivo assessment of neuromagnetic brain oscillations in brain regions directly below the stimulation electrode [77]. Using this paradigm, Soekadar et al. [78] showed for the first time that a chronic stroke patient without residual finger movements can utilize SMR of the primary motor cortex’ (M1) hand knob to control an orthotic device to perform grasping motions, while this region, the ipsilesional M1, underwent anodal tDCS (Fig. 1.1). While this new strategy may lead to the refinement of existing stimulation protocols, it may further improve understanding of the relationship between brain physiology, cognition, and behavior, particularly in individuals with brain lesions or stroke.

A326253_1_En_1_Fig1_HTML.gif

Fig. 1.1

Illustration of a brain–machine interface (BMI) system for stroke neurorehabilitation training (adapted from Soekadar et al. 2015 [79]). Biosignals, here assessed by whole-head magnetoencephalography (MEG) associated with attempted movements of the paralyzed hand and fingers, are translated into online feedback delivered, e.g., by an orthotic device attached to the paralyzed hand and fingers and combined with simultaneous (brain-state-dependent) transcranial electric stimulation

1.4 Current Challenges of Clinical BMI in Stroke Neurorehabilitation

Almost all BMIs used for stroke neurorehabilitation have been noninvasive, so far. Yet having provided remarkable results, there are some challenges ahead that have to be mastered before broad and out-of-the-lab application of assistive and rehabilitative BMI will belong to the standard treatment options of severe motor paralysis. While low-frequency rhythms, such as SMR, showed limited correlation and contingency with intended movements [80–83], decoding of other brain signals, e.g., high gamma band and action potentials (single- or multiunit) [81–83], allowed for control of high degree-of-freedom prosthetic limbs [9, 36] or FES [12]. As it is hypothesized that Hebbian plasticity involving simultaneous activation of pre- and postsynaptic neurons is critical to driving functional connectivity in an optimal way, invasive BMIs or BMIs providing repetitive and precise contingency between efferent and afferent neural fibers’ activity may prove superior compared to previously used approaches [84]. Recently, it was demonstrated that high gamma band signals allow reconstruction of highly fractionated movements [28] most effectively obtained by invasive recordings using intracortical, subdural, or epidural electrodes [81, 82]. While intracranial electrodes require implantation, epidural or subdural electrodes could ultimately be implanted through a small burr hole reducing risk and cost of surgery.

A substantial barrier here is the lack of fully implantable, wireless intracranial devices. Once fully internalized systems are available, the risk–benefit ratio for paralyzed individuals may change significantly. Similarly, as noninvasive BMIs advance, these may, if providing comparable motor function similarly, change the risk–benefit ratio. Ultimately, the decision of a BMI implantation will be strongly influenced by individual patient circumstances.

In a recent study, the primary motor cortex of a rat was injured leading to a disruption of fibers between the motor and somatosensory areas [85]. An implanted neural prosthesis that translated action potentials in premotor cortex into contingent electrical stimulation in somatosensory cortex led to motor restoration of grasping movements that was indistinguishable from movements performed before the lesion.

Another study used ECoG signals and linked them to direct electric stimulation of a monkey’s spinal cord’s anterior horns. Besides reestablishing motor control of lower limb movements, this approach may prove useful for the rehabilitation of spinal cord injuries or subcortical stroke [86].

Although there is increasing clinical evidence for the efficacy of BMI-related tools in stroke neurorehabilitation, more and larger clinical studies are needed. Also, it is critical to further investigate the underlying mechanisms of BMI-induced functional recovery. A better understanding of the mechanisms underlying motor recovery could also lead to identification of specific and reliable biomarkers predicting treatment response (e.g., [57, 65]. The optimal dosage (frequency and intensity) of BMI training and/or brain stimulation applied in its context still needs to be investigated. Expansion of BMIs from hospital-based to home-based applications may further facilitate applicability of BMI in stroke neurorehabilitation and lower the threshold of including these devices in daily rehabilitation programs. In cases in which ipsilesional BMI training is not feasible, exploration of other strategies, e.g., training of contralesional, ipsilateral brain activity, may be effective [87, 88].

Another challenge is to identify and provide optimal frameworks for generalization of skills learned in the lab or hospital to the patient’s daily life activities. BMI systems following a hybrid assistive and rehabilitative approach may fill this gap and facilitate generalization of motor function reducing the necessity of behavioral physiotherapy. An important aspect when evaluating the efficacy of novel treatment strategies in severe stroke with no residual movements is the availability of reliable and valid instruments for assessment of motor function, as the established assessment scales (e.g., the Fugl-Meyer scale, Wolf test, NIH stroke scale, or ARAT) are rather insensitive to small but significant changes in upper limb movements. Kinematic measures such as sub-movement changes, joint flexibility, and proximal and distal limb movement accuracies [84, 89] should be taken into account as complementary measures to assess BMI-related functional motor recovery in severely affected stroke survivors.

Conclusions

BMIs are novel and powerful tools allowing stroke survivors with severe motor impairment to regain motor function. While larger clinical studies are needed to further investigate mechanisms underlying BMI-related stroke recovery and predictors of treatment response, BMI technology is evolving towards a substantial component of stroke neurorehabilitation. Combination of BMIs with invasive and noninvasive brain stimulation will further improve BMI applicability and promises to close an important knowledge gap linking brain physiology and recovery of brain function after stroke.

Acknowledgments

This work was supported by the Intramural Research Program (IRP) of the National Institute of Neurological Disorders and Stroke (NINDS), USA; the German Federal Ministry of Education and Research (BMBF, grant number 01GQ0831, 16SV5838K to SRS and NB); the BMBF to the German Center for Diabetes Research (DZD e.V., grand number 01GI0925), the Deutsche Forschungsgemeinschaft (DFG, grant number SO932-2 to SRS and Reinhart Koselleck support to NB); the European Commission under the project WAY (grant number 288551 to SRS and NB); and the Volkswagenstiftung (VW) and the Baden-Württemberg Stiftung, Germany. This chapter is an updated and shortened version of Brain–Computer Interfaces in the Rehabilitation of Stroke and Neurotrauma published in the first edition of Systems Neuroscience and Rehabilitation and the most recent review article on this topic [79] by the same authors.

References

1.

Feigin VL, Forouzanfar MH, Krishnamurthi R, Mensah GA, Connor M, Bennett DA, Moran AE, Sacco RL, Anderson L, Truelsen T, O’Donnell M, Venketasubramanian N, Barker-Collo S, Lawes CM, Wang W, Shinohara Y, Witt E, Ezzati M, Naghavi M, Murray C (2014) Global and regional burden of stroke during 1990–2010: findings from the Global Burden of Disease Study 2010. Global Burden of Diseases, Injuries, and Risk Factors Study 2010 (GBD 2010) and the GBD Stroke Experts Group. Lancet 383:245–254PubMedCentralPubMedCrossRef

2.

Gustavsson A, Svensson M, Jacobi F, Allgulander C, Alonso J, Beghi E, Dodel R, Ekman M, Faravelli C, Fratiglioni L, Gannon B, Jones DH, Jennum P, Jordanova A, Jönsson L, Karampampam K, Knappm M, Kobelt G, Kurthm T, Lieb R, Linde M, Ljungcrantz C, Maercker A, Melin B, Moscarelli M, Musayev A, Norwood F, Preisig M, Pugliatti M, Rehm J, Salvador-Carulla L, Schlehofer B, Simon R, Steinhausen HC, Stovner LJ, Vallat JM, Van den Bergh P, van Os J, Vos P, Xu W, Wittchen HU, Jönsson B, Olesen J, CDBE2010Study Group (2011) Cost of disorders of the brain in Europe 2010. Eur Neuropsychopharmacol 21:718–779PubMedCrossRef

3.

Birbeck GL, Hanna MG, Griggs RC (2014) Global opportunities and challenges for clinical neuroscience. JAMA 311:1609–1610PubMedCrossRef

4.

Kwakkel G, Kollen BJ, van der Grond J, Prevo AJ (2003) Probability of regaining dexterity in the flaccid upper limb: impact of severity of paresis and time since onset in acute stroke. Stroke 34:2181–2186PubMedCrossRef

5.

Vidal JJ (1973) Toward direct brain–computer communication. Annu Rev Biophys Bioeng 2:157–180PubMedCrossRef

6.

Birbaumer N, Murguialday AR, Cohen L (2008) Brain–computer interface in paralysis. Curr Opin Neurol 21:634–638PubMedCrossRef

7.

Silvoni S, Ramos-Murguialday A, Cavinato M, Volpato C, Cisotto G, Turolla A, Piccione F, Birbaumer N (2011) Brain–computer interface in stroke: a review of progress. Clin EEG Neurosci 42:245–252PubMedCrossRef

8.

Miller NE (1969) Learning of visceral and glandular responses. Science 163:434–445PubMedCrossRef

9.

Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, McMorland AJ, Velliste M, Boninger ML, Schwartz AB (2013) High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381:557–564PubMedCentralPubMedCrossRef

10.

Moritz CT, Perlmutter SI, Fetz EE (2008) Direct control of paralysed muscles by cortical neurons. Nature 456:639–642PubMedCentralPubMedCrossRef

11.

Pohlmeyer EA, Oby ER, Perreault EJ, Solla SA, Kilgore KL, Kirsch RF, Miller LE (2009) Toward the restoration of hand use to a paralyzed monkey: brain-controlled functional electrical stimulation of forearm muscles. PLoS One 4:e5924PubMedCentralPubMedCrossRef

12.

Ethier C, Oby ER, Bauman MJ, Miller LE (2012) Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485:368–371PubMedCentralPubMedCrossRef

13.

McGie SC, Zariffa J, Popovic MR, Nagai MK (2014) Short-term neuroplastic effects of brain-controlled and muscle-controlled electrical stimulation. Neuromodulation (in press). doi:10.​1111/​ner.​12185

14.

Pfurtscheller G, Müller GR, Pfurtscheller J, Gerner HJ, Rupp R (2003) ‘Thought’—control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia. Neurosci Lett 351:33–36PubMedCrossRef

15.

Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, Donoghue JP (2006) Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442:164–171PubMedCrossRef

16.

Dimyan MA, Cohen LG (2011) Neuroplasticity in the context of motor rehabilitation after stroke. Nat Rev Neurol 7:76–85PubMedCrossRef

17.

Dobkin BH (2007) Brain–computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation. J Physiol 579:637–642PubMedCentralPubMedCrossRef

18.

Pfurtscheller G, Brunner C, Schlögl A, Lopes da Silva FH (2006) Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks. NeuroImage 31:153–159PubMedCrossRef

19.

Soekadar SR, Witkowski M, Mellinger J, Ramos A, Birbaumer N, Cohen LG (2011) ERD-based online brain–machine interfaces (BMI) in the context of neurorehabilitation: optimizing BMI learning and performance. IEEE Trans Neural Syst Rehabil Eng 19:542–549PubMedCrossRef

20.

Soekadar SR, Witkowski M, Birbaumer N, Cohen LG (2014) Enhancing Hebbian learning to control brain oscillatory activity. Cereb Cortex Neuromodulation (in press). doi:10.​1093/​cercor/​bhu043

21.

Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, Kübler A, Perelmouter J, Taub E, Flor H (1999) A spelling device for the paralysed. Nature 398:297–298PubMedCrossRef

22.

Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70:510–523PubMedCrossRef

23.

Sakurada T, Kawase T, Takano K, Komatsu T, Kansaku K (2013) A BMI-based occupational therapy assist suit: asynchronous control by SSVEP. Front Neurosci 7:172PubMedCentralPubMedCrossRef

24.

Weiskopf N, Veit R, Erb M, Mathiak K, Grodd W, Goebel R, Birbaumer N (2003) Physiological self-regulation of regional brain activity using real-time functional magnetic resonance imaging (fMRI): methodology and exemplary data. NeuroImage 19:577–586PubMedCrossRef

25.

Sitaram R, Caria A, Birbaumer N (2009) Hemodynamic brain–computer interfaces for communication and rehabilitation. Neural Netw 22:1320–1328PubMedCrossRef

26.

Rea M, Rana M, Lugato N, Terekhin P, Gizzi L, Brötz D, Fallgatter A, Birbaumer N, Sitaram R, Caria A (2014) Lower limb movement preparation in chronic stroke: a pilot study toward an fNIRS-BCI for gait rehabilitation. Neurorehabil Neural Repair 28(6):564–575PubMedCrossRef

27.

Hwang EJ, Andersen RA (2009) Brain control of movement execution onset using local field potentials in posterior parietal cortex. J Neurosci 29:14363–14370PubMedCentralPubMedCrossRef

28.

Flint RD, Wright ZA, Scheid MR, Slutzky MW (2013) Long term, stable brain machine interface performance using local field potentials and multiunit spikes. J Neural Eng 10:056005PubMedCentralPubMedCrossRef

29.

Leuthardt EC, Schalk G, Wolpaw JR, Ojemann JG, Moran DW (2004) A brain–computer interface using electrocorticographic signals in humans. J Neural Eng 1:63–71PubMedCrossRef

30.

Schalk G, Brunner P, Gerhardt LA, Bischof H, Wolpaw JR (2008) Brain–computer interfaces (BCIs): detection instead of classification. J Neurosci Methods 167:51–62PubMedCrossRef

31.

Wang W, Collinger JL, Degenhart AD, Tyler-Kabara EC, Schwartz AB, Moran DW, Weber DJ, Wodlinger B, Vinjamuri RK, Ashmore RC et al (2013) An electrocorticographic brain interface in an individual with tetraplegia. PLoS One 8:e55344PubMedCentralPubMedCrossRef

32.

Taylor DM, Tillery SIH, Schwartz AB (2002) Direct cortical control of 3D neuroprosthetic devices. Science 296:1829–1832PubMedCrossRef

33.

Serruya MD, Hatsopoulos NG, Paninski L, Fellows MR, Donoghue JP (2002) Instant neural control of a movement signal. Nature 416:141–142PubMedCrossRef

34.

Carmena JM, Lebedev MA, Crist RE, O’Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, Nicolelis MAL (2003) Learning to control a brain–machine interface for reaching and grasping by primates. PLoS Biol 1:e42PubMedCentralPubMedCrossRef

35.

Birbaumer N, Gallegos-Ayala G, Wildgruber M, Silvoni S, Soekadar SR (2014) Direct brain control and communication in paralysis. Brain Topogr 27:4–11PubMedCrossRef

36.

Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, Haddadin S, Liu J, Cash SS, van der Smagt P, Donoghue JP (2012) Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485:372–375PubMedCentralPubMedCrossRef

37.

Sterman MB, Wyrwicka W, Howe R (1969) Behavioral and neurophysiological studies of the sensorimotor rhythm in the cat. Electroencephalogr Clin Neurophysiol 27:678–679PubMed

38.

Sterman MB, Macdonald LR (1978) Effects of central cortical EEG feedback training on incidence of poorly controlled seizures. Epilepsia 19:207–222PubMedCrossRef

39.

Lubar JF, Shouse MN (1976) EEG and behavioral changes in a hyperkinetic child concurrent with training of the sensorimotor rhythm (SMR): a preliminary report. Biofeedback Self Regul 1:293–306PubMedCrossRef

40.

Monastra VJ, Lynn S, Linden M, Lubar JF, Gruzelier J, LaVaque TJ (2005) Electroencephalographic biofeedback in the treatment of attention-deficit/hyperactivity disorder. Appl Psychophysiol Biofeedback 30:95–114PubMedCrossRef

41.

Strehl U, Leins U, Goth G, Klinger C, Hinterberger T, Birbaumer N (2006) Self-regulation of slow cortical potentials: a new treatment for children with attention-deficit/hyperactivity disorder. Pediatrics 118:e1530–e1540PubMedCrossRef

42.

Linden DE, Habes I, Johnston SJ, Linden S, Tatineni R, Subramanian L, Sorger B, Healy D, Goebel R (2012) Real-time self-regulation of emotion networks in patients with depression. PLoS One 7:e38115PubMedCentralPubMedCrossRef

43.

Rozelle GR, Budzynski TH (1995) Neurotherapy for stroke rehabilitation: a single case study. Biofeedback Self Regul 20:211–228PubMedCrossRef

44.

Platz T, Kim IH, Engel U, Kieselbach A, Mauritz K-H (2002) Brain activation pattern as assessed with multi-modal EEG analysis predict motor recovery among stroke patients with mild arm paresis who receive the arm ability training. Restor Neurol Neurosci 20:21–35PubMed

45.

Calautti C, Jones PS, Naccarato M, Sharma N, Day DJ, Bullmore ET, Warburton EA, Baron JC (2010) The relationship between motor deficit and primary motor cortex hemispheric activation balance after stroke: longitudinal fMRI study. J Neurol Neurosurg Psychiatry 81:788–792PubMedCrossRef

46.

Basmajian JV (1981) Biofeedback in rehabilitation: a review of principles and practices. Arch Phys Med Rehabil 62:469–475PubMed

47.

Basmajian JV, Gowland C, Brandstater ME, Swanson L, Trotter J (1982) EMG feedback treatment of upper limb in hemiplegic stroke patients: a pilot study. Arch Phys Med Rehabil 63:613–616PubMed

48.

Birbaumer N, Cohen LG (2007) Brain–computer interfaces: communication and restoration of movement in paralysis. J Physiol 579:621–636PubMedCentralPubMedCrossRef

49.

Buch E, Weber C, Cohen LG, Braun C, Dimyan MA, Ard T, Mellinger J, Caria A, Soekadar SR, Fourkas A, Birbaumer N (2008) Think to move: a neuromagnetic brain–computer interface (BCI) system for chronic stroke. Stroke 39:910–917PubMedCrossRef

50.

Prasad G, Herman P, Coyle D, McDonough S, Crosbie J (2010) Applying a brain–computer interface to support motor imagery practice in people with stroke for upper limb recovery: a feasibility study. J Neuroeng Rehabil 7:60PubMedCentralPubMedCrossRef

51.

Daly JJ, Wolpaw JR (2008) Brain–computer interfaces in neurological rehabilitation. Lancet Neurol 7:1032–1043PubMedCrossRef

52.

Broetz D, Braun C, Weber C, Soekadar SR, Caria A, Birbaumer N (2010) Combination of brain–computer interface training and goal-directed physical therapy in chronic stroke: a case report. Neurorehabil Neural Repair 24:674–679PubMedCrossRef

53.

Caria A, Weber C, Brötz D, Ramos A, Ticini LF, Gharabaghi A, Braun C, Birbaumer N (2011) Chronic stroke recovery after combined BCI training and physiotherapy: a case report. J Psychophysiol 48:578–582CrossRef

54.

Daly JJ, Cheng R, Rogers J, Litinas K, Hrovat K, Dohring M (2009) Feasibility of a new application of noninvasive brain computer interface (BCI): a case study of training for recovery of volitional motor control after stroke. J Neurol Phys Ther 33:203–211PubMedCrossRef

55.

Wang W, Collinger JL, Perez MA, Tyler-Kabara EC, Cohen LG, Birbaumer N, Brose SW, Schwartz AB, Boninger ML, Weber DJ (2010) Neural interface technology for rehabilitation: exploiting and promoting neuroplasticity. Phys Med Rehabil Clin N Am 21:157–178PubMedCentralPubMedCrossRef

56.

Ramos-Murguialday A, Broetz D, Rea M, Läer L, Yilmaz O, Brasil FL, Liberati G, Curado MR, Garcia-Cossio E, Vyziotis A, Cho W, Agostini M, Soares E, Soekadar SR, Caria A, Cohen LG, Birbaumer N (2013) Brain–machine interface in chronic stroke rehabilitation: a controlled study. Ann Neurol 74:100–108PubMedCentralPubMedCrossRef

57.

Brasil F, Curado MR, Witkowski M, Garcia E, Broetz D, Birbaumer N, Soekadar SR (2012) MEP predicts motor recovery in chronic stroke patients undergoing 4-weeks of daily physical therapy. Human brain mapping annual meeting, Beijing, June 10–14

58.

Shaikhouni A, Donoghue JP, Hochberg LR (2013) Somatosensory responses in a human motor cortex. J Neurophysiol 109:2192–2204PubMedCentralPubMedCrossRef

59.

Ang KK, Chua KS, Phua KS, Wang C, Chin ZY, Kuah CW, Low W, Guan C (2014) A randomized controlled trial of EEG-based motor imagery brain–computer interface robotic rehabilitation for stroke. Clin EEG Neurosci (in press)

60.

Várkuti B, Guan C, Pan Y, Phua KS, Ang KK, Kuah CW, Chua K, Ang BT, Birbaumer N, Sitaram R (2013) Resting state changes in functional connectivity correlate with movement recovery for BCI and robot-assisted upper-extremity training after stroke. Neurorehabil Neural Repair 27:53–62PubMedCrossRef

61.

Mukaino M, Ono T, Shindo K, Fujiwara T, Ota T, Kimura A, Liu M, Ushiba J (2014) Efficacy of brain–computer interface-driven neuromuscular electrical stimulation for chronic paresis after stroke. J Rehabil Med 46:378–382PubMedCrossRef

62.

Sitaram R, Veit R, Stevens B, Caria A, Gerloff C, Birbaumer N, Hummel F (2012) Acquired control of ventral premotor cortex activity by feedback training: an exploratory real-time FMRI and TMS study. Neurorehabil Neural Repair 26:256–265PubMedCrossRef

63.

Sulzer J, Sitaram R, Blefari ML, Kollias S, Birbaumer N, Stephan KE, Luft A, Gassert R (2013) Neurofeedback-mediated self-regulation of the dopaminergic midbrain. NeuroImage 83:817–825PubMedCrossRef

64.

Ruiz S, Buyukturkoglu K, Rana M, Birbaumer N, Sitaram R (2014) Real-time fMRI brain computer interfaces: self-regulation of single brain regions to networks. Biol Psychol 95:4–20PubMedCrossRef

65.

Buch ER, Modir Shanechi A, Fourkas AD, Weber C, Birbaumer N, Cohen LG (2012) Parietofrontal integrity determines neural modulation associated with grasping imagery after stroke. Brain 135:596–614PubMedCentralPubMedCrossRef

66.

Dayan E, Censor N, Buch ER, Sandrini M, Cohen LG (2013) Noninvasive brain stimulation: from physiology to network dynamics and back. Nat Neurosci 16:838–844PubMedCrossRef

67.

Bolwig TG (2014) Neuroimaging and electroconvulsive therapy: a review. J ECT 30:138–142PubMedCrossRef

68.

Reis J, Robertson E, Krakauer JW, Rothwell J, Marshall L, Gerloff C, Wassermann E, Pascual-Leone A, Hummel F, Celnik PA, Classen J, Floel A, Ziemann U, Paulus W, Siebner HR, Born J, Cohen LG (2008) Consensus: Can tDCS and TMS enhance motor learning and memory formation?. Brain Stimul 1:363–369PubMedCentralCrossRef

69.

Marshall L, Mölle M, Hallschmid M, Born J (2004) Transcranial direct current stimulation during sleep improves declarative memory. J Neurosci 24:9985–9992PubMedCrossRef

70.

Hummel FC, Voller B, Celnik P, Floel A, Giraux P, Gerloff C, Cohen LG (2006) Effects of brain polarization on reaction times and pinch force in chronic stroke. BMC Neurosci 7:73PubMedCentralPubMedCrossRef

71.

Khedr EM, Etraby AE, Hemeda M, Nasef AM, Razek AA (2010) Long-term effect of repetitive transcranial magnetic stimulation on motor function recovery after acute ischemic stroke. Acta Neurol Scand 121:30–37PubMedCrossRef

72.

Takeuchi N, Chuma T, Matsuo Y, Watanabe I, Ikoma K (2005) Repetitive transcranial magnetic stimulation of contralesional primary motor cortex improves hand function after stroke. Stroke 36:2681–2686PubMedCrossRef

73.

Sung WH, Wang CP, Chou CL, Chen YC, Chang YC, Tsai PY (2013) Efficacy of coupling inhibitory and facilitatory repetitive transcranial magnetic stimulation to enhance motor recovery in hemiplegic stroke patients. Stroke 44:1375–1382PubMedCrossRef

74.

Stagg CJ, Jayaram G, Pastor D, Kincses ZT, Matthews PM, Johansen-Berg H (2011) Polarity and timing-dependent effects of transcranial direct current stimulation in explicit motor learning. Neuropsychologia 49:800–804PubMedCentralPubMedCrossRef

75.

Volpato C, Cavinato M, Piccione F, Garzon M, Meneghello F, Birbaumer N (2013) Transcranial direct current stimulation (tDCS) of Broca’s area in chronic aphasia: a controlled outcome study. Behav Brain Res 247:211–216PubMedCrossRef

76.

Soekadar SR, Witkowski M, Garcia-Cossio E, Birbaumer N, Cohen LG (2014) Learned EEG-based brain self-regulation of motor-related oscillations during application of transcranial electric brain stimulation: feasibility and limitations. Front Behav Neurosci 8:93PubMedCentralPubMedCrossRef

77.

Soekadar SR, Witkowski M, Garcia-Cossio E, Birbaumer N, Robinson SE, Cohen LG (2013) In vivo assessment of human brain oscillations during application of transcranial electric currents. Nat Commun 4:2032PubMedCrossRef

78.

Soekadar SR, Witkowski M, Robinson SE, Birbaumer N (2013) Combining electric brain stimulation and source-based brain–machine interface (BMI) training in neurorehabilitation of chronic stroke. J Neurol Sci 333:e542CrossRef

79.

Soekadar SR, Birbaumer N, Slutzky MW, Cohen LG (2015) Brain-Machine Interfaces In Neurorehabilitation of Stroke. Neurobiology of Disease (in press)

80.

Schalk G, Kubánek J, Miller KJ, Anderson NR, Leuthardt EC, Ojemann JG, Limbrick D, Moran D, Gerhardt LA, Wolpaw JR (2007) Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J Neural Eng 4:264–275PubMedCrossRef

81.

Mehring C, Nawrot MP, de Oliveira SC, Vaadia E, Schulze-Bonhage A, Aertsen A, Ball T (2004) Comparing information about arm movement direction in single channels of local and epicortical field potentials from monkey and human motor cortex. J Physiol Paris 98:498–506PubMedCrossRef

82.

Stark E, Abeles M (2007) Predicting movement from multiunit activity. J Neurosci 27:8387–8394PubMedCrossRef

83.

Flint RD, Lindberg EW, Jordan LR, Miller LE, Slutzky MW (2012) Accurate decoding of reaching movements from field potentials in the absence of spikes. J Neural Eng 9:046006PubMedCentralPubMedCrossRef

84.

Silvoni S, Cavinato M, Volpato C, Cisotto G, Genna C, Agostini M, Turolla A, Ramos-Murguialday A, Piccione F (2013) Kinematic and neurophysiological consequences of an assisted-force-feedback brain–machine interface training: a case study. Front Neurol 4:173PubMedCentralPubMedCrossRef

85.

Guggenmos DJ, Azin M, Barbay S, Mahnken JD, Dunham C, Mohseni P, Nudo RJ (2013) Restoration of function after brain damage using a neural prosthesis. PNAS 110:21177–21182PubMedCentralPubMedCrossRef

86.

Nishimura Y, Perlmutter SI, Fetz EE (2013) Restoration of upper limb movement via artificial corticospinal and musculospinal connections in a monkey with spinal cord injury. Front Neural Circuits 7:57PubMedCentralPubMedCrossRef

87.

Bundy DT, Wronkiewicz M, Sharma M, Moran DW, Corbetta M, Leuthardt EC (2012) Using ipsilateral motor signals in the unaffected cerebral hemisphere as a signal platform for brain–computer interfaces in hemiplegic stroke survivors. J Neural Eng 9:036011PubMedCentralPubMedCrossRef

88.

Carmena JM (2013) Advances in neuroprosthetic learning and control. PLoS Biol 11:e1001561PubMedCentralPubMedCrossRef

89.

Dipietro L, Krebsa HI, Fasoli SE, Volpe BT, Hogan N (2009) Submovement changes characterize generalization of motor recovery after stroke. Cortex 45:318–324PubMedCrossRef

© Springer Japan 2015

Kenji Kansaku, Leonardo G. Cohen and Niels Birbaumer (eds.)Clinical Systems Neuroscience10.1007/978-4-431-55037-2_2

2. Practical Noninvasive Brain–Machine Interface System for Communication and Control

Kenji Kansaku¹  

(1)

Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with Disabilities, 4-1 Namiki, Tokorozawa Saitama, 359-8555, Japan

Kenji Kansaku

Email: [e-mail]

Abstract

The brain–machine interface (BMI) or brain–computer interface (BCI) is an interface technology that utilizes neurophysiological signals from the brain to control external machines or computers. We have developed electroencephalography (EEG)-based BMI systems to help persons with physical disabilities. We first applied the P300 paradigm for environmental control and communication. We attempted to optimize the visual stimuli for our P300-BMI and prepared a green/blue flicker matrix. We showed that the new matrix was associated with a better subjective feeling of comfort than was the conventional white/gray flicker matrix and that the new matrix was associated with better performance. We further proposed an advanced system by adding augmented reality (AR) in which an agent robot was applied as a moving remote controller.

For clinical application, we created an in-house environmental control system comprising in-house hardware (e.g., an EEG amplifier) and software, in which the P300, steady-state visual evoked potential (SSVEP), and sensorimotor rhythm were set for use. We also developed peripheral devices including a nonadhesive solid gel EEG electrode and a soft cap with electrode holders. The P300-based environmental control system was operated successfully by patients with cervical spinal cord injury and amyotrophic lateral sclerosis.

We also developed in-house robotic exoskeletons to support the arm and finger movements of paralyzed patients, and the SSVEP paradigm was used for their asynchronous control. Moreover, we developed a real-time magnetoencephalography system, aiming to further develop new BMI and neurofeedback technologies. Research along these lines may help persons with disabilities to expand their range of activities.

Keywords

BMIEnvironmental controlMotor assistanceNeurofeedbackNeurorehabilitation

2.1 Introduction

The brain–machine interface (BMI) or brain–computer interface (BCI) is an interface technology that utilizes neurophysiological signals from the brain to control external machines or computers [1, 2]. One approach used in BMI research involves neurophysiological signals directly from neurons or the cortical surface. This approach is categorized as invasive BMI because it requires neurosurgery [3, 4]. Another approach utilizes neurophysiological signals from the brain accessed nonsurgically, termed noninvasive BMI. Electroencephalography (EEG), a technique for recording neurophysiological signals using electrodes placed on the scalp, constitutes the primary noninvasive methodology for studying BMIs.

Several approaches have been proposed in EEG-based noninvasive BMI, such as slow cortical potentials [5], sensorimotor rhythm [6, 7], and steady-state visually evoked potentials (SSVEP) [8, 9]. One popular EEG-based BMI system, the P300 speller, uses elicited P300 responses to target stimuli placed among row and column flashes [10].

Our research group first applied the P300 paradigm for environmental control and communication. We attempted to optimize the visual stimuli for our P300-BMI and prepared a green/blue flicker matrix. For practical application, we created an in-house environmental control system comprising in-house hardware and software, in which the P300, SSVEP, and sensorimotor rhythm were set for use. Further, to support arm and finger movements of paralyzed patients, we developed in-house robotic exoskeletons, and the SSVEP paradigm was used for their asynchronous control. This chapter introduces a series of our studies on the BMI and discusses how these new technologies can contribute to expansion of the range of activities of those with disabilities.

2.2 BMI for Environmental Control and Communication

2.2.1 P300-BMI-Based Environmental Control System

Our research group used EEG signals to develop a BMI system that enables environmental control and communication (Fig. 2.1). We first modified the so-called P300 speller [10], which uses the P300 paradigm and presents a selection of icons arranged in a white/gray flicker matrix. With this protocol, the participant focuses on one icon in the matrix as the target, and each row/column or single icon of the matrix is then intensified in a random sequence. The target stimuli are presented as rare stimuli (i.e., the oddball paradigm). We elicited P300 responses to the target stimuli and then extracted and classified these responses with regard to the target.

A326253_1_En_2_Fig1_HTML.gif

Fig. 2.1

Diagram of a BMI system for environmental control and communication (modified from Kansaku [11])

We constructed a prototype of the BMI-based environmental control system. For the visual stimuli, we first prepared four panels with white/gray flicker matrices, one each for the desk light, primitive agent robot, television control, and Japanese alphabet (hiragana) spelling. We tested the system on both quadriplegic and able-bodied participants and reported that the system could be operated effortlessly [12].

One specific merit of the P300 speller algorithms is that we can easily translate the subject’s thoughts as a command preassigned to each icon. Therefore, our system can be used for various applications in environmental control and communication.

2.2.2 Effects of Visual Stimuli for P300-BMI

The white/gray flicker matrix has been used as a visual stimulus for the P300-BMI, but the white/gray flash stimuli might induce discomfort, particularly in subjects with a history of epilepsy. Parra et al. evaluated the safety of chromatic combinations for those with photosensitive epilepsy [13]. Five single-color stimuli (white, blue, red, yellow, and green) and four alternating-color stimuli (blue/red, red/green, green/blue, and blue/yellow with equal luminance) of four frequencies (10, 15, 20, and 30 Hz) were used as the visual stimuli. Under white stimulation, flickering stimuli with higher frequencies, especially those greater than 20 Hz, are potentially provocative. Under the alternating-color stimulation condition, as suggested by the Pokemon incidence [14], the 15-Hz blue/red flicker was the most provocative. Notably, the green/blue chromatic flicker emerged as the safest and evoked the lowest rates of EEG spikes. Accordingly, we investigated the effectiveness of green/blue flicker matrices as visual stimuli.

We evaluated the subjective feeling of comfort when using the green/blue chromatic flicker matrices (n = 9; 8 able-bodied and 1 spinal cord injury). Two panels with matrices (3 × 3 and 8 × 10) were intensified with two-color combinations (green/blue and white/gray); these were used for desk-light control and hiragana spelling. We used a visual analogue scale (range: 0–100 %) to evaluate the subjective feeling of comfort. The measured value for the green/blue (white/gray) flicker matrices was 79.3 % (49.0 %) in the desk-light control condition and 58.0 % (34.5 %) in the hiragana spelling condition. Significant differences between the two-color combinations were observed for each condition (p < 0.05) [15].

We further recruited able-bodied, untrained subjects to perform hiragana spelling using an 8 × 10 matrix with three types of intensification/rest flicker combinations (L, luminance; C, chromatic; LC, luminance and chromatic); both online and offline performances were evaluated (n = 10). The accuracy rate under the online LC condition was 80.6 %. Offline analysis showed that the LC condition was associated with significantly higher accuracy than was the L or C condition (p < 0.05) (Fig. 2.2). No significant difference was observed between the L and C conditions. The LC condition, which used the green/blue flicker matrix, was associated with better performances in the P300-BMI [16]. We showed that the green/blue flicker matrices were associated not only with a better subjective feeling of comfort but also with better performance.

A326253_1_En_2_Fig2_HTML.gif

Fig. 2.2

Green/blue flicker matrix provided better performance in P300-BMI operation (modified from Takano et al. [16]). Mean performance curves at each condition for all ten sequences are shown. Mean performances in luminance (L), chromatic (C), and luminance and chromatic (LC) conditions are plotted by the broken line with black circles, the dotted line with gray triangles, and the solid line with white squares, respectively

To increase accuracy when operating the BMI system and to develop better classification methods [17–20], some studies have attempted to identify better, more efficient experimental settings by manipulating factors such as the matrix size and duration of intensification [21], channel set of the EEG [22], and flash pattern of the flicker matrix [23]. We proposed a method that combines luminance and chromatic information to increase the accuracy of performance using the P300-BMI, and this method can be applied along with those proposed in the aforementioned reports.

2.2.3 Neuronal Substrates Underlying the P300-BMI

To identify brain areas that were more enhanced in the green/blue flicker matrix than in the white/gray flicker matrix, we applied simultaneous EEG–functional magnetic resonance imaging (fMRI) recordings (n = 12). We did this because such recordings may highlight areas devoted to improved P300-BMI performance. The peak of the positive wave in the EEG data was detected under both conditions, and the peak amplitudes were larger at the parietal and occipital electrodes (particularly in the late components) under the green/blue condition than under the white/gray condition. fMRI data showed activation in the bilateral parietal and occipital cortices, and these areas, particularly those in the right hemisphere, were more activated under the green/blue condition than under the white/gray condition. The parietal and occipital regions more involved in the green/blue condition were part of the areas devoted to conventional P300s. These results suggest that the green/blue flicker matrix was useful for enhancing the so-called P300 responses [24].

We then applied magnetoencephalography (MEG) and aimed to map functional connectivities during the use of the P300-BMI using the two types of visual stimuli: green/blue and white/gray flicker matrices (n = 8). We used alphabet flickers (6 × 6) as the two types of visual stimuli. Whole-head 306-channel MEG data (Neuromag, Elekta) were collected during the experiment. The gradiometer data were segmented, and current sources were estimated with a narrowband scalar minimum-variance spatial filter [25, 26]. Anatomical MR images of each subject were used for coregistration and normalization. The mean imaginary coherence was computed in the alpha band (8–12 Hz) [27]. Significantly greater coherence was observed in the right posterior parietal cortex in the green/blue condition than in the white/gray condition. The larger functional connectivity observed in the green/blue condition may play a significant role in driving the P300-BMI [28].

2.2.4 Effects of Visual Stimuli for SSVEP

The SSVEP is another main approach in noninvasive BMIs; however, relatively low-frequency flickers have been used to elicit SSVEP signals, so these flickers are usually visible. If the frequencies used exceed the critical flicker frequency (CFF), these high-frequency flickers can be invisible. We aimed to prepare a new SSVEP-based BMI system using flickers with a higher frequency than the CFF. We prepared flickering devices with green and blue LEDs, and each color LED flickered alternately to elicit the SSVEP. We first evaluated the CFF in the participants (n = 6), and the mean CFF was 55.7 Hz (range: 51.2–60.3). We then evaluated the SSVEP amplitude from Oz while the participants fixated on an LED flicker with different frequencies of 30–70 Hz. The mean luminance was 500 cd/m². We showed that the SSVEP was effectively induced by adjusting the luminance of green and blue color stimuli even if the frequency was