Artificial Intelligence-Based Brain-Computer Interface

Artificial Intelligence-Based Brain Computer Interface provides concepts of AI for the modeling of non-invasive modalities of medical signals such as EEG, MRI and FMRI. These modalities and their AI-based analysis are employed in BCI and related applications. The book emphasizes the real challenges in non-invasive input due to the complex nature of the human brain and for a variety of applications for analysis, classification and identification of different mental states. Each chapter starts with a description of a non-invasive input example and the need and motivation of the associated AI methods, along with discussions to connect the technology through BCI.

Major topics include different AI methods/techniques such as Deep Neural Networks and Machine Learning algorithms for different non-invasive modalities such as EEG, MRI, FMRI for improving the diagnosis and prognosis of numerous disorders of the nervous system, cardiovascular system, musculoskeletal system, respiratory system and various organs of the body. The book also covers applications of AI in the management of chronic conditions, databases, and in the delivery of health services.

• Provides readers with an understanding of key applications of Artificial Intelligence to Brain-Computer Interface for acquisition and modelling of non-invasive biomedical signal and image modalities for various conditions and disorders
• Integrates recent advancements of Artificial Intelligence to the evaluation of large amounts of clinical data for the early detection of disorders such as Epilepsy, Alcoholism, Sleep Apnea, motor-imagery tasks classification, and others
• Includes illustrative examples on how Artificial Intelligence can be applied to the Brain-Computer Interface, including a wide range of case studies in predicting and classification of neurological disorders

• Language: English
• Publisher: Academic Press
• Release date: Feb 4, 2022
• ISBN: 9780323914123

Book preview

9780323914123_FC

Artificial Intelligence-Based Brain-Computer Interface

First Edition

Varun Bajaj

PDPM-Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India

G.R. Sinha

Myanmar Institute of Information Technology (MIIT), Mandalay, Myanmar

Table of Contents

Cover

Title page

Copyright

Contributors

1: Multiclass sleep stage classification using artificial intelligence based time-frequency distribution and CNN

Abstract

1.1: Introduction

1.2: Materials and methods

1.3: Results

1.4: Discussion

1.5: Conclusions

References

2: A comprehensive review of the movement imaginary brain-computer interface methods: Challenges and future directions

Abstract

2.1: Introduction

2.2: PRISMA guideline

2.3: Results

2.4: Discussion

2.5: Conclusion and future scope

References

3: A new approach to feature extraction in MI-based BCI systems

Abstract

3.1: Introduction

3.2: Types and applications

3.3: BSS and its application in BCI

3.4: Related work

3.5: Proposed method

3.6: Computer simulation and result

3.7: Discussion

3.8: Conclusion

References

4: Evaluation of power spectral and machine learning techniques for the development of subject-specific BCI

Abstract

4.1: Introduction

4.2: Materials

4.3: Methods

4.4: Performance verification

4.5: Parameters selection

4.6: Results

4.7: Discussions

4.8: Conclusion

Conflicts of interest

References

5: Concept of AI for acquisition and modeling of noninvasive modalities for BCI

Abstract

5.1: Introduction

5.2: Electroencephalogram

5.3: Artificial intelligence for signal analysis

5.4: Communication interface between brain and machine

5.5: Methodology

5.6: Results

5.7: Discussion and future scope

5.8: Conclusion

References

6: Bi-LSTM-deep CNN for schizophrenia detection using MSST-spectral images of EEG signals

Abstract

6.1: Introduction

6.2: Methods and materials

6.3: Results and discussion

6.4: Conclusions and recommendations

References

7: Detection of epileptic seizure disorder using EEG signals

Abstract

7.1: Introduction

7.2: Background on EEG signals

7.3: Machine learning approaches for epilepsy

7.4: Deep learning approaches for epilepsy

7.5: Related work and comparative analysis

7.6: Conclusion

References

8: Customized deep learning algorithm for drowsiness detection using single-channel EEG signal

Abstract

8.1: Introduction

8.2: Drowsiness detection (DD)

8.3: Database

8.4: Proposed methodology

8.5: Experimental results

8.6: Conclusion

References

9: EEG-based deep learning neural net for apnea detection

Abstract

9.1: Introduction

9.2: Database

9.3: Proposed methodology

9.4: Experimental results

9.5: Conclusion

References

10: Classification of mental states from rational dilation wavelet transform and bagged tree classifier using EEG signals

Abstract

10.1: Introduction

10.2: Methodology

10.3: Results and discussions

10.4: Conclusions

References

11: A novel metaheuristic optimization method for robust spatial filter designation and classification of speech imagery tasks in EEG Brain-Computer Interface

Abstract

Acknowledgments

11.1: Introduction

11.2: Experiments

11.3: Robust spatial filter designation in multisession, multitrial EEG data

11.4: Adaptive network-based fuzzy inference system

11.5: Experimental results

11.6: Conclusion and future work

References

12: Variational mode decomposition-based finger flexion detection using ECoG signals

Abstract

12.1: Introduction

12.2: Variational mode decomposition

12.3: Features studied

12.4: Support vector machine

12.5: Methodology

12.6: Proposed method

12.7: Results and discussion

12.8: Conclusion

References

13: An insight into the hardware and software aspects of a BCI system with focus on ultra-low power bulk driven OTA and Gm-C based filter design, and a detailed review of the recent AI/ML techniques

Abstract

Acknowledgments

13.1: Introduction

13.2: Literature survey of the BCI system

13.3: Review of the signal preprocessing, feature extraction and selection, and classification techniques for a BCI system

13.4: Hardware analysis of BCI focusing on the design of low power OTA and its Gm-C based filter for EEG signal processing

13.5: Conclusion

References

14: Deep autoencoder-based automated brain tumor detection from MRI data

Abstract

14.1: Introduction

14.2: Methodology, dataset, and techniques

14.3: Classification

14.4: Performance metrics

14.5: Experimental studies

14.6: Conclusion

References

15: Measure the superior functionality of machine intelligence in brain tumor disease prediction

Abstract

Acknowledgments

15.1: Introduction

15.2: Related work

15.3: Materials and methods

15.4: Results and discussion

15.5: Conclusion

References

Index

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

Copyright © 2022 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN 978-0-323-91197-9

For information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Image 1

Publisher: Mara Conner

Acquisitions Editor: Chris Katsaropoulos

Editorial Project Manager: Charlotte Rowley

Production Project Manager: Kamesh Ramajogi

Cover Designer: Matthew Limbert

Typeset by STRAIVE, India

Contributors

Jaya Prakash Allam

DMIMS (DU), Wardha, Maharashtra

Department of ECE, NIT Rourkela, Rourkela, India

Mohammad Reza Aslani     Electrical Engineering Department, Shahab Danesh University, Qom, Iran

Varun Bajaj     PDPM-Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India

Chinmaya Behara     DMIMS (DU), Wardha, Maharashtra, India

Charmi Daftari     Department of Information and Communication Technology, LJ Institute of Engineering and Technology, Ahmedabad, Gujarat, India

Fatih Demir     Biomedical Department, Vocational School of Technical Sciences, Firat University, Elazig, Turkey

Morteza Farahi     Department of Medical Physics and Biomedical Engineering, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran

Anil Kumar Gautam     Department of ECE, North Eastern Regional Institute of Science and Technology, Nirjuli, Arunachal Pradesh, India

Zahra Ghanbari     Department of Biomedical Engineering, Amirkabir University of Technology, Tehran, Iran

Komal Jindal     Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, India

Sadaf Khademi     Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran

Smith K. Khare     PDPM-Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India

Ketan Kishor Kurkute     DMIMS (DU), Wardha, Maharashtra, India

Govinda Rao Locharla     Department of ECE, GMR Institute of Technology, Rajam, India

Luca Longo     Applied Intelligence Research Centre, School of Computer Science, Technological University Dublin, Dublin, Ireland

Hamid Reza Marateb

Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran

Biomedical Engineering Research Centre (CREB), Automatic Control Department (ESAII), Polytechnic University of Catalonia (UPC), Barcelona, Spain

Mohammad Hassan Moradi     Department of Biomedical Engineering, Amirkabir University of Technology, Tehran, Iran

Mehrnoosh Neghabi     Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran

Neelamshobha Nirala     Department of Biomedical Engineering, National Institute of Technology Raipur, Raipur, Chhattisgarh, India

Arefeh Nouri     Department of Biomedical Engineering, Amirkabir University of Technology, Tehran, Iran

Prabin Kumar Padhy     Electronics and Communication Engineering Discipline, Indian Institute of Information Technology Design and Manufacturing Jabalpur, Jabalpur, Madhya Pradesh, India

Saurabh Pal     Department of MCA, VBS Purvanchal University, Jaunpur, India

Revathi Pogiri     Department of ECE, SVCET, Etcherla, India

Ateeq Ur Rehman     Department of Electrical Engineering, Government College University, Lahore, Pakistan

Muhammad Tariq Sadiq     Department of Electrical Engineering, The University of Lahore, Lahore, Pakistan

Saunak Samantray     DMIMS (DU), Wardha, Maharashtra, India

Abdulkadir Sengur     Electrical-Electronic Engineering Department, Firat University, Elazig, Turkey

Jainish Shah     Department of Information and Communication Technology, LJ Institute of Engineering and Technology, Ahmedabad, Gujarat, India

Manan Shah     Department of Chemical Engineering, School of Technology, Pandit Deendayal Petroleum University, Gandhinagar, Gujarat, India

Tripurari Sharan     Department of ECE, North Eastern Regional Institute of Science and Technology, Nirjuli, Arunachal Pradesh, India

Rishi Raj Sharma     Department of Electronics Engineering, Defence Institute of Advanced Technology, Pune, Maharashtra, India

Shivam Sharma     Department of Electronics Engineering, Defence Institute of Advanced Technology, Pune, Maharashtra, India

Mehdi Shirzadi     Biomedical Engineering Research Centre (CREB), Automatic Control Department (ESAII), Polytechnic University of Catalonia (UPC), Barcelona, Spain

Resham Raj Shivwanshi     Department of Biomedical Engineering, National Institute of Technology Raipur, Raipur, Chhattisgarh, India

Nameirakpam Premjit Singh     Department of ECE, North Eastern Regional Institute of Science and Technology, Nirjuli, Arunachal Pradesh, India

G.R. Sinha     Myanmar Institute of Information Technology (MIIT), Mandalay, Myanmar

Vikas Kumar Sinha     DMIMS (DU), Wardha, Maharashtra, India

Siuly Siuly     Institute for Sustainable Industries & Liveable Cities, Victoria University, Melbourne, VIC, Australia

Sachin Taran     Delhi Technological University (DTU), New Delhi, India

Rahul Upadhyay     Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, India

Dhyan Chandra Yadav     VBS Purvanchal University, Jaunpur, India

1: Multiclass sleep stage classification using artificial intelligence based time-frequency distribution and CNN

Smith K. Kharea; Varun Bajaja; Sachin Taranb; G.R. Sinhac    a PDPM-Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India

b Delhi Technological University (DTU), New Delhi, India

c Myanmar Institute of Information Technology (MIIT), Mandalay, Myanmar

Abstract

Background: The conventional sleep stage scoring uses interview, questionnaire, and visual inspection by trained neurologists. These methodologies are time taking, inefficient, and error-prone. Thus, need of an automatic classification of sleep stages is felt that can provide accurate and efficient diagnosis of various neuropsychological diseases.

Method: In this chapter, convolutional neural network (CNN) and the utility of time-frequency distribution (TFD) are utilized in identification of sleep stages automatically using electroencephalogram (EEG) signals. The short-time Fourier transform, continuous wavelet transforms, Zhao-Atlas-Marks distribution, and smoothed pseudo Wigner-Ville distribution are used for transforming the time-domain EEG signals into TFD. The images of TFD are subjected to an adaptive convolutional neural network (A-CNN). The efficacy of the proposed A-CNN is evaluated with the help of comparison of the proposed method with three pretrained AlexNet, VGG-16, and ResNet50.

Results: Effectiveness of the proposed method is assessed by six performance evaluation parameters, namely accuracy, sensitivity, specificity, Mathew's correlation coefficient, F-1 score, and Cohen's Kappa that were obtained during the experimentation. An accuracy of 98.83%, specificity of 97.75%, sensitivity of 99.24%, Cohen's Kappa of 0.989, and F-1 score of 0.987 are obtained using A-CNN. In addition, the method was found outperforming in comparison with the existing state-of-the-art approaches.

Conclusion: The proposed method enables automatic feature extraction and classification methods from single channel EEG signals. The results and comparison report prove that the idea of TFD and CNN is justified. This evolutionary technique can be used to model a portable real-time sleep stage identification system.

Keywords

Sleep stage classification; Artificial intelligence; Zhao-Atlas-Marks distribution; Convolutional neural network; Electroencephalogram; Time-frequency distribution

1.1: Introduction

Sleep is an integral part of human life, and adequate sleep is required for carrying healthy brain activities. Researchers have recommended an average sleep of 8 h from teenage (11 years) to older adults (65 years) [1]. Enhancement in technology and digitization has resulted in disturbance of sleeping hours. Moreover, hectic lifestyle and social behavior have drastically changed the normal sleeping patterns to lesser or higher sleeping hours. The disturbed sleeping hours have been the primary reason and symptom of neurophysiological and psychological disorders [2]. Thus, sleep disorder identification becomes necessary so that automated diagnosis can be achieved. Generally, as per the neuro specialists, the whole sleep is divided into rapid eye movement (REM) and nonrapid eye movement (NREM) [3]. The NREM is further divided into four stages, namely S1, S2, S3, and S4. The sleep stages S1 and S2 are grouped to constitute a shallow sleep (SHS) state, whereas the sleep stages S3 and S4 form a slow-wave sleep (SWS) [4,5]. The Rechtschaffen and Kales (R & K) and American Academy of Sleep Medicine (AASM) are seen as most common standards for obtaining scores of sleep stages [6,7]. The R & K standard annotations are used to determine the sleep epochs in anyone of six stages: REM, S1, S2, S3, S4, and Awake (AS) state, and AASM standard annotation depict the epochs, namely REM, SWS, S1, S2, and AS. Trained psychologists use manual scoring methods and questionnaire for the identification of sleep disorders [8]. Techniques are time-consuming, burdensome, and prone to error. This creates a gap in the automatic identification of sleep stages. Electroencephalography is one such source that can be employed for an automatic sleep stage identification. Electroencephalogram (EEG) signals are characterized by noninvasive, portable, high temporal resolution, low cost, and nonradioactive solution.

Many researchers have proposed sleep stage classification using EEG signals. Peker in Ref. [9] extracted correlation dimension, approximate entropy, Hurst exponent, largest Lyapunov exponent, Higuchi, Hjorth parameters, and Petrosian fractal dimension EEG signals as useful features for the purpose of analysis and classification. These nonlinear features are categorized with the help of construction of a complex-valued neural network (CVNN). In Ref. [10], Zhang et al. utilized the EEG signal features extracted from raw EEG signals, and the features were categorized using fast discriminative CVNN. The model in this work was combined with complex-valued backpropagation and the Fisher criterion in order to extract, select, and classify the features. In Ref. [11], a number of nonlinear as well as linear features were extracted from EEG signals and utilized in the classification by Fisher discriminant analysis method. The methods discussed in Refs. [9–11] employ the extraction of features as direct extraction way from EEG signals and thus probably could not produce appropriate distinguishable features. Doroshenkov et al. in Ref. [12] utilized fast Fourier transform (FFT) method for obtaining multiple rhythms from EEG signals. The alpha, beta, delta, theta, and gamma rhythms are classified using hidden Markov model (HMM). Hassan et al. in Refs. [13,14] used discrete Fourier transform (DFT) to obtain frequency bands of EEG signals. Multiple statistical and spectral features elicited from the frequency bands have been classified using bagging and adaptive boost (Adaboost) bagging classifiers. Huang et al. have used short-time Fourier transform (STFT) to separate the EEG signal into rhythms in Ref. [15]. The power spectral density (PSD) of each rhythm has been obtained and classified with a relevance vector machine (RVM) classifier. In Ref. [16], Welch method, correlation, and high pass filtering were used for the extraction of features. Features selected with forward and best subset methods have been classified with quadratic discriminant analysis. The methods in Refs. [12–16] create a trade-off between time and frequency domain. Moreover, the proper selection of windows and its length is required while employing these methods. Diykh et al. in Refs. [17,18] used the segmentation method for partitioning the EEG signals. The segmented portion has been used to extract various statistical features. These statistical features have been classified with k-means clustering, support vector machine (SVM), and structural graph similarity techniques. The segmentation methods in Refs. [17,18] used empirical sample size for partitioning due to which representative portion of the original signal is ignored.

In Ref. [19], Berthomier et al. used the band pass filter (BPF) technique to obtain the rhythms. Several spectral features elicited from the rhythms have been classified with fuzzy logic classifier. Liang et al. in Ref. [20] used BPF with multiscale entropy (MSE) and autoregressive (AR) methods for feature extraction. These features have been classified with a linear discriminant analysis (LDA) classifier. Hsu et al. in Ref. [21] evaluated energy of the rhythms obtained by BPF. These energy features have been classified with Elman recurrent neural classifier (ERNC). In Ref. [22], Seifpour et al. used BPF for the segregation of EEG signals into rhythms. The entropy of delta, theta, gamma, alpha, and beta rhythms has been classified with SVM. Tian et al. in Ref. [23] used a hierarchical method for the identification of sleep stages. Rhythms have been obtained by filtering the EEG signal using BPF. The segmentation technique has been used to partition the rhythms. Multiscale entropies of segmented rhythms have been classified with SVM. Memar et al. in Ref. [24] used infinite impulse response BPF to separate various rhythms. Multiple nonlinear features extracted from the rhythms have been classified with random forest (RF) classifier. The filtering methods in Refs. [19–24] use fixed filter coefficients. Also, improper selection of coefficients may discard informative samples. In Refs. [25–27], Hassan et al. used the analysis of amplitude and frequency modulated components. Empirical mode decomposition (EMD), ensemble EMD (EEMD), and complete ensemble EMD with adaptive noise (CEEMDAN) have been used to obtain instantaneous mode functions. Four statistical moments extracted from the instantaneous mode functions have been classified with bagging, Adaboost, and random under sampling boosting (RUSboost) classifiers. These EMD based methods in Refs. [25–27] are completely experimental and suffer mathematical modeling.

In Ref. [28], Ebrahimi et al. proposed Daubechies wavelet transform (WT) for extraction of features from multiple subbands. Three features were extracted and subjected for evaluation, which were taken from the subbands, and artificial neural networks (ANNs) were used for classification task. In Ref. [29], dual-tree complex wavelet transform (DTCWT) was used for feature extraction, and five features were obtained. The CVNN was used for classification of five statistical features extracted by DTCWT. In Ref. [30], Hassan et al. employed DTCWT for extraction of spectral features from signal subbands. The segmentation was also evaluated in this work using Kruskal-Wallis test. In Ref. [31], continuous wavelet transform (CWT), Choi-Williams distribution, and Hilbert-Huang transform were used for determining the time-frequency distribution of EEG signals. The features were extracted using Renyi's entropy, and RF classifier was used for classification. Sharma et al. in Ref. [32] employed wavelet filter banks (WFBs), which decompose the signal into multibands, and then features were extracted from the multibands, with the help of SVM. In Ref. [33], Fraiwan et al. implemented a multiwavelet analysis for decomposition purpose, which results subbands from the signal. The LDA method was used for extracting the features from the subbands. The subband features of discrete wavelet transform were classified by using SVM, in the work by Sousa et al. [34]. Ronzhina et al. in Ref. [35] employed STFT, FFT, and wavelet transform for extracting hybrid features, which were further classified using ANN method. The wavelet based methods reported in Refs. [22,28–33] have requirement of selection of appropriate type of wavelet and number of decomposition levels. In addition, improper choice of wavelet parameters results in a hard band limit of filter banks. Hassan et al. in Refs. [36–38] used tunable Q wavelet transform (TQWT) for signal decomposition. The subbands obtained after decomposition have been used to extract several spectral, statistical moments, and normal inverse Gaussian features. Bagging, Adaboost, and RF classifiers have been employed to classify these features. Taran et al. in Ref. [39] used the analytic form of wavelet transform called optimize flexible analytic wavelet transform (OFAWT) to extract time-domain features from the subbands. The ensemble bagged tree (EBT) classifier has been employed to classify the sleep stages. The TQWT and OFAWT methods in Refs. [36–39] use an empirical choice of tuning parameters that may result in localization issues, disappearing modes, and noisy subbands.

In Ref. [40], Bajaj et al. suggested time-frequency representation for evaluating statistical measures, and multiclass least square SVM (MC-LSSVM) was used for classification of the measures obtained using the time frequency representation. Karimzadeh et al. used Shannon entropy for classification that utilizes frequency and instantaneous phase, and k-nearest neighbor and decision tree were used as classifiers [41]. Zhu et al. in Ref. [42] obtained visibility graph from the EEG signals using mapping process, and horizontal visibility graph was also obtained. The SVM was used for determining the difference between the two graphs obtained through mapping process. Salih et al. in Ref. [43] used the Welch method for feature extraction, and the discriminative features were chosen using K-means clustering method that was based on feature weighting concept. The decision tree classifier was used for classification of extracted features. Koley and Dey in Ref. [44] used time-domain, frequency domain, and nonlinear features; optimal features from total 39 features were chosen using recursive feature elimination technique; and SVM was used as classifier. In Ref. [45], Gaussian observation HMM was used for the extraction and classification of features. The methods in this literature use empirical choice of parameters for feature extraction, selection, and classification. Direct extraction of features from raw EEG signals does not provide discriminative information; FFT and STFT based feature extraction methods create a trade-off in the time-frequency domain; filtering methods require tuning of filter coefficients for steep boundaries of filters; wavelet-based methods degrade due to hard band limits, vanishing modes and noisy subbands; and EMD-based method lacks proper mathematical modeling and causes mode mixing [46–48]. Moreover, these methods along with classifiers require tuning of multiple parameters. This results in loss of representative information, increasing number of misclassification, and underfitting and overfitting problems.

To address these shortcomings, a convolutional neural network (CNN) is used for automatic extraction and classification of deep features. CNN extracts multiple linear and nonlinear deep feature maps by a convolutional operation. These features are then converted to one dimension with fully connected layer; finally, the classification layer assigns the proper class after identifying the scores of each class. CNNs have been used extensively in various physiological and pathological problems like drowsiness detection, motor imagery classification, and focal identification [49–52]. In addition, the methods in Refs. [49–52] use pretrained networks that require multiple convolutional layers and a huge set of learnable parameters. To overcome this, an automatic identification of multiclass sleep stages combining time-frequency distribution (TFD) and adaptive convolutional neural network (A-CNN) is proposed. In this chapter, EEG signals are transformed to TFD using STFT, CWT, Zhao-Atlas-Marks distribution (ZAM), and smoothed pseudo Wigner-Ville distribution (SPWVD). The images obtained from transformation techniques are fed to A-CNN. It is a flexible network that requires fewer convolutional layers and learnable parameters to classify binary and multiclass problems. The performance of the proposed A-CNN is compared with benchmark pretrained AlexNet, VGG-16, and ResNet50 networks. Six performance parameters are evaluated to test system efficacy. Finally, the effectiveness of the proposed method is compared with the existing state-of-the-art using the same dataset.

The rest of the chapter is organized as follows: The methodology used is discussed in Section 1.2; Section 1.3 presents the results obtained using the proposed method; the detailed discussion of the results is presented in Section 1.4 including a comparison with state-of-the-art existing research on the proposed topic; and finally the conclusions are reported in Section 1.5.

1.2: Materials and methods

This section consists of dataset description, transformations techniques, and convolutional neural networks. The steps involved in the proposed method are shown in Fig. 1.1.

Fig. 1.1

Fig. 1.1 Steps of the proposed method.

1.2.1: Dataset

EEG recordings of the sleep-EDF dataset from PhysioNet data banks are used for the evaluation of the proposed method. The detailed description of the dataset is publicly available online [53,54]. Eight Caucasian females and males have been employed without any medication for EEG recording. The age group of these subjects lies in the range of 21–35 years. Recordings of a dataset have been partitioned in two subsets (marked as st and sc). The recordings of sc subset (sc4102e0, sc4012e0, sc4002e0, and sc4112e0) from ambulatory healthy volunteers have been obtained during 24 h of their daily normal life [55]. The recordings of st subset (st7121j0, st7052j0, st7022j0, and st7132j0) performed at night using a miniature system from subjects with mild difficulty of falling asleep otherwise healthy [56]. The sleep recordings of EOG, Fpz-Cz, and Pz-Oz have been utilized at a sampling rate of 100 Hz. It has been found that the EEG recordings of Pz-Oz channels play a significant role in the classification of sleep stages [19,35,42]. Hence, this methodology considers the data of the Pz-Oz channel for system evaluation. In this study, the duration of each epoch of EEG signal is defined as 30 s or 3000 data points following the R & K recommendation. Based on the expert scoring system, each epoch has been scored anyone out of six classes, namely AS, S1, S2, S3, S4, and REM. The number of signals belonging to each class is summarized in Table 1.1. EEG signals of each class are shown in Fig. 1.2. As seen in the figure, all the signals do not show any distinguishable characteristics.

Table 1.1

Fig. 1.2

Fig. 1.2 Example of EEG signals of sleep stages.

1.2.2: Transformation techniques

EEG signals are considered to be complex, nonstationary, and nonlinear: due to this, analysis of raw EEG signals is very difficult. A signal is transformed to other domain for finding the representative information. A signal in time domain can be transformed to frequency domain, analytic form, and time-frequency representation. Analysis of nonstationary signals in time-frequency domain has outperformed conventional time-domain and frequency-domain approaches [57]. Hence, four TFD methods, namely STFT, CWT, ZAM, and SP-WVD are employed in this methodology. TFD is defined as a technique that is use to represent the spectral variations (frequency and energy) of a signal with respect to time. The details of these are discussed later.

a.Short Time Fourier Transform

STFT is a modified version of FFT. It can be viewed as a block that partitions the signal and independently applies FFT to each block. The signal is partitioned by taking off a part of the time domain signal with the help of windowing function. Later, FFT is applied to the partitioned portion to identify different representative characteristics of the signal. The TFD of an input signal s(t) is defined as follows [49]:

si1_e

   (1.1)

where p(t) is the window function. The spectrogram of a signal is a magnitude squared function of STFT and represented as spectrogram = S(t,f)². The type and length of a window must be same. In addition, a signal is assumed to be stationary in the duration of window. STFT can be picturized as symmetric bandpass filters equally spaced in frequency.

b.Continuous Wavelet Transform

The STFT provides the signals that can be seen as localized in Fourier domain in the form of some combination of cosine and sine signals having infinite length. This is why some localized time domain information is lost. Wavelets are used as combinations of small oscillatory functions as these are highly localized in time. The CWT functions are used to produce Fourier space as well as localized real space which is scaled and shifted, and these versions become as time-localized mother wavelet. The CWT also maps the TFD information in order to get good time and frequency localization. The CWT of a signal s(t) can be expressed as in Ref. [49]:

si2_e    (1.2)

where the scaled and shifted basis wavelet function (ψ(t)) is denoted by si3_e . The parameters α and β denote the scaling and shifting variables. By adjusting the scaling and shifting variables, the spread and the shift of time-domain signals can be regulated and localized. Therefore, we can see CWT as linear combination of frequency varying bandpass filters. The window is used to analyze the lower frequency components of the signal, having larger window size or high frequency analysis made using low sized window. This method is especially advantageous regarding accurate time-frequency distribution and redundancy issues.

c.Zhao-Atlas-Marks Distribution

ZAM distribution is another one such technique for the representation of TFD. ZAM is also known as a cone-shaped kernel. The advantage of the ZAM is that finite time support is maintained in the time dimension along with an enhanced resolution in the frequency dimension. The advantage of ZAM is that the cross-terms in time-domain are smoothed out. A generalized TFD of an input signal s(t) and the kernel required to compute ZAM are denoted as follows [58]:

si4_e

   (1.3)

where φ(t, τ) is the time-frequency representation kernel; ϕ1(t) is the scaling function to be specified during run time; τ is the running time; ξ is the running frequency; t′ is the position variable; a is the constant scaling variable; and s⁎(t) denotes the complex conjugate. The larger length window function captures low-frequency details, while a smaller length window captures high-frequency resolution of the signal.

d.Smoothed Pseudo Wigner-Ville Distribution (SPWVD)

The SPWVD is seen as very efficient method used for representing the signal energy over time-frequency. The transformation produced by CWT and STFT mainly suffers from localization issue that is associated with time-frequency. In addition, a cross-term in the time-frequency domain is unnecessarily introduced. The TFD suggested by ZAM attempts to address the challenges related to localization, and also the cross-term is reduced in time-domain. Nevertheless, the window having smaller size helps to obtain satisfactory level of localization, but at the expenses of poor frequency resolution. This is overcome by the introduction of a cross-term reduction window in frequency-domain by SPWVD. The TFD of an input signal s(t) is obtained by SPWVD [59–61]:

si5_e

   (1.4)

where g(t) is the time-domain cross-term reduction window; u(f) is frequency-domain cross-term reduction window; and [Fcy](t, f) is the Wigner-Ville distribution. SPWVD has an option to select the type and the length of the window independently. Time-frequency distribution obtained by using transformation techniques is shown in Fig. 1.3. Fig. 1.3A–D represents the TFD obtained by SPWVD, STFT, CWT, and ZAM, respectively. As evident from the figure, TFD of SPWVD provides the highest insight of the signal, followed by ZAM and CWT, and the worst resolution of sleep signals is provided by STFT. In addition, SPWVD-based CNN feature extraction and classification have been used previously for detection of emotions and schizophrenia [60–62].

Fig. 1.3

Fig. 1.3 Example of time-frequency distribution of sleep signal: (A) SPWVD, (B) STFT, (C) CWT, and (D) ZAM.

1.2.3: Convolutional neural network (CNN)

An extensive and systematic analysis is required for appropriate extraction, selection, and classification of distinct features in the classification or its application. There are numerous conventional methods, which are used for the purpose, but the methods suffer with many problems such as tuning of several multiple parameters and more training time. In addition, the methods have more computational complexity and are prone to human errors. The CNN is reported as promising method that has ability to address the issues in conventional methods for classification. The classification including feature extraction and selection becomes much automated with the introduction of CNN due to the convolution operations which are integral parts of CNN. The CNN is an extension of the machine learning family and composed of self-optimized neurons. It is designed to work with images by taking into the configurable structure and special input into account. In recent years, CNN has been used in various disciplines for object detection, edge detection, fault detection, etc. CNN is applied to various applications of biomedical informatics including drowsiness detection, and motor classification seizure classification [49–51]. CNN architecture is inspired by the visual perception mechanism of the living creatures. The features are extracted and classified with interconnected multilayer neurons trained rigorously. The previously learned knowledge of preceding layer is used by these self-optimized neurons. The knowledge acquired by the system assists in training one task and reusing the same for other tasks. The CNN generally contains an input layer, an output layer, and hidden layers. The hidden layer can be visualized as combination of layers for convolution, pooling, fully connected operation, and classification.

The components of hidden layer are discussed as follows [63]:

Convolutional Layer: This is the heart and core building block of CNN. Convolutional layer (CL) uses learnable filters or kernel functions. This kernel is moved along the length and convolved with the part of the input image. Each spatial dimension of learnable filters is convolved to find the representative information. The two-dimensional convolution operation is expressed as follows:

si6_e

   (1.5)

where X is the portion of the image and Y is the kernel. Zero padding (z) is used to keep the original size of the image. A stride (p) is used to move the kernel with a fixed pixel. The output volume (L0 B0 F0), for an image with a dimension Lc, Bc, and Fc, can be expressed as follows:

si7_e    (1.6)

where Lc is the width; Bc is the length, and Fc is the number of channels of an image. The filter is denoted by F0 with size k × k. The activation function is used to increase the nonlinearity in the network. The rectified linear unit (ReLu) is the most common activation function.

Pooling Layer: The CL layer is followed by poling layer which is also called as subsampling layer. The main aim of using the pooling layer (PL) is to address the dimensionality probe, and thus, this layer reduces the dimensionality of the feature maps provided by CL layer. This is done by using the concept of down sampling, and the feature maps are reduced. One important point here is noteworthy that output and input feature maps are not affected by the down sampling process. The feature map after down sampling, as reduced feature map, is represented as follows:

si8_e    (1.7)

where additive and multiplicative constants are denoted by dlm and elm, and down sampling operation is represented by down(.).

Fully Connected Layer: The PL is succeeded by a fully connected (FC) layer. The down sampled two-dimensional feature maps are converted to one-dimension with a feed-forward neural network. These one-dimensional deep features are then given to the classification layer. The classification layer assigns the score to each object of a class. These scores are then converted to probabilities with the help of some algorithms. Finally, a voting scheme is used to assign a class to an object with the highest probabilities.

The steps involved in feature extraction and classification of images are shown in Fig. 1.4. Users can use these elements of the hidden layer to build a convolutional neural network. The number of components of a hidden layer can be added or deleted until the desired performance is obtained. Besides, there is no standardized CNN available for the diagnosis of brain abnormalities. Now, several pretrained CNN architectures and models are available having multiple number of CL layers. These types of deep network models can handle huge number of learning attributes, and thus, both computational complexity and computational time are increased. Though, probability of performance success of CNN in many cases becomes bleak. So, the network having less complexity and adaptive in nature can overcome the CNN issues. The current literature suggests that CNN development is taking place at fast pace CNN [64,65]. To deal with these challenges, we have used an adaptive CNN in this chapter, which has four convolutional layers, two pooling layers, and two fully connected layers. In the first two convolutional layers, 96 and 32 number of filters, each of size 7 × 7 and 5 × 5, respectively, were used. The third and fourth CL layers have 32 and 16 filters, and each filter is having a size of 3 × 3. A 50% additive downsampling operation is used to reduce the dimensionality of output maps. The ReLu activation function is used to increase the nonlinearity in the network. The two fully connected layers composed of 500 and 128 hidden neurons, respectively. The detailed working of the proposed CNN is shown in Fig. 1.5. The efficacy of the proposed A-CNN is compared with the benchmark pretrained network. The pretrained AlexNet, VGG-16, and ResNet50 are used for performance comparison. The detailed descriptions of these networks are available in Ref. [66].

Fig. 1.4

Fig. 1.4 Feature extraction and classification using CNN.

Fig. 1.5

Fig. 1.5 Proposed adaptive convolutional neural network for sleep stage classification.

1.3: Results

The R & K and AASM standards primarily divide the sleep stages based on scoring. The standards proposed by R & K says epochs of sleep annotated as anyone of AS, S1, S2, S3, S4, and REM, respectively. On the other hand, AASM annotates the SWS state as a combination of S3 and S4 stages, while remaining stages are maintained the same. The proposed methodology aims to classify a different sleep stage (two-class to six-class). The formation of two classes is accomplished by combining S1–S4 and REM stage as sleep class while others as AS. In three class problem, AS, REM, and NREM (a combination of S1–S4) are considered. Four class problem is defined as AS, SHS (S1 & S2), SWS (S3 & S4), and REM. Five and six class problems are maintained as per R & K and AASM standards. The details of class formation are shown in Table 1.2.

Table 1.2

A common experimental platform is maintained for transforming the signals into TFD. A nonoverlapping Hamming window and 256 points FFT are used to obtain the spectrogram using STFT. The Morse wavelet is used to transform a signal into a scalogram by CWT. To obtain the time-frequency representation using ZAM, Kaiser's window of length 63 is used. In SPWVD, Kaiser window of length 63 is used to reduce the cross-term in the time-frequency domain. The problems of low resolution and high memory can be introduced due to the selection of smaller and larger window sizes. Hence, a medium length window is selected empirically. The spectrograms, scalograms, and time-frequency representations obtained with STFT, CWT, ZAM, and SPWVD are resized before applying to CNNs. AlexNet and A-CNN take input with an image of size 224 × 224, while the images taken by VGG-16 and ResNet50 are of size 227 × 227, respectively. The resized images are then given as input to CNNs. The hyperparameters of every CNN are maintained uniform. Seventy-five percent of the input data is used for training while the remaining is used for testing and validation. The batch size and learning rate are selected empirically to 32 and 0.0001, respectively. The learning rate of weights and bias is kept at 20, and number of epochs is set to 10. The learning rate of weights is scaled by Adam optimizer. Table 1.3 shows the classification accuracy provided by A-CNN, AlexNet, VGG-16, and ResNet50 using spectrograms. Classification accuracy with AS and sleep class is 100% for A-CNN, AlexNet, VGG-16, and ResNet. An accuracy of 98.4%, 95.64%, 96.15%, and 96.54% is obtained for AS, NREM, and REM classification using A-CNN, AlexNet, VGG-16, and ResNet50. The AS, SHS, SWS, and REM classification has an accuracy of 95.48%, 94.26%, 93.56%, and 92.59% for A-CNN, AlexNet, VGG-16, and ResNet50. Classification accuracy of AS, S1, S2, SWS, and REM class is 94.12%, 93.84%, 93.12%, and 93.56%, while, for six class, accuracy is 91.46%, 91.02%, 90.65%, and 92.15% using A-CNN, AlexNet, VGG-16, and ResNet50. The average accuracy with A-CNN, AlexNet, VGG-16, and ResNet50 using spectrogram is 95.89%, 94.95%, 94.69%,