Wavelet Analysis of Electroencephalography Signals of Visual Emotion Induction in Schizophrenia Patients

1Department of Mechanical Engineering, National Chin-Yi University, of Technology, No. 57, Sec. 2, Zhongshan Rd., Taiping Dist., Taichung City 41170, Taiwan (R.O.C.) 2Department of Physical Therapy and Graduate Institute of Rehabilitation Science, China Medical University, No. 91, Xueshi Rd., North Dist., Taichung City 404333, Taiwan (R.O.C.) 3MOST AI Biomedical Research Center at NCKU, No. 1, Daxue Rd., East Dist., Tainan City 701401, Taiwan (R.O.C.) 4Department of Psychiatry, Chiayi Branch, Taichung Veterans General Hospital, No. 600, Sec. 2, Shixian Rd., West Dist., Chia-Yi City 600573, Taiwan (R.O.C.)


Introduction
The pace of life and work in modern society is very fast and people these days are under much more stress than in the past. As a result, the number of patients with physical and mental illnesses is also increasing. Many people are unaware of the amount, or even of the existence, of stress to which they are regularly exposed. The overwhelming stress of a tumultuous daily life can lead to physical aging, mental exhaustion, and even emotional breakdown. Physical and mental illnesses have become an issue that cannot be ignored by society. The clinical classification of physical and mental illnesses includes commonly seen but less serious conditions, such as depression, obsessive-compulsive disorder, and panic attacks. However, the more serious disorders encountered include schizophrenia, affective psychosis, and delusional disorder. In this study, we focused on schizophrenia, which usually arises in individuals between the ages of 15 and 25, with men usually showing symptoms at an earlier age than women. (1) According to statistics published in 2017 by the Ministry of Health and Welfare of Taiwan, 119,461 patients have been diagnosed with schizophrenia, which is 0.4% of the total population of Taiwan. The worldwide incidence of 1% shows that many patients are going untreated. Schizophrenia is a psychosis that involves barriers between various mental processes such as perception, emotion, thinking, and behavior, as well as the coordination of psychiatric activities. The neuropsychiatric symptoms appear gradually and self-detection is very difficult. The main core characteristics include auditory and visual hallucinations. (2,3) In clinics, a questionnaire is commonly used for the quantification of schizophrenia, and the guidelines used include the Diagnostic and Statistical Manual of Mental Disorders (DSM), International Statistical Classification of Diseases and Related Health Problems (ICD), and Positive and Negative Syndrome Scale (PANSS). Discussions and investigations of these scales regarding their defects and differences between them have been dealt with in several studies. (4)(5)(6) Among them, PANSS is an extensively used tool, despite a controversy about its scale. Van den Oord et al. (7) carried out an analysis with PANSS and DSM-IV on 500 schizophrenia patients. The results indicated that the controversies about PANSS are centered around the requirement for large amounts of factor loadings. Moreover, they also discussed the relationship between the PANSS score and single symptoms with the quality of life. (8) All these studies showed that the different scales used in outpatient clinics can lead to different results. In this study, an attempt was made to utilize different visual stimuli to induce emotions. The resulting brainwaves were collected and feature extraction was carried out. The extracted features were then used in analysis. A great deal of research has been conducted into the induction of emotions, which is a very complicated phenomenon not easily recognized by machines or humans. (9) Many different approaches have been used to induce emotions. Gross and Levenson (10) extracted the stimulant segments in movies for induction. Kim et al. (11) used different colored lighting and music to create different atmospheres to induce emotions. Jian et al. (12) induced emotions in schizophrenia patients and simultaneously collected the signals of thermal facial images to analyze and observe differences in facial temperature. In addition, Bandara et al. (13) utilized functional near-infrared spectroscopy (fNIRS) to classify different emotional variations. From these studies, it can be seen that emotions can be induced under different circumstances and that the human judgment of emotions can be affected at different psychological and physical levels. This will result in differences in judgment. Therefore, the most commonly used method for the quantification of emotions is mapping into a two-dimensional space representing emotional valence and arousal. Lang et al. (14) used an emotional stimulation database (IAPS, http://csea.phhp.ufl.edu/Media.html) established for visual emotion induction in their study.
Over the past few years, many studies on the development of concentration and implementation functions in the fields of psychology and cognitive neuroscience have been conducted, with extensive and diverse results. Electroencephalography (EEG) plays a very important role in brain research and has several advantages: it allows long-term measurement, is noninvasive, involves no radiation, and is very useful in diagnosis and applications. Sestito et al. (15) studied and discussed the visual perception of aircraft pilots based on EEG signals. Kreither et al. (16) used electrophysiological signals in a study of visuospatial attention in both schizophrenia patients and normal people. Their results showed that schizophrenia patients were better able to focus on fixation points and filter out interference in their surroundings than normal people. In addition, many papers have been published on the measurement of brainwaves associated with concentration and emotions, and many relationships between emotions and the brain have been proposed. (17)(18)(19) However, EEG signals are unstable and complex, and nonlinear (20) EEG signals in various frequency ranges have different levels of physiological significance. (21) Many different approaches have been used to record EEG signals in various situations. Li et al. (22) applied the discrete wavelet transform (DWT) and neural network sets to the classification of epileptic EEG signals. Kumar et al. (23) utilized the DWT for the feature extraction of approximate entropy (ApEn). Fu et al. (24) utilized the Hilbert-Huang transform (HHT) for the analysis of brain waves and a support vector machine (SVM) for classification. Brainwave signals are very complicated, and the selection of features is based on the capture of maximum, minimum, and standard featured data. (25,26) In addition, several studies used entropy for the analysis of EEG signals. (19,27,28) Therefore, in this study, we used entropy as the feature to extract EEG signals.

Participants and materials
Diagnostic judgments regarding schizophrenia made in clinics are mainly based on the use of PANSS. The P scale of PANSS was used in this study to separate two groups of test participants. The patients in one group were moderately ill (having scores of 24 or more), and those in the other group were markedly ill (scores below 24). The same amount of medication was used for the two groups as shown in Table 1. The subjects were recruited from the outpatient clinic at the Department of Psychiatry, Chiayi and Wanqiao Branches, Taichung Veterans General Hospital, Chia-Yi, Taiwan. Subjects underwent screening that included medical and psychiatric histories, laboratory test results, drug screening, and physical examination. The psychiatric diagnosis of depression was established using the structured clinical interview from DSM-IV and a semi-structured interview performed by a research psychiatrist. All the subjects were given a comprehensive explanation of the study procedures and all gave written informed consent as approved by the institutional review board. This research was approved by the ethics committee of Taichung Veterans General Hospital and conducted in accordance with Good Clinical Practice procedures and the current revision of the Helsinki Declaration. (22)  The experiments in this study were based on visual emotion induction. A total of 45 images were selected from the IAPS image database for induction and the images were displayed to 100 normal people (48 male and 52 female). Two-dimensional space was used to classify valence and arousal using the Likert scale for analysis. The results of the experiments are shown in Table 2.
The selected images were displayed to the participants as a video of a series of stimulation units. The display time for each image was 3 s, and five images with the same dimensions were selected for each stimulation unit of 15 s length. There was a 10 s interval between stimulations. Nine exposures to stimulation units were conducted over a total period of 225 s. Brainwave signals were collected from each of the participants as they watched the video. The three points of the EEG signals collected were Fz, Cz, and Pz with (A1+A2)/2 as the reference electrode. The ProComp Infiniti System (Thought Technology Ltd., Canada) was used to digitize the collected signals and save the data as a .txt file in a computer. Unnecessary objects were cleared away from the experimental area, and disturbing light and noise were reduced to prevent interference with the experiments. A diagram of the testing environment is shown in Fig. 1.

Analysis of signal preprocessing
By the collection approach introduced in the previous section, EEG signals were collected from three points, and the clinical aspects of the α, β, and θ wave signals were based on different frequencies. The significance of one-dimensional EEG brainwave signals was also different in different frequency bands. The signals were initially noisy because they were  collected from patches through leads and then amplified. A preprocessing step that removes noise by filtration was necessary before analysis. The conventional approach for noise removal is linear or nonlinear filtering using a median filter, a Butterworth filter, or some other filter. However, this can increase entropy after conversion and cause the signals to become even more unstable. Therefore, wavelet conversion was used in this study for noise filtration. The filtered signals were cut at different times to match the visual stimulation then subjected to wavelet decomposition. The decomposed signals were used for feature extraction, and the physiological features were used to establish a database. The features from the database were analyzed by the processing steps shown in Fig. 2.
Frequency is a very important concept in signal processing and is often used in the analysis of signal features. Methods used include Fourier analysis, Gabor transformation, and continuous wavelet transform. However, the Fourier transform is more useful for periodic signals and has relatively little effect on partial signals. The continuous wavelet transform does not have the base variation restriction of the Fourier transform and uses the translation scaling of the function as a basis for frequency spectrum analysis. ψ(t) is used to express the wavelet function to satisfy integral ψ(t) on the interval (−∞, ∞). From this equation, the integral boundary is from negative infinity to positive infinity. However, there are boundaries that are from negative infinite to positive infinite. Therefore, ψ(t) is defined as a space and can be integrated. The mathematical formula can be written as ψ(t) ∈ L 2 (R).
The DWT is widely applied in many signal processing applications. In this study, the source signals were reconstructed, and the wavelet function was required to satisfy the admissible condition in Eq. (1). (29)  A wavelet function that satisfies this condition is called an admissible wavelet function. The wavelet function ψ(t) is usually referred to as the mother wavelet, and a series of functions can be obtained by the translation scaling of ψ(t) as shown in Eq. (2).
This normalizes ||ψ(t)|| 2 = 1, where a is the scale or extension factor and b is the translation factor.
Let f(t) ∈ L 2 (R) and ψ(t) be an admissible wavelet. Then, The DWT was used; hence, a and b are continuous in the wavelet reconstruction given by Eq. (4). Therefore, it is necessary to find discrete a and b to reconstruct the original signals, which leads to the discrete wavelet From Eq. (4), a and b are real numbers, and the integral is continuous; thus it is not convenient for computer calculation. Therefore, we reconstructed a and b to a digital type, which is called wavelet transform. Here, a = 0 j a and b = 0 j a k (a 0 > 0 is a constant; j, k ∈ Z).
Let f(t) ∈ L 2 (R), then Its stationary value is given by The wavelet transform is very different from the Fourier transform. The wavelet base function is not the only one that will satisfy these conditions. In many studies, wavelets have been proposed for a range of purposes, and Haar, Daubechies, Morlet, and biorthogonal wavelets are commonly seen. In this study, the Daubechies (dbN) wavelet was used, which is a discrete orthogonal wavelet based on a two-scale equation {h k }, where dbN is asymmetric and its transfer function {h k } is given as Here, 1 N k k C − + is the binomial coefficient and N is the order of the wavelet. Except when N = 1, In this study, we used the db6 Daubechies wavelet family for analysis. The analyzed wavelet level corresponds to the defined range of brainwave frequencies, and we focused on the analysis of the D2-D5 signals. The frequency bandwidth of D2 is 32-64 Hz, that of D3 is 16-32 Hz, that of D4 is 8-16 Hz, and that of D5 is 4-8 Hz.

Extraction of signal features
The concept of entropy is used for feature extraction. Next, we introduce Shannon entropy, log entropy, and amplitude-aware permutation entropy (AAPE). The use of entropy as a definition of the level of information variation was first proposed by Shannon in 1984. Since then, many similar entropy calculation methods have been proposed and applied to the analysis of EEG signals. In this study, we used the Shannon entropy E 1 (s), which is given by In the following expressions, s is the signal and i is the orthonormal coefficient of s. The log energy E 2 (s) is given by Permutation entropy (PE) is an entropy calculation proposed by Bandt and Pompe in 2002. It allows the simple computation and magnification of fine details in a time sequence. (30) AAPE is sensitive to changes in signal amplitude and frequency, and is more flexible than classical PE in the quantification of signal motifs. (31) The AAPE algorithm takes into account the mean of amplitudes and differences between amplitudes. PE is defined as where the relative frequency p(π i ) is the probability of various signal permutations. AAPE modifies the calculation method for the probability of p(π i ) and adds sensitivity to changes in amplitude. (31)

SVM
SVM is a supervised learning method proposed by Vapnik and coworkers in 1999. (32,33) It is similar to the multilayer perceptron and radial basis function networks, which can be utilized for pattern classification and nonlinear regression. SVM is used to establish a hyperplane decision curve to maximize the marginal distance between two categories that need to be distinguished. The hyperplane can be defined as If f ≥ 0, it belongs in one class, and if f ≤ 0, it belongs in another class. To find the maximum boundary of the hyperplane, the minimum value of w 2 has to be found first. Lagrange optimization can be used for processing, which is expressed as It is also necessary for alignment to be made with the kernel function in SVM to obtain good classification. For separation, projection into a higher-dimensional space is required when the linear classification of the kernel function cannot be done. In this study, the SVM was trained and classified by Linear SVM, Cubic SVM, and Fine Gaussian SVM in Matlab R2019a (Math Works, USA) for analysis and discussion.

Results and Discussion
In this study of emotion induction in schizophrenia patients, the three main electrode points used for brain detection were Fz, Cz, and Pz with (A1+A2)/2 as the reference electrode. The sampling rate was 256 Hz and the received EEG signals were converted with the wavelet function signals in the D2-D5 frequency range. The signals were cut to coincide with the stimulation time and classified into two groups, stimulation and nonstimulation, which were used for feature extraction; AAPE, log, and Shannon entropy were used to calculate the features of signals in the different frequency bands to establish a database. The features were entered into the Linear SVM, Cubic SVM, and Fine Gaussian SVM (cross-validation: 10-fold). The results are shown in Tables 3-5. In Table 3, the analysis is based on Fz signals. The results show that the highest recognition rate is for D3. The accuracy is 75.9% under Linear SVM with the feature of AAPE and becomes as high as 87% under Cubic SVM with AAPE. In addition, the accuracy is 75.9% under Fine Gaussian with the log feature.
In Table 4, the analysis is based on Cu signals. The identification rate is good in the D2-D4 frequencies. In Table 5, the analysis is based on Pz signals and the results indicate that the recognition rate is best in D4, where the accuracies are 75% under Linear SVM with AAPE, 82.1% under Cubic SVM with AAPE, and 82.1% with Fine Gaussian SVM with AAPE. In addition, it can be seen from the features of EEG signals in Tables 3-5 that schizophrenia patients show a higher EEG response with stimulation than without stimulation and that the recognition rate is as high as 87% on Fz.

Conclusions
In this study, emotions were induced visually with two groups of schizophrenia patients and brainwaves were collected from three points (Fz, Cz, and Pz) at the same time. The collected signals were converted by the wavelet transform, and features (AAPE, log, Shannon) of the signals were extracted. These signal features from different frequency bands were used to create a database. The results showed that the frequency band of D3 on Fz, bands D2 to D4 on Cz, and band D4 on Fz have the best identification rates. It was found that D2 brainwaves corresponded to γ waves, D3 brainwaves to ꞵ waves, and D4 brainwaves to α waves. The EEG signals in these frequency bands are closely related to concentration and emotional fluctuations. Furthermore, the results also indicated that there are measurable differences in response to visual stimulation between moderately ill and markedly ill schizophrenia patients. This study can be used as a reference for the classification of the severity of schizophrenia from the different EEG signals generated by inducing emotions.