ISSN (print) 0914-4935
ISSN (online) 2435-0869
Sensors and Materials
is an international peer-reviewed open access journal to provide a forum for researchers working in multidisciplinary fields of sensing technology.
Sensors and Materials
is covered by Science Citation Index Expanded (Clarivate Analytics), Scopus (Elsevier), and other databases.

Instructions to authors
English    日本語

Instructions for manuscript preparation
English    日本語


 Sensors and Materials
 1-23-3-303 Sendagi,
 Bunkyo-ku, Tokyo 113-0022, Japan
 Tel: 81-3-3827-8549
 Fax: 81-3-3827-8547

MYU Research, a scientific publisher, seeks a native English-speaking proofreader with a scientific background. B.Sc. or higher degree is desirable. In-office position; work hours negotiable. Call 03-3827-8549 for further information.

MYU Research

(proofreading and recording)

(translation service)

The Art of Writing Scientific Papers

(How to write scientific papers)
(Japanese Only)

Sensors and Materials, Volume 31, Number 11(1) (2019)
Copyright(C) MYU K.K.
pp. 3451-3463
S&M2023 Research Paper of Special Issue
Published: November 8, 2019

Dual-input Control Interface for Deep Neural Network Based on Image/Speech Recognition [PDF]

Neng-Sheng Pai, Yi-Hsun Chen, Chin-Pao Hung, Pi-Yun Chen, Ying-Che Kuo, and Jun-Yu Chen

(Received May 24, 2019; Accepted October 8, 2019)

Keywords: Adaboost, convolutional neural network, Mel-frequency cepstral parameters, long short-term memory neural network

The objective of this study was to design a control interface for dual-input video/audio recognition consisting of two input interface systems, hand posture and speech recognition, with the use of specific hand postures or voice commands for control without the need for wearable devices. Original video camera images were preprocessed for hand posture recognition, and the face in the image was used as the reference point and identified using the Adaboost classifier. An image of a specific size was selected as the recognition input image to increase the recognition speed. A neural network comprising convolutional, activation, max pooling, and fully connected layers was used to classify and recognize hand posture images as well as speech. Long short-term memory (LSTM) in a recurrent neural network (RNN) was used to achieve speech recognition. Speech features were extracted by preprocessing, and Mel-frequency cepstral coefficients (MFCCs) and a fast Fourier transform (FFT) were then used to convert the signals from the time domain to the frequency domain. The frequency domain signals subsequently underwent a discrete cosine transform through triangular bandpass filters to derive MFCCs as the speech eigenvalue input. The speech feature parameters were then input to the LSTM neural network to make predictions and achieve speech recognition. Experimental results showed the image/speech dual-input control interface had good sound recognition capability, supporting the findings of this study.

Corresponding author: Neng-Sheng Pai

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Cite this article
Neng-Sheng Pai, Yi-Hsun Chen, Chin-Pao Hung, Pi-Yun Chen, Ying-Che Kuo, and Jun-Yu Chen, Dual-input Control Interface for Deep Neural Network Based on Image/Speech Recognition, Sens. Mater., Vol. 31, No. 11, 2019, p. 3451-3463.

Forthcoming Regular Issues

Forthcoming Special Issues

Special Issue on Advanced Materials on Electronic and Mechanical Devices and their Application on Sensors (5)
Guest editor, Teen-Hang Meen (National Formosa University), Wenbing Zhao (Cleveland State University), and Cheng-Fu Yang (National University of Kaohsiung)

Special Issue on Advances in Shape Memory Materials
Guest editor, Ryosuke Matsui (Aichi Institute of Technology) and Hiroyuki Miki (Tohoku University)

Special Issue on Perceptual Deep Learning in Computer Vision and its Application
Guest editor, Chih-Hsien Hsia (National Ilan University)

Special Issue on Materials, Devices, Circuits, and Analytical Methods for Various Sensors (3)
Guest editor, Chien-Jung Huang (National University of Kaohsiung), Cheng-Hsing Hsu (National United University), Ja-Hao Chen (Feng Chia University), and Wei-Ling Hsu (Huaiyin Normal University)
Conference website

Special Issue on Sensing Technologies and Their Applications (1)
Guest editor, Rey-Chue Hwang (I-Shou University)
Call for paper

Special Issue on New Trends in Smart Sensor Systems
Guest editor, Takahiro Hayashi (Kansai University)
Call for paper

Copyright(C) MYU K.K. All Rights Reserved.