Our ServicesResearch and InnovationCambodia Journal for Business and Professional PracticePublicationPublication DetailFeature Extraction and Convolutional Neural Networks for Static Hand Gesture Recognition and Context Sentence Generation

Feature Extraction and Convolutional Neural Networks for Static Hand Gesture Recognition and Context Sentence Generation

Kimlong Ngin

University of Heng Samrin Thbongkhmum

Pakrigna Long

ACLEDA University of Business

Pichkhemara Morn

ACLEDA University of Business

ISSN: 3078-4131 (Print)

DOI: https://doi.org/10.XXXXX/XXXXX

Keywords: Feature extraction, convolutional neural networks, static hand gesture recognition, human-computer extraction, context sentence generation

Published: 2024-08-30

How to Cite: Ngin.K., Long.P., & Morn.P.(2024). Feature Extraction and Convolutional Neural Networks for Static Hand Gesture Recognition and Context Sentence Generation. Cambodia Journal for Business and Professional Practice, 2024(1), 31–51. https://doi.org/10.XXXXX/XXXXX

Download

Abstract

Hand gestures based on human-computer interaction are both intuitive and versatile, with multiple and diverse applications including in smart homes, games, operating theaters and vehicle infotainment systems. This research presents a novel architecture by combining a convolutional neural network (CNN) and traditional feature extractors to examine the accuracy of static hand gesture recognition. This research provides three significant contributions. First, we use the Non-Dominated Sorting Genetic Algorithm II (NSGAII), an evolutionary algorithm to classify and select image features across five methods, including the Gabor filter, the Hu-moment, the Zernike moment, the Complex moment, and the Fourier moment. Experimental results demonstrated that the combination of the Gabor filter, the Hu moment, and the Zernike moment achieved the best result with an accuracy of 98.3% to 99.0%. The Zernike moment combined with the Hu-moment output had an accuracy of 95.5% to 98.0%. The second contribution proposes the use of the Multiple Feature Convolutional Neural Network (MFCNN) model to generate better image recognition through the combination of validation techniques and features descriptors. Extensive experimentation was conducted utilizing binary and grayscale, as well as two different validation techniques - the Holdout technique and the Cross-validation of leaving one subject out of the validation. The proposed architecture was evaluated on two dataset types and is compared with the state-of-the art convolutional neural networks (CNN). The Massey’s dataset, contained 2,524 images and 36 gestures, and the OUHANDs dataset contained 3,000 images and 10 gestures. Experimental results demonstrated a high recognition rate using descriptors with low computational cost and reduced size. The third contribution is the sequence sentences generation based on the Beam Search (BS) algorithm. The data obtained from CNN/Daily Mail documents and results of image recognition, i.e., the image’s label, were used to test various question size with four different sizes of questions, including 100, 1,000, 10,000, and 40,000. The experimental results showed that our method could achieve high-quality sentence generation.

Authors’ Biography

Kimlong Ngin, born in 1990 in Kandal Province, obtained a Master's degree in Computer Science (Applied Computer Application) from the University of Science and Technology of China in 2019. He earned his bachelor's degree in Computer Science from the Royal University of Phnom Penh in 2013. With extensive experience in software development and team leadership, Mr. Ngin has made significant contributions to the field. In recognition of his expertise, he was awarded the title of Assistant Professor in 2023. Currently, he serves as a lecturer at the Institute of Information Technology at the University of Heng Samrin Thbong Khmum, where he continues to educate and inspire future generations of computer scientists.

Pakrigna Long is a passionate technology lecturer in Phnom Penh. He has a dual bachelor's degree; in Computer Science from National Polytechnic Institute of Cambodia and Informatics Engineering from STMIK IKMI Cirebon, Indonesia. He holds a Master of Engineering in Computing Engineering Systems from King Mongkut's Institute of Technology Ladkrabang, Thailand. Currently, he is a PhD candidate at the Universiti Kuala Lumpur, researching on Data Science and Analytics. In addition to his qualifications and current role, he used to be an IT officer, IT Programmer, IT Supervisor in Phnom Penh, and a data analyst in Kuala Lumpur. His research and development interests are Natural Language Processing, Data Science, Robotics, and Machine Learning.

Pichkhemara Morn was born in Kampong Cham Province in 1979, and holds a master's degree in Information Technology from Norton University of Cambodia in 2011. He has worked as a lecturer in a related field since 2003 in Phnom Penh and provinces. Currently, he is a full-time senior lecturer at ACLEDA University of Business (AUB). Besides teaching, he was a developing software conductor. Furthermore, he contributes research in team. He's interested in desktop and mobile app development the most. He is an outstanding lecturer and developer at AUB.

Authorship Disclaimer

The authors are solely responsible for the content of this article. The views expressed herein are those of the authors and do not necessarily reflect the views of the journal, its editors, or the publisher.