DEVELOPING A SOFTWARE MODULE FOR TRANSLATING WORDS FROM SIGN LANGUAGE INTO AN AUDIO FORMAT
https://doi.org/10.34822/1999-7604-2022-1-46-54
Abstract
The article determines and formulates the main requirements to a neural network that detects hand gestures. A study of existing approaches is conducted on the matter of their compliance with the requirements. A list of real conditions is compiled to imitate an application of a sign recognition module “Dialogue Sign Language Translator”. In addition to that, the required camera characteristics were measured for proper operation of the neural network in order to obtain a real-time palm recognition quality of at least 75 %. A prototype of a neural network was developed. The study shows the efficiency at differentiating left and right hands during imitation. From the prospect point of view, it is necessary to build the software operating both with signs and with speech in order to continue developing the technologies for inclusion of people with communication difficulties.
About the Authors
V. O. Semenova
Surgut State University, Surgut
Russian Federation
Russian Federation
Bachelor’s Degree Student
Е-mail: semenova_vo@bk.ru
L. L. Semenova
Surgut State University, Surgut
Russian Federation
Russian Federation
Senior Lecturer
Е-mail: semenova_ll@surgu.ru
References
1. Бизюкин Г. А., Майков К. А. Адаптивный метод распознавания динамических жестов // Новые информ. технологии в автоматизир. системах. 2017. № 20. URL: https://cyberleninka.ru/article/n/adaptivnyy-metod-raspoznavaniya-dinamicheskihzhestov (дата обращения: 15.12.2021).
2. Тухбатуллин М. С., Кирпичников А. П., Ляшева С. А., Шлеймович М. П. Распознавание динамических жестов на основе вычитания фона // Вестник Казан. технолог. ун-та. 2016. № 18. URL: https://cyberleninka.ru/article/n/raspoznavaniedinamicheskih-zhestov-na-osnove-vychitaniya-fona (дата обращения: 15.11.2021).
3. MediaPipe Hands. URL: https://google.github.io/mediapipe/solutions/hands.html (дата обращения: 17.12.2021).
4. Zhang F., Bazarevsky V., Vakunov A. et al. Media-Pipe Hands: On-device Real-time Hand Tracking // arXiv. 2020. URL: https://arxiv.org/pdf/2006.pdf (дата обращения: 17.12.2021).
5. Lugaresi C., Tang J., Nash H. et al. MediaPipe: A Framework for Building Perception Pipelines // Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR). 2019. URL: https://arxiv.org/pdf/1906. 08172.pdf (дата обращения: 17.12.2021).
6. Маликов А. В. Адаптация структуры диагностической искусственной нейронной сети при появлении новых обучающих примеров // Тр. учеб. заведений связи. 2020. № 4. URL: https://cyberleninka.ru/article/n/adaptatsiya-struktury-diagnosticheskoy-iskusstvennoy-neyronnoy-setipri-poyavlenii-novyh-obuchayuschih-primerov (дата обращения: 20.02.2022).
7. Тормозов В. С., Василенко К. А., Золкин А. Л. Настройка и обучение многослойного персептрона для задачи выделения дорожного покрытия на космических снимках города // Программные продукты и системы. 2020. № 2. URL: https://cyberleninka.ru/article/n/nastroyka-i-obucheniemnogosloynogo-perseptrona-dlya-zadachi vydeleniyadorozhnogo-pokrytiya-na-kosmicheskih-snimkah-goroda (дата обращения: 20.02.2022).
8. Graph API. URL: https://docs.opencv.org/4.x/d0/d1e/gapi.html (дата обращения: 20.02.2022).
Review
For citations:
Semenova V.O., Semenova L.L. DEVELOPING A SOFTWARE MODULE FOR TRANSLATING WORDS FROM SIGN LANGUAGE INTO AN AUDIO FORMAT. Proceedings in Cybernetics. 2022;(1 (45)):46-54. (In Russ.) https://doi.org/10.34822/1999-7604-2022-1-46-54
Views:
242
Email this article 