Implementation of Contactless Interaction with Computer Interface Prototype Using Unity 3D

Abstract

The article presents the prototype implementation process of contactless user-machine interaction using the Unity 3D game engine. It describes the creation of system objects: five desktops, a drop-down menu bar, a taskbar, a welcome screen and a program window. To control the prototype using the Kinect, a number of scripts, which were originally developed by Microsoft Corporation, are used. These scripts allow recognizing and identifying all possible gestures in software. A set of classes that allow identifying pre-set user gesture and performing relevant action are also described. Objects animation parameters for a transition between different states, which can be changed by scripts, are described. The entire system of methods and classes is assembled into one piece to make control and verification functionality easier to manage. A password gesture is set as a certain unique movement by the user to start the user session. This prototype responds to six human gestures: active hand waving, one-time hand wave to the left, to the right, up, down and holding a hand still. Mouse’s motions that can be recognized as alternative user gestures are provided.