Intuitive 3D hand tracking and gesture recognition

In 2018, touchless interaction was a promising field but faced limitations with existing technologies. Cameras raised privacy concerns, radar based solutions were costly and lacked resolution, and other options weren't yet ready for real-world use. Nimbus sought to change this, aiming for a touchless solution that delivered performance and privacy without excessive cost.

Hand Position Tracking: Nimbus achieved precise hand position tracking in 3D space through careful analysis of the reflected IR light patterns. Custom circuit boards ensured microsecond-level precision in LED illumination timing, creating distinct reflections based on hand position. Combined with a carefully calibrated system, these reflections allowed for statistical triangulation to determine the hand's coordinates within the interaction zone. Techniques like Kalman and 1€ filtering were employed to ensure more stable hand position tracking that wouldn't be jittery or erratic.

Hand Gesture Identification: To recognize gestures, Nimbus relied on a multi-step process. First, signal processing techniques, including noise reduction and edge detection, helped isolate the hand's outline from the raw sensor data. Principal Component Analysis (PCA) was used to simplify these representations, making them more computationally efficient. Finally, a shallow Random Forest classifier, for its speed and accuracy, matched these features to a library of predefined hand gestures, enabling real-time gesture recognition.

Nimbus showcased reasonable accuracy while excelling in cost and privacy, opening doors for new applications. It offered intuitive hands-free control in environments like medical settings or AR/VR. This success demonstrated a refined approach to touchless interaction. Additionally, Nimbus was patented and showcased in CES 2018 as part of the smart car concept by Samsung, highlighting its real-world impact and potential.