From Wrist to Finger: Hand Pose Tracking Using Ring-Watch Wearables
Our ring-watch wearable system for hand pose tracking.Project Overview (CHI 2025 Late-Breaking Work)
Hand pose tracking is a cornerstone for advancing human-computer interaction applications, from virtual reality to prosthetics control. However, existing vision-based systems and wearable devices often face limitations in portability, usability, and practicality. In this CHI 2025 Late-Breaking Work, we introduce a novel multimodal hand pose tracking framework that integrates data from an IMU-equipped ring and EMG sensors embedded in a wrist-worn device.
This work represents an important step towards enabling more practical and accessible hand tracking solutions for everyday use.
Key Innovations & Progress:
1. Novel Ring-Watch Wearable Design
We propose a compact and ergonomic wearable system that combines a single IMU-equipped ring (worn on the thumb) with EMG sensors integrated into a smartwatch. This design prioritizes wearability and comfort, addressing the common limitations of bulky or multi-device tracking systems while accurately capturing intricate finger and hand motion data.
2. Multi-Sensor Fusion for Precise Tracking
Our framework leverages the complementary strengths of both motion dynamics (from the IMU) and muscle activity (from the EMG sensors). This deep learning-based sensor fusion approach achieves precise 3D hand pose reconstruction, providing robust performance even in complex or high-speed gestures. We developed a transformer-based model with time encoding and cross-modal attention mechanisms for optimal data integration.
Media & Resources:
- Paper: Extended Abstract (ACM DL)
- Video: Project Video
Did you find this page helpful? Consider sharing it ๐
