Human-Centered AI · Wearable Intelligence · Human-Robot Interaction
Associate Professor
South China University of Technology
Yang Gao is an Associate Professor in the School of Future Technology at South China University of Technology (SCUT). He is a member of the Human-Centered AI Lab within the School of Future Technology. Prior to his current role, he was a Research Professor at East China Normal University from September 2022 to August 2025 and a Postdoctoral Researcher at Northwestern University from 2021 to 2022. His research interests include Human-Computer Interaction, Human-Robot Interaction, Wearable Computing, and human-centered AI. He leads the Human-Computer Interaction group at SCUT. He has been selected for the Shanghai Leading Talent Program (Overseas Young Talents) and the Guangzhou Innovation and Entrepreneurship Young Top Talent Program, and serves as principal investigator of a Young Scientists Fund project supported by the National Natural Science Foundation of China. He currently serves as an Associate Editor of ACM IMWUT, a guest editor for special issues of multiple international journals, and an executive committee member of the CCF Technical Committees on Pervasive Computing and Human-Computer Interaction.
Students
I am looking for self-motivated Ph.D. and Master's students with backgrounds in Artificial Intelligence, Robotics, Human-Computer Interaction, wearable systems, sensing hardware, or circuit design.
Interested students are welcome to contact me at gaoyang2025 [AT] scut.edu.cn with a brief research statement and CV.
Updates
Two of our papers, “HoloHand: Bidirectional Motion–Language Modeling for Semantic Hand Interaction in Immersive Environments” and “EmbodiedRecall: A …
Our paper, “HMotionGPT: Aligning Hand Motions and Natural Language for Activity Understanding with Smart Rings,” has been accepted to ACM IMWUT and will be presented at …
Our paper, “Design of a Multimodal Companion Robot for Elderly Care: Enhancing Affective Intelligence with Context-Aware Enactments,” has been accepted to the 2026 …
We are organizing WearAgent 2026, the 1st International Workshop on Interactive AI for Personal Wearable Agents, co-located with UbiComp/ISWC 2026 in Shanghai.
We are excited to share that our team has three papers accepted to CHI 2026, including one main conference paper and two poster papers: RageSense, Touch Through Sound, and …
Two of our papers from the IMWUT February cycle, Motion2Press and KineticsSense, have been accepted and will be presented at UbiComp 2025. Both works advance wearable sensing for …
Our paper, “Wrist2Finger” has been conditional accepted to UIST 2025! Stay tuned for the full paper and presentation details.
We’re delighted to announce a double acceptance for CHI 2025, one full paper “SandTouch” and one Late-Breaking Work have both been accepted!
Background
Experience
Courses
Research
My research lies at the intersection of Human-Computer Interaction and Ubiquitous Computing. I study how wearable sensing, physiological signals, motion data, and multimodal large language models can support activity understanding, intent inference, affect-aware interaction, and human-robot collaboration.
Output
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026 Extended Abstracts)
Touch Through Sound explores ring-mounted auditory pseudo-haptics for material perception. The work investigates how sound feedback from a wearable ring can enrich touch experiences and help users perceive material qualities without requiring bulky haptic …
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026)
RageSense investigates emotion regulation in mobile gaming by combining acoustic sensing with LLM-based intervention. The project focuses on detecting rage-related affective states during gameplay and delivering context-aware support that can help users …
The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), Main Conference (accepted)
Persona-E^2 introduces a human-grounded dataset for studying how personality shapes emotional responses to textual events. The work supports research on emotion understanding, personalized affective modeling, and human-centered natural language processing by …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
HMotionGPT studies how smart-ring motion signals can be aligned with natural language to support activity understanding. The work explores a language-grounded representation of hand motion, enabling wearable systems to reason about fine-grained activities …
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026 Extended Abstracts)
EmoType-Zero studies adaptive fonts as a way to render speech emotion in text. Instead of treating text as emotionally neutral, the project explores how typography can communicate affective cues from speech and make emotional nuance more visible in written …
Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp Companion 2025), pp. 1209-1216
MindChat-R0 is a Chinese emotionally supportive dialogue model trained through reinforcement learning with structured emotional reasoning. The work introduces an Empathetic Chain-of-Thought framework and trains the model using reward signals designed to …
Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST 2025)
Hand pose tracking is essential for advancing applications in human-computer interaction. Current approaches, such as vision-based systems and wearable devices, face limitations in portability, usability, and practicality. We present a novel wearable system …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Human locomotion is a fundamental aspect of daily life and a critical indicator of overall health. This study proposes a cross-modal learning framework based on inertial measurement units (IMUs) to infer plantar pressure distribution, ground reaction force …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Current human motion analysis primarily emphasizes kinematics, often neglecting underlying biomechanical factors such as muscle activation and force generation. This restricts a comprehensive understanding of movement mechanics, particularly in rehabilitation …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Accurately recognizing face-touching behavior at any time and place can help prevent potential health risks and improve personal habits. However, there remains a lack of effective methods that can be applied in real-world scenarios. In this paper, we propose …
Extended Abstracts of the 2025 CHI Conference on Human Factors in Computing Systems (CHI 2025 Extended Abstracts)
Hand pose tracking is essential for advancing applications in humancomputer interaction. Current approaches, such as vision-based systems and wearable devices, face limitations in portability, usability, and practicality. This paper proposes a novel multimodal …
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI 2025)
Sand painting is a highly aesthetic and valuable form of art but often constrained by the need for specific equipment and the associated learning curve. To address these challenges, we developed a VR sand painting system, SandTouch, offering an immersive and …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
The accurate assessment of human body posture through wearable technology has significant implications for sports science, clinical diagnostics, rehabilitation, and VR interaction. Traditional methods often require complex setups or are limited by the …
CCF Transactions on Pervasive Computing and Interaction (TPCI), 6, 380-393
As the contradiction between the pursuit of health and the increasing duration of sedentary office work intensifies, there has been a growing focus on maintaining correct sitting posture while working. This paper introduces a sitting posture recognition system …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Sign language is a natural language widely used by Deaf and hard of hearing (DHH) individuals. Advanced wearables are developed to recognize sign language automatically. However, they are limited by the lack of labeled data, which leads to a small vocabulary …
IEEE Journal of Biomedical and Health Informatics (IEEE JBHI)
Automated detection of intake gestures with wearable sensors has been a critical area of research for advancing our understanding and ability to intervene in people's eating behavior. Numerous algorithms have been developed and evaluated in terms of accuracy. …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Sign language builds up an important bridge between the d/Deaf and hard-of-hearing (DHH) and hearing people. Regrettably, most hearing people face challenges in comprehending sign language, necessitating sign language translation. However, state-of-the-art …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Intelligent speech interfaces have been developing vastly to support the growing demands for convenient control and interaction with wearable/earable and portable devices. To avoid privacy leakage during speech interactions and strengthen the resistance to …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Recognition of facial expressions has been widely explored to represent people's emotional states. Existing facial expression recognition systems primarily rely on external cameras which make it less accessible and efficient in many real-life scenarios to …
2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)
Screen time is associated with several health risk behaviors including mindless eating, sedentary behavior, and decreased academic performance. Screen time behavior is traditionally assessed with self-report measures, which are known to be burdensome, …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Accurate recognition of facial expressions and emotional gestures is promising to understand the audience's feedback and engagement on the entertainment content. Existing methods are primarily based on various cameras or wearable sensors, which either raise …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
We propose SonicASL, a real-time gesture recognition system that can recognize sign language gestures on the fly, leveraging front-facing microphones and speakers added to commodity earphones worn by someone facing the person making the gestures. In a user …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
With the rapid growth of wearable computing and increasing demand for mobile authentication scenarios, voiceprint-based authentication has become one of the prevalent technologies and has already presented tremendous potentials to the public. However, it is …
IEEE Transactions on Information Forensics and Security (IEEE TIFS)
To address the increasing challenges of counterfeit detection and IP protection for 3D printing, we propose that every 3D printer holds unique fingerprinting features characterized by the thermodynamic properties of the extruder hot-end and can be used as a …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
With the rapid growth of artificial intelligence and mobile computing, intelligent speech interface has recently become one of the prevalent trends and has already presented huge potentials to the public. To address the privacy leakage issue during the speech …
IEEE Transactions on Instrumentation and Measurement (IEEE TIM)
Noninvasive sensors capable of measuring weak biopotential signals, such as electrocardiogram (ECG) and EEG, and communicating results wirelessly to a host computer are developing rapidly. Some of them utilize capacitively coupled electrodes in an attempt to …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
Smart wearable devices have recently become one of the major technological trends and been widely adopted by the general public. Wireless earphones, in particular, have seen a skyrocketing growth due to its great usability and convenience. With the goal of …
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT / UbiComp)
The increasing adoption of 3D printing in many safety and mission critical applications exposes 3D printers to a variety of cyber attacks that may result in catastrophic consequences if the printing process is compromised. For example, the mechanical …
Systems
An open-source multimodal framework that aligns smart-ring IMU signals with natural language for hand-centric activity understanding.
Explore our novel multimodal hand pose tracking system, integrating IMU-equipped rings and EMG sensors in smartwatches for precise 3D hand pose reconstruction.
Discover SandTouch, our innovative VR sand painting system that brings realistic hand-sand interaction, AI guidance, and emotional relief to virtual art creation.