Yang Gao

🎉 Our Paper Accepted to CHI 2026!

Sat, 17 Jan 2026 00:00:00 +0000

Table of Contents

Overview

Stay tuned for the full paper and presentation details.

Wrist2Finger: Sensing Fingertip Force for Force-Aware Hand Interaction with a Ring-Watch Wearable

Wed, 01 Oct 2025 00:00:00 +0000

KineticsSense: A Multimodal Wearable Sensor Framework for Modeling Lower-Limb Motion Kinetics

Wed, 03 Sep 2025 00:00:00 +0000

Motion2Press: Cross Model Learning from IMU to Plantar Pressure for Gait Analysis

Wed, 03 Sep 2025 00:00:00 +0000

KineticsSense: A Multimodal Wearable Sensor Framework for Modeling Lower-Limb Motion Kinetics

Mon, 01 Sep 2025 00:00:00 +0000

Motion2Press: Cross Model Learning from IMU to Plantar Pressure for Gait Analysis

Mon, 01 Sep 2025 00:00:00 +0000

Family Moments

Sun, 06 Jul 2025 00:00:00 +0000

It’s always wonderful to spend quality time with my family. Here are a few highlights from our precious time together:

I cherish these memories and look forward to many more!

🎉 Two Papers Newly Accepted for UbiComp 2025 Presentation!

Sat, 05 Jul 2025 00:00:00 +0000

Table of Contents

Overview

Stay tuned for the full paper and presentation details.

🎉 Our Paper Accepted to UIST 2025!

Fri, 04 Jul 2025 00:00:00 +0000

Table of Contents

Overview

Stay tuned for the full paper and presentation details.

EchoTouch: Low-power Face-touching Behavior Recognition Using Active Acoustic Sensing on Glasses

Sun, 01 Jun 2025 00:00:00 +0000

CHI 2025 & Exploring Yokohama, Japan

Tue, 29 Apr 2025 00:00:00 +0000

Attending CHI 2025 in Yokohama, Japan, was an absolutely incredible experience – a perfect blend of diving deep into the latest Human-Computer Interaction research and exploring the captivating beauty of Japan. It was a week filled with inspiring talks, engaging discussions, and unforgettable cultural moments.

The Conference Experience 🔬🤝

CHI 2025, as always, was a powerhouse of innovation and a fantastic platform for presenting cutting-edge research.

I had the honor of presenting our work, “SandTouch: Empowering Virtual Sand Art in VR with AI Guidance and Emotional Relief,” which was met with great interest and engaging questions. It was truly rewarding to share our findings and receive valuable feedback from the community.

Presenting ‘SandTouch’ at CHI 2025.

The conference featured a diverse range of talks, from fundamental research to innovative applications. Listening to fellow researchers share their groundbreaking work was truly inspiring. I particularly enjoyed connecting with researchers working on similar topics. Here’s a glimpse of one of the engaging presentations:

An insightful presentation on ‘Conveying Dancers’ Expressiveness Through A Haptic-Installed Contemporary Dance Performance’.

The conference venue itself was impressive, offering a conducive environment for learning and networking.

The sleek architecture of the CHI 2025 conference center.

Exploring Yokohama’s Charms 🌃🇯🇵

Beyond the conference halls, Yokohama proved to be a vibrant and charming city, offering a wonderful backdrop for unwinding and exploring. Its beautiful waterfront, modern architecture, and serene streets made for a delightful experience.

I took the chance to visit the Nissan Gallery, where cutting-edge automotive design meets an immersive brand experience. The displays were truly impressive!

A peek into the future of automotive design at the Nissan Gallery.

Strolling through Yokohama’s streets, I was captivated by the city’s natural beauty, including the vibrant flora that lines its avenues.

Beautiful white azaleas blooming along a Yokohama street.

This trip was a fantastic opportunity to combine academic growth with cultural immersion. I’m already looking forward to the next CHI!

Did you find this page helpful? Consider sharing it 🙌

🎉 Double Acceptance at CHI 2025, One Full Paper & Late-Breaking Work!

Fri, 25 Apr 2025 00:00:00 +0000

Table of Contents

Overview

For more in-depth information about these exciting works, please visit their dedicated project pages:

SandTouch: Explore our full paper here
From Wrist to Finger: Discover our Late-Breaking Work here

From Wrist to Finger: Hand Pose Tracking Using Ring-Watch Wearables

Fri, 25 Apr 2025 00:00:00 +0000

Project Overview (CHI 2025 Late-Breaking Work)

Hand pose tracking is a cornerstone for advancing human-computer interaction applications, from virtual reality to prosthetics control. However, existing vision-based systems and wearable devices often face limitations in portability, usability, and practicality. In this CHI 2025 Late-Breaking Work, we introduce a novel multimodal hand pose tracking framework that integrates data from an IMU-equipped ring and EMG sensors embedded in a wrist-worn device.

This work represents an important step towards enabling more practical and accessible hand tracking solutions for everyday use.

Key Innovations & Progress:

1. Novel Ring-Watch Wearable Design

We propose a compact and ergonomic wearable system that combines a single IMU-equipped ring (worn on the thumb) with EMG sensors integrated into a smartwatch. This design prioritizes wearability and comfort, addressing the common limitations of bulky or multi-device tracking systems while accurately capturing intricate finger and hand motion data.

2. Multi-Sensor Fusion for Precise Tracking

Our framework leverages the complementary strengths of both motion dynamics (from the IMU) and muscle activity (from the EMG sensors). This deep learning-based sensor fusion approach achieves precise 3D hand pose reconstruction, providing robust performance even in complex or high-speed gestures. We developed a transformer-based model with time encoding and cross-modal attention mechanisms for optimal data integration.

Media & Resources:

Paper: Extended Abstract (ACM DL)
Video: Project Video

Did you find this page helpful? Consider sharing it 🙌

SandTouch: Empowering Virtual Sand Art in VR with AI Guidance and Emotional Relief

Fri, 25 Apr 2025 00:00:00 +0000

Project Overview

In today’s data-rich world, simply having data isn’t enough – we need to understand it, communicate it, and extract insights from it. This course empowers you to transform raw numbers and complex datasets into compelling, insightful visual narratives. We’ll blend art with science, learning how to design interactive visualizations that not only look good but also enable powerful data exploration and decision-making. Get ready to become a data storyteller!

Sand painting is a unique and valuable art form, but it’s often constrained by physical equipment and a steep learning curve. To address these challenges, we developed SandTouch, a novel VR sand painting system that offers an immersive and intuitive experience, closely mirroring real-world sand manipulation. This project was presented at CHI 2025 (the premier conference in Human-Computer Interaction).

Key Features & Contributions:

1. Realistic Hand-Sand Interaction

We designed SandTouch to create a highly realistic and natural interaction between the user’s hands and virtual sand, allowing direct manipulation without external devices or controllers. This approach provides an intuitive, device-free interface and emphasizes the authenticity of the interaction. Our system captures fine sensations of real sand manipulation, complemented by realistic sound feedback.

2. AI Guidance for Creative Flow

A pioneering aspect of SandTouch is the integration of an AI agent, powered by a large language model (LLM). This AI intelligently interprets users’ creative intentions in real-time, offering contextually relevant artistic suggestions. This feature simplifies the creative process, enhances interactivity, and helps users (especially beginners) refine their artwork through technique recommendations and composition analysis.

3. Emotional Relief & Immersion

Beyond artistic creation, SandTouch prioritizes user well-being. It incorporates calming and responsive soundscapes that react to user gestures, reinforcing a relaxing atmosphere. Comprehensive evaluations demonstrated a significant increase in user engagement and immersion, with the realistic sound feedback enhancing emotional relief and deepening the painting experience. This highlights its potential for therapeutic applications.

Media & Resources:

Paper: Full Paper (ACM DL)
Video: Project Video

Did you find this page helpful? Consider sharing it 🙌

From Wrist to Finger: Hand Pose Tracking Using Ring-Watch Wearables

Tue, 01 Apr 2025 00:00:00 +0000

SandTouch: Empowering Virtual Sand Art in VR with AI Guidance and Emotional Relief

Wed, 01 Jan 2025 00:00:00 +0000

Wrist2Finger: Sensing Fingertip Force for Force-Aware Hand Interaction with a Ring-Watch Wearable

Wed, 01 Jan 2025 00:00:00 +0000

🎉 New Paper Accepted to UbiComp 2025 (IMWUT 2024 May Cycle)!

Thu, 21 Nov 2024 00:00:00 +0000

Table of Contents

Overview

For more in-depth information about this exciting work, please visit the dedicated project page:

PressInPose: Explore our full paper here

PressInPose: Integrating Pressure and Inertial Sensors for Full-Body Pose Estimation in Activities

Thu, 21 Nov 2024 00:00:00 +0000

Project Overview

Accurate human body posture assessment through wearable technology has significant implications across various fields, including sports science, clinical diagnostics, rehabilitation, and VR interaction. Traditional methods often face limitations due to complex setups or environmental constraints. To address these challenges, we developed PressInPose, an innovative system that integrates pressure and inertial sensors for precise full-body pose estimation in dynamic activities. This work was published in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) and will be presented at UbiComp 2025.

Key Innovations & Contributions:

1. Novel Multi-Sensor Fusion

PressInPose employs an advanced shoe insole embedded with pressure sensors and an Inertial Measurement Unit (IMU), coupled with a single wrist-mounted IMU. This unique multi-modal sensor fusion approach allows for a comprehensive analysis of human biomechanics, capturing intricate body dynamics that traditional single-sensor systems often miss.

2. LLM-Powered Virtual Data Augmentation

To enhance the robustness and generalization of our system, we leveraged large language models (LLMs) to generate virtual human motion sequences. These sequences were utilized to create synthetic IMU data for data augmentation, effectively addressing the challenge of limited real-world data availability and variability, especially for complex and dynamic movements.

3. Physical Kinematics Modeling & Deep Learning Network

Our approach uniquely combines physical kinematics modeling based on pressure data with a multi-region human posture estimation network. This integration allows PressInPose to accurately capture interactions and dependencies between different body parts, leading to superior accuracy in pose reconstruction.

Media & Resources:

Paper: Full Paper (ACM DL)

Did you find this page helpful? Consider sharing it 🙌

PressInPose: Integrating Pressure and Inertial Sensors for Full-Body Pose Estimation in Activities

Fri, 01 Nov 2024 00:00:00 +0000

Data Visualization & Interaction (Offered Every Fall)

Sun, 01 Sep 2024 00:00:00 +0000

In our increasingly data-rich world, simply having data isn’t enough – we need to understand it, communicate it effectively, and extract actionable insights from it. This course empowers you to transform raw numbers and complex datasets into compelling, insightful visual narratives. We’ll blend the art of design with the science of data, learning how to create interactive visualizations that not only look good but also enable powerful data exploration and decision-making. Get ready to become a data storyteller!

What You’ll Explore:

1. Visual Perception & Cognition 👁️🧠

The human element is key! We’ll start by understanding how our eyes and brains process visual information, exploring principles of pre-attentive attributes, Gestalt psychology, and cognitive biases. This lays the groundwork for designing effective and intuitive visualizations.

2. Data Fundamentals & Preparation 🧹

Before we visualize, we clean and structure! Learn about different data types (quantitative, categorical, ordinal), data quality issues, and essential techniques for wrangling, cleaning, and transforming your data to make it suitable for visualization.

3. Visualizing Spatial Data 🗺️

Explore techniques for representing geographic locations and spatial relationships. From simple scatter plots on maps to choropleth maps, heatmaps, and flow maps, we’ll learn to convey location-based insights effectively.

4. Visualizing Tree Structures 🌲

Dive into hierarchical data! Learn how to effectively visualize relationships and structures in organizational charts, file systems, category taxonomies, and other tree-like datasets using methods like treemaps, sunburst charts, and node-link diagrams.

5. Visualizing Graph Structures (Networks) 🕸️

Uncover connections! Master techniques for visualizing complex networks, social graphs, co-occurrence relationships, and other graph data to reveal hidden patterns, communities, and central nodes.

6. Visualizing Time-Varying Data 📈

See trends unfold! Learn how to effectively represent changes over time, from traditional line charts and area charts to more advanced streamgraphs, sparklines, and animated timelines that show evolution and patterns over temporal dimensions.

7. Visualizing Text Data 💬

Transform words into insights! Explore methods for visualizing textual information, including word clouds, n-gram charts, sentiment analysis visualizations, topic models, and network graphs derived from text.

8. Geospatial Information Systems (GIS) & Advanced Mapping 🌍

Beyond basic maps, delve into the power of GIS for analyzing and visualizing complex geographical datasets with real-world applications in urban planning, environmental science, and public health.

9. Visualizing High-Dimensional Data ✨

Tame complexity! Learn strategies to visualize datasets with many attributes (dimensions), employing techniques like parallel coordinates, scatter plot matrices, dimensionality reduction (e.g., PCA, t-SNE), and glyph-based representations.

10. Interactive Views & Dashboard Design 🕹️

Bring your data to life! Explore principles of interactive design, enabling users to filter, zoom, pan, drill down, and explore data dynamically. We’ll also cover best practices for designing effective and user-friendly dashboards.

By the end of this course, you’ll not only be able to create beautiful and effective visualizations but also to communicate complex data stories with clarity and empower users to gain actionable insights. This is a critical skill for anyone navigating our data-driven world!

Did you find this page helpful? Consider sharing it 🙌

Projects

Sun, 19 May 2024 00:00:00 +0000

Computer Organization (Offered Every Spring)

Thu, 01 Feb 2024 00:00:00 +0000

Ever wondered what magic truly happens inside your computer when you click a mouse, launch an app, or even just boot it up? This course takes you on an exciting journey deep into the heart of modern computing systems. We’ll peel back the layers of abstraction to understand the fundamental building blocks and principles that make software run efficiently on hardware. Get ready to gain a profound appreciation for the intricate dance between bits, bytes, and transistors!

Syllabus:

1. Introduction to Computer Systems 🚀

An exciting overview of the computer’s anatomy – from high-level software to low-level hardware. We’ll set the stage for our adventure into the machine’s core, understanding the big picture before diving into the details.

2. Information Encoding & Data Representation 🔡

How do computers understand numbers, text, images, and sound? We’ll decode the secrets of binary, hexadecimal, and various data formats that form the computer’s native language. Learn how all digital information is just a series of bits!

3. Arithmetic Operations & The ALU ➕➖

Dive into the “brain” of the computer! Learn how the Arithmetic Logic Unit (ALU) performs lightning-fast calculations and logical operations, making complex computations possible. We’ll explore the algorithms behind addition, subtraction, multiplication, and division at the hardware level.

4. Memory Systems 🧠

From lightning-fast caches to vast hard drives, explore the hierarchy and magic of how computers store and retrieve information efficiently. Understand why some memory is faster than others and how virtual memory provides an illusion of infinite storage.

5. Instruction Set Architecture (ISA) ✍️

This is the “vocabulary” the CPU understands! We’ll learn how software instructions are designed and executed, bridging the gap between your high-level code and the low-level hardware. Discover the fundamental operations a processor can perform.

6. Control Unit Design 🚦

The orchestrator of the entire system! Discover how the control unit directs the flow of data and instructions, ensuring every component plays its part in perfect harmony. It’s the conductor of the CPU’s orchestra.

7. Input/Output (I/O) Systems 🌐

How do computers talk to the outside world? We’ll explore the mechanisms that allow your computer to interact seamlessly with keyboards, screens, networks, and other peripheral devices, managing their diverse speeds and requirements.

8. Bus Structures 🛣️

The digital highways connecting all the internal components! Understand how data travels reliably and efficiently between the CPU, memory, and I/O devices, and the different types of buses that facilitate this communication.

By the end of this course, you’ll not only understand how computers work at their core but also why they are designed the way they are. This foundational knowledge is essential for anyone aspiring to build, optimize, or secure computing systems!

Did you find this page helpful? Consider sharing it 🙌

Experience

Tue, 24 Oct 2023 00:00:00 +0000

SignRing: Continuous American Sign Language Recognition Using IMU Rings and Virtual IMU Data

Fri, 01 Sep 2023 00:00:00 +0000

An End-to-End Energy-Efficient Approach for Intake Detection With Low Inference Time Using Wrist-Worn Sensor

Tue, 01 Aug 2023 00:00:00 +0000

SmartASL: \"Point-of-Care\" Comprehensive ASL Interpreter Using Wearables

Thu, 01 Jun 2023 00:00:00 +0000

EarCommand: \"Hearing\" Your Silent Speech Commands In Ear

Fri, 01 Jul 2022 00:00:00 +0000

PPGface: Like What You Are Watching? Earphones Can \"Feel\" Your Facial Expressions

Fri, 01 Jul 2022 00:00:00 +0000

Detecting Screen Presence with Activity-Oriented RGB Camera in Egocentric Videos

Tue, 01 Mar 2022 00:00:00 +0000

SonicFace: Tracking Facial Expressions Using a Commodity Microphone Array

Wed, 01 Dec 2021 00:00:00 +0000

SonicASL: An Acoustic-based Sign Language Gesture Recognizer Using Earphones

Tue, 01 Jun 2021 00:00:00 +0000

Voice In Ear: Spoofing-Resistant and Passphrase-Independent Body Sound Authentication

Mon, 01 Mar 2021 00:00:00 +0000

ThermoTag: A Hidden ID of 3D Printers for Fingerprinting and Watermarking

Fri, 01 Jan 2021 00:00:00 +0000

EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users

Tue, 01 Sep 2020 00:00:00 +0000

Heart Monitor Using Flexible Capacitive ECG Electrodes

Wed, 01 Jul 2020 00:00:00 +0000

EarEcho: Using Ear Canal Echo for Wearable Authentication

Sun, 01 Sep 2019 00:00:00 +0000

Watching and Safeguarding Your 3D Printer: Online Process Monitoring Against Cyber-Physical Attacks

Sat, 01 Sep 2018 00:00:00 +0000