HapticPipette

HapticPipette: Seamless Capture of Object Compliance in Mixed Reality

Arata Jingu, Jianhui Yan, Maja Fehlberg, Roland Bennewitz, Jürgen Steimle

Augmented Humans International Conference 2026 · Okinawa, Japan

Figure 1: HapticPipette wearable setup, finger-based compliance capture, and assignment of captured responses in mixed reality — Figure 1. A wearable setup captures an object’s force–displacement response through touch, then lets the user assign the captured curve to objects in mixed reality.

Overview

HapticPipette lets people capture how a real object deforms under pressure and reuse that response when designing mixed-reality experiences. Inspired by pipetting and copy-and-paste interaction, it brings haptic capture into the same MR environment as the design task.

How it works

A finger-worn capacitive force sensor measures applied force, while a head-worn RGB-D camera tracks surface displacement.
Egocentric measurement and visual-inertial odometry support capture while the user moves, preserving non-linear force–displacement behavior and hysteresis.
The captured curve becomes an interactive element in MR that can be assigned to an object to specify its haptic behavior. Rendering that behavior is outside this paper’s contribution.

Evaluation & findings

Two user studies assessed measurement performance and interaction usability. The system captured the overall non-linear compliance profiles of 11 materials, including elastic hysteresis, and users found the interaction usable. Three applications explore virtual-object design, augmentation of physical objects, and remote sharing of haptic properties.

Takeaway

Capturing haptic properties through familiar finger movements can connect everyday materials directly to MR design workflows.

My contributions

Contributed to the interaction concept and application design.
Implemented the mixed-reality interaction system for Quest 3.
Supported integration of the wearable setup and communication between the computer and headset.

Publication

Arata Jingu, Jianhui Yan, Maja Fehlberg, Roland Bennewitz, Jürgen Steimle. HapticPipette: Seamless Capture of Object Compliance in Mixed Reality. Augmented Humans International Conference 2026 · Okinawa, Japan. DOI: 10.1145/3795011.3795022.