arXiv subjects

Liuchuan Yu

Publications and source records attributed to Liuchuan Yu.

7 recordsLinked to original sources

arXiv2026

EgoKit: Towards Unified Low-Cost Egocentric Data Collection with Heterogeneous Devices

Egocentric video is increasingly used as a data source for robot learning, activity understanding, and embodied AI research, but collecting it at scale remains fragmented in practice: each candidate host device, such as an Android phone, iPhone, iPad, smart glasses, or extended reality (XR) headset, exposes a different SDK, a different policy on raw camera access, and different limitations on external USB cameras and on-device tracking. Synchronized ego-view and wrist-view capture is therefore typically obtained by either committing to a single proprietary platform or building one-off rigs that do not transfer across devices. To address this gap, we present EgoKit, a toolkit that exposes the same egocentric recording workflow across six heterogeneous host devices. Across all supported devices, EgoKit presents the same recording interaction and produces locally stored video with a uniform log format; on XR headsets, it additionally logs head pose and OpenXR-standard 26-joint hand tracking aligned to the video streams. The companion accessories, including two wrist cameras with mounts, a head strap, and a USB-C hub, add wrist-view capture to any supported host without custom hardware fabrication. EgoKit is available at \url{https://egokit.chuange.org/}.

cs.CV↗

arXiv2026

Moving Through Clutter: Scaling Data Collection and Benchmarking for 3D Scene-Aware Humanoid Locomotion via Virtual Reality

Recent advances in humanoid locomotion have enabled dynamic behaviors such as dancing, martial arts, and parkour, yet these capabilities are predominantly demonstrated in open, flat, and obstacle-free settings. In contrast, real-world environments such as homes, offices, and public spaces, are densely cluttered, three-dimensional, and geometrically constrained, requiring scene-aware whole-body coordination, precise balance control, and reasoning over spatial constraints imposed by furniture and household objects. However, humanoid locomotion in cluttered 3D environments remains underexplored, and no public dataset systematically couples full-body human locomotion with the scene geometry that shapes it. To address this gap, we present Moving Through Clutter (MTC), an opensource Virtual Reality (VR) based data collection and evaluation framework for scene-aware humanoid locomotion in cluttered environments. Our system procedurally generates scenes with controllable clutter levels and captures embodiment-consistent, whole-body human motion through immersive VR navigation, which is then automatically retargeted to a humanoid robot model. We further introduce benchmarks that quantify environment clutter level and locomotion performance, including stability and collision safety. Using this framework, we compile a dataset of 348 trajectories across 145 diverse 3D cluttered scenes. The dataset provides a foundation for studying geometry-induced adaptation in humanoid locomotion and developing scene-aware planning and control methods.

cs.RO↗

arXiv2026

Reality Copilot: Voice-First Human-AI Collaboration in Mixed Reality Using Large Multimodal Models

Large Multimodal Models (LMMs) have shown strong potential for assisting users in tasks, such as programming, content creation, and information access, yet their interaction remains largely limited to traditional interfaces such as desktops and smartphones. Meanwhile, advances in mixed reality (MR) hardware have enabled applications that extend beyond entertainment and into everyday use. However, most existing MR systems rely primarily on manual input (e.g., hand gestures or controllers) and provide limited intelligent assistance due to the lack of integration with large-scale AI models. We present Reality Copilot, a voice-first human-AI assistant for mixed reality that leverages LMMs to enable natural speech-based interaction. The system supports contextual understanding of physical environments, realistic 3D content generation, and real-time information retrieval. In addition to in-headset interaction, Reality Copilot facilitates cross-platform workflows by generating context-aware textual content and exporting generated assets. This work explores the design space of LMM-powered human-AI collaboration in mixed reality.

cs.HC↗

arXiv2025

You Are Not Alone: Designing Body Doubling for ADHD in Virtual Reality

Adults with Attention Deficit Hyperactivity Disorder (ADHD) experience challenges sustaining attention in the workplace. Body doubling, the concept of working alongside another person, has been proposed as a productivity aid for ADHD and other neurodivergent populations (NDs). However, prior work found no conclusive effectiveness and noted NDs' discomfort with social presence. This work investigates body doubling as an ADHD centered productivity strategy in construction tasks. In Study 1, we explored challenges ADHD workers face in construction and identified design insights. In Study 2, we implemented a virtual reality bricklaying task under three conditions: (C1) alone, (C2) with a human body double, and (C3) with an AI body double. Results from 12 participants show they finished tasks faster and perceived greater accuracy and sustained attention in C2 and C3 compared to C1. While body doubling was clearly preferred, opinions diverged between conditions. Our findings verify its effect and offer design implications for future interventions.

cs.HC↗

arXiv2025

XRoboToolkit: A Cross-Platform Framework for Robot Teleoperation

The rapid advancement of Vision-Language-Action models has created an urgent need for large-scale, high-quality robot demonstration datasets. Although teleoperation is the predominant method for data collection, current approaches suffer from limited scalability, complex setup procedures, and suboptimal data quality. This paper presents XRoboToolkit, a cross-platform framework for extended reality based robot teleoperation built on the OpenXR standard. The system features low-latency stereoscopic visual feedback, optimization-based inverse kinematics, and support for diverse tracking modalities including head, controller, hand, and auxiliary motion trackers. XRoboToolkit's modular architecture enables seamless integration across robotic platforms and simulation environments, spanning precision manipulators, mobile robots, and dexterous hands. We demonstrate the framework's effectiveness through precision manipulation tasks and validate data quality by training VLA models that exhibit robust autonomous performance.

cs.RO↗

arXiv2025

Enriching physical-virtual interaction in AR gaming by tracking identical objects via an egocentric partial observation frame

Augmented reality (AR) games, particularly those designed for head-mounted displays, have grown increasingly prevalent. However, most existing systems depend on pre-scanned, static environments and rely heavily on continuous tracking or marker-based solutions, which limit adaptability in dynamic physical spaces. This is particularly problematic for AR headsets and glasses, which typically follow the user's head movement and cannot maintain a fixed, stationary view of the scene. Moreover, continuous scene observation is neither power-efficient nor practical for wearable devices, given their limited battery and processing capabilities. A persistent challenge arises when multiple identical objects are present in the environment-standard object tracking pipelines often fail to maintain consistent identities without uninterrupted observation or external sensors. These limitations hinder fluid physical-virtual interactions, especially in dynamic or occluded scenes where continuous tracking is infeasible. To address this, we introduce a novel optimization-based framework for re-identifying identical objects in AR scenes using only one partial egocentric observation frame captured by a headset. We formulate the problem as a label assignment task solved via integer programming, augmented with a Voronoi diagram-based pruning strategy to improve computational efficiency. This method reduces computation time by 50% while preserving 91% accuracy in simulated experiments. Moreover, we evaluated our approach in quantitative synthetic and quantitative real-world experiments. We also conducted three qualitative real-world experiments to demonstrate the practical utility and generalizability for enabling dynamic, markerless object interaction in AR environments. Our video demo is available at https://youtu.be/RwptEfLtW1U.

cs.HC↗

arXiv2024

Col-Con: A Virtual Reality Simulation Testbed for Exploring Collaborative Behaviors in Construction

Virtual reality is widely adopted for applications such as training, education, and collaboration. The construction industry, known for its complex projects and numerous personnel involved, relies heavily on effective collaboration. Setting up a real-world construction site for experiments can be expensive and time-consuming, whereas conducting experiments in VR is relatively low-cost, scalable, and efficient. We propose Col-Con, a virtual reality simulation testbed for exploring collaborative behaviors in construction. Col-Con is a multi-user testbed that supports users in completing tasks collaboratively. Additionally, Col-Con provides immersive and realistic simulated construction scenes, where real-time voice communication, along with synchronized transformations, animations, sounds, and interactions, enhances the collaborative experience. As a showcase, we implemented a pipe installation construction task based on Col-Con. A user study demonstrated that Col-Con excels in usability, and participants reported a strong sense of immersion and collaboration. We envision that Col-Con will facilitate research on exploring virtual reality-based collaborative behaviors in construction.

cs.HC↗