arXiv Science⌕ Search

arXiv subjects

Xueyu Hou

Publications and source records attributed to Xueyu Hou.

3 recordsLinked to original sources

Why Today's Internet Cannot Support Tomorrow's Immersive Experiences: Insights from the NSF Workshop

Immersive computing is poised to become a major paradigm for future human-computer interaction (HCI), enabling spatial and embodied experiences across applications such as education, healthcare, manufacturing, emergency response, and scientific discovery. However, today's Internet and computing infrastructure were not designed for real-time, multi-user, sensor-rich, and privacy-sensitive immersive applications. They require coordination across sensing, representation, compression, networking, edge/cloud computing, rendering, security/privacy, and human experience. This report summarizes key findings from the NSF Workshop on Networking and Systems Challenges in Immersive Computing. The workshop brought together researchers from networking, systems, extended reality (XR), artificial intelligence (AI), graphics, HCI, security, privacy, and applications to identify challenges and opportunities. We conclude that immersive computing cannot be enabled by isolated advances in network bandwidth, device capabilities, AI models, or rendering alone. The field needs end-to-end architectures that adapt to user behavior, application requirements, resource constraints, and social expectations. We organize the agenda into six themes: networked systems for immersive computing; real-time edge-cloud orchestration for XR applications; AI for immersive experiences; privacy, security, and trust in XR systems; human-centered design and user experience in XR; and scalable infrastructure, testbeds, and open ecosystems for XR. We further identify key research questions, future trends, risks, ten-year success metrics, and action items for academia, industry, government, and research community. The report calls for coordinated investment in open platforms, reproducible benchmarks, privacy-preserving systems, and human-centered evaluation to build trustworthy, scalable, and broadly impactful infrastructure.

cs.NI↗

DeepMix: Mobility-aware, Lightweight, and Hybrid 3D Object Detection for Headsets

Mobile headsets should be capable of understanding 3D physical environments to offer a truly immersive experience for augmented/mixed reality (AR/MR). However, their small form-factor and limited computation resources make it extremely challenging to execute in real-time 3D vision algorithms, which are known to be more compute-intensive than their 2D counterparts. In this paper, we propose DeepMix, a mobility-aware, lightweight, and hybrid 3D object detection framework for improving the user experience of AR/MR on mobile headsets. Motivated by our analysis and evaluation of state-of-the-art 3D object detection models, DeepMix intelligently combines edge-assisted 2D object detection and novel, on-device 3D bounding box estimations that leverage depth data captured by headsets. This leads to low end-to-end latency and significantly boosts detection accuracy in mobile scenarios. A unique feature of DeepMix is that it fully exploits the mobility of headsets to fine-tune detection results and boost detection accuracy. To the best of our knowledge, DeepMix is the first 3D object detection that achieves 30 FPS (an end-to-end latency much lower than the 100 ms stringent requirement of interactive AR/MR). We implement a prototype of DeepMix on Microsoft HoloLens and evaluate its performance via both extensive controlled experiments and a user study with 30+ participants. DeepMix not only improves detection accuracy by 9.1--37.3% but also reduces end-to-end latency by 2.68--9.15x, compared to the baseline that uses existing 3D object detection models.

cs.CV↗

DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices

As the number of edge devices with computing resources (e.g., embedded GPUs, mobile phones, and laptops) increases, recent studies demonstrate that it can be beneficial to collaboratively run convolutional neural network (CNN) inference on more than one edge device. However, these studies make strong assumptions on the devices' conditions, and their application is far from practical. In this work, we propose a general method, called DistrEdge, to provide CNN inference distribution strategies in environments with multiple IoT edge devices. By addressing heterogeneity in devices, network conditions, and nonlinear characters of CNN computation, DistrEdge is adaptive to a wide range of cases (e.g., with different network conditions, various device types) using deep reinforcement learning technology. We utilize the latest embedded AI computing devices (e.g., NVIDIA Jetson products) to construct cases of heterogeneous devices' types in the experiment. Based on our evaluations, DistrEdge can properly adjust the distribution strategy according to the devices' computing characters and the network conditions. It achieves 1.1 to 3x speedup compared to state-of-the-art methods.

cs.DC↗