arXiv subjects

Tianyi Zhang

Publications and source records attributed to Tianyi Zhang.

At least 217 records · Page 12Linked to original sources

arXiv2025

Mean of Means: A 10-dollar Solution for Human Localization with Calibration-free and Unconstrained Camera Settings

Accurate human localization is crucial for various applications, especially in the Metaverse era. Existing high precision solutions rely on expensive, tag-dependent hardware, while vision-based methods offer a cheaper, tag-free alternative. However, current vision solutions based on stereo vision face limitations due to rigid perspective transformation principles and error propagation in multi-stage SVD solvers. These solutions also require multiple high-resolution cameras with strict setup constraints. To address these limitations, we propose a probabilistic approach that considers all points on the human body as observations generated by a distribution centered around the body's geometric center. This enables us to improve sampling significantly, increasing the number of samples for each point of interest from hundreds to billions. By modeling the relation between the means of the distributions of world coordinates and pixel coordinates, leveraging the Central Limit Theorem, we ensure normality and facilitate the learning process. Experimental results demonstrate human localization accuracy of 95% within a 0.3m range and nearly 100% accuracy within a 0.5m range, achieved at a low cost of only 10 USD using two web cameras with a resolution of 640x480 pixels.

cs.CV↗

arXiv2025

Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video

We aim to redefine robust ego-motion estimation and photorealistic 3D reconstruction by addressing a critical limitation: the reliance on noise-free data in existing models. While such sanitized conditions simplify evaluation, they fail to capture the unpredictable, noisy complexities of real-world environments. Dynamic motion, sensor imperfections, and synchronization perturbations lead to sharp performance declines when these models are deployed in practice, revealing an urgent need for frameworks that embrace and excel under real-world noise. To bridge this gap, we tackle three core challenges: scalable data generation, comprehensive benchmarking, and model robustness enhancement. First, we introduce a scalable noisy data synthesis pipeline that generates diverse datasets simulating complex motion, sensor imperfections, and synchronization errors. Second, we leverage this pipeline to create Robust-Ego3D, a benchmark rigorously designed to expose noise-induced performance degradation, highlighting the limitations of current learning-based methods in ego-motion accuracy and 3D reconstruction quality. Third, we propose Correspondence-guided Gaussian Splatting (CorrGS), a novel test-time adaptation method that progressively refines an internal clean 3D representation by aligning noisy observations with rendered RGB-D frames from clean 3D map, enhancing geometric alignment and appearance restoration through visual correspondence. Extensive experiments on synthetic and real-world data demonstrate that CorrGS consistently outperforms prior state-of-the-art methods, particularly in scenarios involving rapid motion and dynamic illumination.

cs.CV↗

arXiv2025

Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation

There is no limit to how much a robot might explore and learn, but all of that knowledge needs to be searchable and actionable. Within language research, retrieval augmented generation (RAG) has become the workhorse of large-scale non-parametric knowledge; however, existing techniques do not directly transfer to the embodied domain, which is multimodal, where data is highly correlated, and perception requires abstraction. To address these challenges, we introduce Embodied-RAG, a framework that enhances the foundational model of an embodied agent with a non-parametric memory system capable of autonomously constructing hierarchical knowledge for both navigation and language generation. Embodied-RAG handles a full range of spatial and semantic resolutions across diverse environments and query types, whether for a specific object or a holistic description of ambiance. At its core, Embodied-RAG's memory is structured as a semantic forest, storing language descriptions at varying levels of detail. This hierarchical organization allows the system to efficiently generate context-sensitive outputs across different robotic platforms. We demonstrate that Embodied-RAG effectively bridges RAG to the robotics domain, successfully handling over 250 explanation and navigation queries across kilometer-level environments, highlighting its promise as a general-purpose non-parametric system for embodied agents.

cs.RO↗

arXiv2025

ModCube: Modular, Self-Assembling Cubic Underwater Robot

This paper presents a low-cost, centralized modular underwater robot platform, ModCube, which can be used to study swarm coordination for a wide range of tasks in underwater environments. A ModCube structure consists of multiple ModCube robots. Each robot can move in six DoF with eight thrusters and can be rigidly connected to other ModCube robots with an electromagnet controlled by onboard computer. In this paper, we present a novel method for characterizing and visualizing dynamic behavior, along with four benchmarks to evaluate the morphological performance of the robot. Analysis shows that our ModCube design is desirable for omnidirectional tasks, compared with the configurations widely used by commercial underwater robots. We run real robot experiments in two water tanks to demonstrate the robust control and self-assemble of the proposed system, We also open-source the design and code to facilitate future research.

cs.RO↗

arXiv2024

Non-perturbative cathodoluminescence microscopy of beam-sensitive materials

Cathodoluminescence microscopy is now a well-established and powerful tool for probing the photonic properties of nanoscale materials, but in many cases, nanophotonic materials are easily damaged by the electron-beam doses necessary to achieve reasonable cathodoluminescence signal-to-noise ratios. Two-dimensional materials have proven particularly susceptible to beam-induced modifications, yielding both obstacles to high spatial-resolution measurement and opportunities for beam-induced patterning of quantum photonic systems. Here pan-sharpening techniques are applied to cathodoluminescence microscopy in order to address these challenges and experimentally demonstrate the promise of pan-sharpening for minimally-perturbative high-spatial-resolution spectrum imaging of beam-sensitive materials.

physics.optics↗

arXiv2024

MMDS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

We present MMDS, a system capable of recognizing medical images and patient facial details, and providing professional medical diagnoses. The system consists of two core components:The first component is the analysis of medical images and videos. We trained a specialized multimodal medical model capable of interpreting medical images and accurately analyzing patients' facial emotions and facial paralysis conditions. The model achieved an accuracy of 72.59% on the FER2013 facial emotion recognition dataset, with a 91.1% accuracy in recognizing the "happy" emotion. In facial paralysis recognition, the model reached an accuracy of 92%, which is 30% higher than that of GPT-4o. Based on this model, we developed a parser for analyzing facial movement videos of patients with facial paralysis, achieving precise grading of the paralysis severity. In tests on 30 videos of facial paralysis patients, the system demonstrated a grading accuracy of 83.3%.The second component is the generation of professional medical responses. We employed a large language model, integrated with a medical knowledge base, to generate professional diagnoses based on the analysis of medical images or videos. The core innovation lies in our development of a department-specific knowledge base routing management mechanism, in which the large language model categorizes data by medical departments and, during the retrieval process, determines the appropriate knowledge base to query. This significantly improves retrieval accuracy in the RAG (retrieval-augmented generation) process.

cs.CV↗

arXiv2024

Effective Two-Stage Knowledge Transfer for Multi-Entity Cross-Domain Recommendation

In recent years, the recommendation content on e-commerce platforms has become increasingly rich -- a single user feed may contain multiple entities, such as selling products, short videos, and content posts. To deal with the multi-entity recommendation problem, an intuitive solution is to adopt the shared-network-based architecture for joint training. The idea is to transfer the extracted knowledge from one type of entity (source entity) to another (target entity). However, different from the conventional same-entity cross-domain recommendation, multi-entity knowledge transfer encounters several important issues: (1) data distributions of the source entity and target entity are naturally different, making the shared-network-based joint training susceptible to the negative transfer issue, (2) more importantly, the corresponding feature schema of each entity is not exactly aligned (e.g., price is an essential feature for selling product while missing for content posts), making the existing methods no longer appropriate. Recent researchers have also experimented with the pre-training and fine-tuning paradigm. Again, they only consider the scenarios with the same entity type and feature systems, which is inappropriate in our case. To this end, we design a pre-training & fine-tuning based Multi-entity Knowledge Transfer framework called MKT. MKT utilizes a multi-entity pre-training module to extract transferable knowledge across different entities. In particular, a feature alignment module is first applied to scale and align different feature schemas. Afterward, a couple of knowledge extractors are employed to extract the common and entity-specific knowledge. In the end, the extracted common knowledge is adopted for target entity model training. Through extensive offline and online experiments, we demonstrated the superiority of MKT over multiple State-Of-The-Art methods.

cs.IR↗

arXiv2024

"I Don't Use AI for Everything": Exploring Utility, Attitude, and Responsibility of AI-empowered Tools in Software Development

AI-empowered tools have emerged as a transformative force, fundamentally reshaping the software development industry and promising far-reaching impacts across diverse sectors. This study investigates the adoption, impact, and security considerations of AI-empowered tools in the software development process. Through semi-structured interviews with 19 software practitioners from diverse backgrounds, we explore three key aspects: the utility of AI tools, developers' attitudes towards them, and security and privacy responsibilities. Our findings reveal widespread adoption of AI tools across various stages of software development. Developers generally express positive attitudes towards AI, viewing it as an efficiency-enhancing assistant rather than a job replacement threat. However, they also recognized limitations in AI's ability to handle complex, unfamiliar, or highly specialized tasks in software development. Regarding security and privacy, we found varying levels of risk awareness among developers, with larger companies implementing more comprehensive risk management strategies. Our study provides insights into the current state of AI adoption in software development and offers recommendations for practitioners, organizations, AI providers, and regulatory bodies to effectively navigate the integration of AI in the software industry.

cs.SE↗

arXiv2024

Robust State Estimation for Legged Robots with Dual Beta Kalman Filter

Existing state estimation algorithms for legged robots that rely on proprioceptive sensors often overlook foot slippage and leg deformation in the physical world, leading to large estimation errors. To address this limitation, we propose a comprehensive measurement model that accounts for both foot slippage and variable leg length by analyzing the relative motion between foot contact points and the robot's body center. We show that leg length is an observable quantity, meaning that its value can be explicitly inferred by designing an auxiliary filter. To this end, we introduce a dual estimation framework that iteratively employs a parameter filter to estimate the leg length parameters and a state filter to estimate the robot's state. To prevent error accumulation in this iterative framework, we construct a partial measurement model for the parameter filter using the leg static equation. This approach ensures that leg length estimation relies solely on joint torques and foot contact forces, avoiding the influence of state estimation errors on the parameter estimation. Unlike leg length which can be directly estimated, foot slippage cannot be measured directly with the current sensor configuration. However, since foot slippage occurs at a low frequency, it can be treated as outliers in the measurement data. To mitigate the impact of these outliers, we propose the beta Kalman filter (beta KF), which redefines the estimation loss in canonical Kalman filtering using beta divergence. This divergence can assign low weights to outliers in an adaptive manner, thereby enhancing the robustness of the estimation algorithm. These techniques together form the dual beta-Kalman filter (Dual beta KF), a novel algorithm for robust state estimation in legged robots. Experimental results on the Unitree GO2 robot demonstrate that the Dual beta KF significantly outperforms state-of-the-art methods.

cs.RO↗

arXiv2024

GraphRPM: Risk Pattern Mining on Industrial Large Attributed Graphs

Graph-based patterns are extensively employed and favored by practitioners within industrial companies due to their capacity to represent the behavioral attributes and topological relationships among users, thereby offering enhanced interpretability in comparison to black-box models commonly utilized for classification and recognition tasks. For instance, within the scenario of transaction risk management, a graph pattern that is characteristic of a particular risk category can be readily employed to discern transactions fraught with risk, delineate networks of criminal activity, or investigate the methodologies employed by fraudsters. Nonetheless, graph data in industrial settings is often characterized by its massive scale, encompassing data sets with millions or even billions of nodes, making the manual extraction of graph patterns not only labor-intensive but also necessitating specialized knowledge in particular domains of risk. Moreover, existing methodologies for mining graph patterns encounter significant obstacles when tasked with analyzing large-scale attributed graphs. In this work, we introduce GraphRPM, an industry-purpose parallel and distributed risk pattern mining framework on large attributed graphs. The framework incorporates a novel edge-involved graph isomorphism network alongside optimized operations for parallel graph computation, which collectively contribute to a considerable reduction in computational complexity and resource expenditure. Moreover, the intelligent filtration of efficacious risky graph patterns is facilitated by the proposed evaluation metrics. Comprehensive experimental evaluations conducted on real-world datasets of varying sizes substantiate the capability of GraphRPM to adeptly address the challenges inherent in mining patterns from large-scale industrial attributed graphs, thereby underscoring its substantial value for industrial deployment.

cs.LG↗

arXiv2024

AWARE Narrator and the Utilization of Large Language Models to Extract Behavioral Insights from Smartphone Sensing Data

Smartphones, equipped with an array of sensors, have become valuable tools for personal sensing. Particularly in digital health, smartphones facilitate the tracking of health-related behaviors and contexts, contributing significantly to digital phenotyping, a process where data from digital interactions is analyzed to infer behaviors and assess mental health. Traditional methods process raw sensor data into information features for statistical and machine learning analyses. In this paper, we introduce a novel approach that systematically converts smartphone-collected data into structured, chronological narratives. The AWARE Narrator translates quantitative smartphone sensing data into English language descriptions, forming comprehensive narratives of an individual's activities. We apply the framework to the data collected from university students over a week, demonstrating the potential of utilizing the narratives to summarize individual behavior, and analyzing psychological states by leveraging large language models.

cs.HC↗

arXiv2024

Context-Aware Token Selection and Packing for Enhanced Vision Transformer

In recent years, the long-range attention mechanism of vision transformers has driven significant performance breakthroughs across various computer vision tasks. However, the traditional self-attention mechanism, which processes both informative and non-informative tokens, suffers from inefficiency and inaccuracies. While sparse attention mechanisms have been introduced to mitigate these issues by pruning tokens involved in attention, they often lack context-awareness and intelligence. These mechanisms frequently apply a uniform token selection strategy across different inputs for batch training or optimize efficiency only for the inference stage. To overcome these challenges, we propose a novel algorithm: Select and Pack Attention (SPA). SPA dynamically selects informative tokens using a low-cost gating layer supervised by selection labels and packs these tokens into new batches, enabling a variable number of tokens to be used in parallelized GPU batch training and inference. Extensive experiments across diverse datasets and computer vision tasks demonstrate that SPA delivers superior performance and efficiency, including a 0.6 mAP improvement in object detection and a 16.4% reduction in computational costs.

cs.CV↗

arXiv2024

AraSync: Precision Time Synchronization in Rural Wireless Living Lab

Time synchronization is a critical component in network operation and management, and it is also required by Ultra-Reliable, Low-Latency Communications (URLLC) in next-generation wireless systems such as those of 5G, 6G, and Open RAN. In this context, we design and implement AraSync as an end-to-end time synchronization system in the ARA wireless living lab to enable advanced wireless experiments and applications involving stringent time constraints. We make use of Precision Time Protocol (PTP) at different levels to achieve synchronization accuracy in the order of nanoseconds. Along with fiber networks, AraSync enables time synchronization across the AraHaul wireless x-haul network consisting of long-range, high-capacity mmWave and microwave links. In this paper, we present the detailed design and implementation of AraSync, including its hardware and software components and the PTP network topology. Further, we experimentally characterize the performance of AraSync from spatial and temporal dimensions. Our measurement and analysis of the clock offset and mean path delay show the impact of the wireless channel and weather conditions on the PTP synchronization accuracy.

cs.NI↗

arXiv2024

Nearly Optimal Dynamic Set Cover: Breaking the Quadratic-in-$f$ Time Barrier

The dynamic set cover problem has been subject to extensive research since the pioneering works of [Bhattacharya et al, 2015] and [Gupta et al, 2017]. The input is a set system $(U, S)$ on a fixed collection $S$ of sets and a dynamic universe of elements, where each element appears in a most $f$ sets and the cost of each set lies in the range $[1/C, 1]$, and the goal is to efficiently maintain an approximately-minimum set cover under insertions and deletions of elements. Most previous work considers the low-frequency regime, namely $f = O(\log n)$, and this line of work has culminated with a deterministic $(1+ε)f$-approximation algorithm with amortized update time $O(\frac{f^2}{ε^3} + \frac{f}{ε^2}\log C)$ [Bhattacharya et al, 2021]. In the high-frequency regime of $f = Ω(\log n)$, an $O(\log n)$-approximation algorithm with amortized update time $O(f\log n)$ was given by [Gupta et al, 2017]. Interestingly, at the intersection of the two regimes, i.e., $f = Θ(\log n)$, the state-of-the-art results coincide: approximation $Θ(f) = Θ(\log n)$ with amortized update time $O(f^2) = O(f \log n) = O(\log^2 n)$. Up to this date, no previous work achieved update time of $o(f^2)$. In this paper we break the $Ω(f^2)$ update time barrier via the following results: (1) $(1+ε)f$-approximation can be maintained in $O\left(\frac{f}{ε^3}\log^*f + \frac{f}{ε^3}\log C\right) = O_{ε,C}(f \log^* f)$ expected amortized update time; our algorithm works against an adaptive adversary. (2) $(1+ε)f$-approximation can be maintained deterministically in $O\left(\frac{1}εf\log f + \frac{f}{ε^3} + \frac{f\log C}{ε^2}\right) = O_{ε,C}(f \log f)$ amortized update time.

cs.DS↗

arXiv2024

A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks

In virtual assistant (VA) systems it is important to reject or redirect user queries that fall outside the scope of the system. One of the most accurate approaches for out-of-scope (OOS) rejection is to combine it with the task of intent classification on in-scope queries, and to use methods based on the similarity of embeddings produced by transformer-based sentence encoders. Typically, such encoders are fine-tuned for the intent-classification task, using cross-entropy loss. Recent work has shown that while this produces suitable embeddings for the intent-classification task, it also tends to disperse in-scope embeddings over the full sentence embedding space. This causes the in-scope embeddings to potentially overlap with OOS embeddings, thereby making OOS rejection difficult. This is compounded when OOS data is unknown. To mitigate this issue our work proposes to regularize the cross-entropy loss with an in-scope embedding reconstruction loss learned using an auto-encoder. Our method achieves a 1-4% improvement in the area under the precision-recall curve for rejecting out-of-sample (OOS) instances, without compromising intent classification performance.

cs.CL↗

arXiv2024

Almost-Optimal Sublinear Additive Spanners

Given an undirected unweighted graph $G = (V, E)$ on $n$ vertices and $m$ edges, a subgraph $H\subseteq G$ is a spanner of $G$ with stretch function $f: \mathbb{R}_+ \rightarrow \mathbb{R}_+$, if for every pair $s, t$ of vertices in $V$, $\text{dist}_{H}(s, t)\le f(\text{dist}_{G}(s, t))$. When $f(d) = d + o(d)$, $H$ is called a sublinear additive spanner; when $f(d) = d + o(n)$, $H$ is called an \emph{additive spanner}, and $f(d) - d$ is usually called the \emph{additive stretch} of $H$. As our primary result, we show that for any constant $δ>0$ and constant integer $k\geq 2$, every graph on $n$ vertices has a sublinear additive spanner with stretch function $f(d)=d+O(d^{1-1/k})$ and $O\big(n^{1+\frac{1+δ}{2^{k+1}-1}}\big)$ edges. When $k = 2$, this improves upon the previous spanner construction with stretch function $f(d) = d + O(d^{1/2})$ and $\tilde{O}(n^{1+3/17})$ edges; for any constant integer $k\geq 3$, this improves upon the previous spanner construction with stretch function $f(d) = d + O(d^{1-1/k})$ and $O\bigg(n^{1+\frac{(3/4)^{k-2}}{7 - 2\cdot (3/4)^{k-2}}}\bigg)$ edges. Most importantly, the size of our spanners almost matches the lower bound of $Ω\big(n^{1+\frac{1}{2^{k+1}-1}}\big)$, which holds for all compression schemes achieving the same stretch function. As our second result, we show a new construction of additive spanners with stretch $O(n^{0.403})$ and $\tilde{O}(n)$ edges, which slightly improves upon the previous stretch bound of $O(n^{3/7+\varepsilon})$ achieved by linear-size spanners. An additional advantage of our spanner is that it admits a subquadratic construction runtime of $\tilde{O}(m + n^{13/7})$, while the previous construction requires all-pairs shortest paths computation which takes $O(\min\{mn, n^{2.373}\})$ time.

cs.DS↗

arXiv2024

Even Faster $(Δ+ 1)$-Edge Coloring via Shorter Multi-Step Vizing Chains

Vizing's Theorem from 1964 states that any $n$-vertex $m$-edge graph with maximum degree $Δ$ can be {\em edge colored} using at most $Δ+ 1$ colors. For over 40 years, the state-of-the-art running time for computing such a coloring, obtained independently by Arjomandi [1982] and by Gabow, Nishizeki, Kariv, Leven and Terada~[1985], was $\tilde O(m\sqrt{n})$. Very recently, this time bound was improved in two independent works, by Bhattacharya, Carmon, Costa, Solomon and Zhang to $\tilde O(mn^{1/3})$, and by Assadi to $\tilde O(n^2)$. In this paper we present an algorithm that computes such a coloring in $\tilde O(mn^{1/4})$ time. Our key technical contribution is a subroutine for extending the coloring to one more edge within time $\tilde O(Δ^2 + \sqrt{Δn})$. The best previous time bound of any color extension subroutine is either the trivial $O(n)$, dominated by the length of a Vizing chain, or the bound $\tilde{O}(Δ^6)$ by Bernshteyn [2022], dominated by the length of {\em multi-step Vizing chains}, which is basically a concatenation of multiple (carefully chosen) Vizing chains. Our color extension subroutine produces significantly shorter multi-step Vizing chains than in previous works, for sufficiently large $Δ$.

cs.DS↗

arXiv2024

Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code

Despite their success, large language models (LLMs) face the critical challenge of hallucinations, generating plausible but incorrect content. While much research has focused on hallucinations in multiple modalities including images and natural language text, less attention has been given to hallucinations in source code, which leads to incorrect and vulnerable code that causes significant financial loss. To pave the way for research in LLMs' hallucinations in code, we introduce Collu-Bench, a benchmark for predicting code hallucinations of LLMs across code generation (CG) and automated program repair (APR) tasks. Collu-Bench includes 13,234 code hallucination instances collected from five datasets and 11 diverse LLMs, ranging from open-source models to commercial ones. To better understand and predict code hallucinations, Collu-Bench provides detailed features such as the per-step log probabilities of LLMs' output, token types, and the execution feedback of LLMs' generated code for in-depth analysis. In addition, we conduct experiments to predict hallucination on Collu-Bench, using both traditional machine learning techniques and neural networks, which achieves 22.03 -- 33.15% accuracy. Our experiments draw insightful findings of code hallucination patterns, reveal the challenge of accurately localizing LLMs' hallucinations, and highlight the need for more sophisticated techniques.

cs.SE↗