arXiv subjects

Jian Li

Publications and source records attributed to Jian Li.

At least 253 records · Page 14Linked to original sources

arXiv2024

LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description

Visual Spatial Description (VSD) aims to generate texts that describe the spatial relationships between objects within images. Traditional visual spatial relationship classification (VSRC) methods typically output the spatial relationship between two objects in an image, often neglecting world knowledge and lacking general language capabilities. In this paper, we propose a Large Language-and-Vision Assistant for Visual Spatial Description, named LLaVA-VSD, which is designed for the classification, description, and open-ended description of visual spatial relationships. Specifically, the model first constructs a VSD instruction-following dataset using given figure-caption pairs for the three tasks. It then employs LoRA to fine-tune a Large Language and Vision Assistant for VSD, which has 13 billion parameters and supports high-resolution images. Finally, a large language model (Qwen-2) is used to refine the generated sentences, enhancing their diversity and accuracy. LLaVA-VSD demonstrates excellent multimodal conversational capabilities and can follow open-ended instructions to assist with inquiries about object relationships in images.

cs.CV↗

arXiv2024

A new multiple-arc model of the resonant Kuiper belt objects -- Plutinos

To incorporate the gravitational influence of Kuiper belt objects (KBOs) in planetary ephemerides, uniform-ring models are commonly employed. In this paper, for representing the KBO population residing in Neptune's 2:3 mean motion resonance (MMR), known as the Plutinos, we introduce a three-arc model by considering their resonant characteristics. Each `arc' refers to a segment of the uniform ring and comprises an appropriate number of point masses. Then the total perturbation of Plutinos is numerically measured by the change in the Sun-Neptune distance ($Δd_{SN}$). We conduct a comprehensive investigation to take into account various azimuthal and radial distributions associated with the resonant amplitudes ($A$) and eccentricities ($e$) of Plutinos, respectively. The results show that over a 100-year period: (1) at the smallest $e=0.05$, the Sun-Neptune distance change $Δd_{SN}$ caused by Plutinos decreases significantly as $A$ reduces. It can deviate from the value of $Δd_{SN}$ obtained in the ring model by approximately 100 km; (2) as $e$ increases in the medium range of 0.1-0.2, the difference in $Δd_{SN}$ between the arc and ring models becomes increasingly significant; (3) at the largest $e\gtrsim0.25$, $Δd_{SN}$ can approach zero regardless of $A$, and the arc and ring models exhibit a substantial difference in $Δd_{SN}$, reaching up to 170 km. Then the applicability of our three-arc model is further verified by comparing it to the perturbations induced by observed Plutinos on the positions of both Neptune and Saturn. Moreover, the concept of the multiple-arc model, designed for Plutinos, can be easily extended to other MMRs densely populated by small bodies.

astro-ph.EP↗

arXiv2024

High-Quality, ROS Compatible Video Encoding and Decoding for High-Definition Datasets

Robotic datasets are important for scientific benchmarking and developing algorithms, for example for Simultaneous Localization and Mapping (SLAM). Modern robotic datasets feature video data of high resolution and high framerates. Storing and sharing those datasets becomes thus very costly, especially if more than one camera is used for the datasets. It is thus essential to store this video data in a compressed format. This paper investigates the use of modern video encoders for robotic datasets. We provide a software that can replay mp4 videos within ROS 1 and ROS 2 frameworks, supporting the synchronized playback in simulated time. Furthermore, the paper evaluates different encoders and their settings to find optimal configurations in terms of resulting size, quality and encoding time. Through this work we show that it is possible to store and share even highest quality video datasets within reasonable storage constraints.

cs.RO↗

arXiv2024

A spatiotemporal knowledge graph-based method for identifying individual activity locations from mobile phone data

In recent years, mobile phone data has been widely used for human mobility analytics. Identifying individual activity locations is the fundamental step for mobile phone data processing. Current methods typically aggregate spatially adjacent location records over multiple days to identify activity locations. However, only considering spatial relationships while overlooking temporal ones may lead to inaccurate activity location identification, and also affect activity pattern analysis. In this study, we propose a spatiotemporal knowledge graph-based (STKG) method for identifying activity locations from mobile phone data. An STKG is designed and constructed to describe individual mobility characteristics. The spatial and temporal relationships of individual stays are inferred and transformed into a spatiotemporal graph. The modularity-optimization community detection algorithm is applied to identify stays with dense spatiotemporal relationships, which are considering as activity locations. A case study in Shanghai was conducted to verify the performance of the proposed method. The results show that compared with two baseline methods, the STKG-based method can limit an additional 45% of activity locations with the longest daytime stay within a reasonable spatial range; In addition, the STKG-based method exhibit lower variance in the start and end times of activities across different days, performing approximately 10% to 20% better than the two baseline methods. Moreover, the STKG-based method effectively distinguishes between locations that are geographically close but exhibit different temporal patterns. These findings demonstrate the effectiveness of STKG-based method in enhancing both spatial precision and temporal consistency.

cs.SI↗

arXiv2024

MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving

This paper introduces MCTrack, a new 3D multi-object tracking method that achieves state-of-the-art (SOTA) performance across KITTI, nuScenes, and Waymo datasets. Addressing the gap in existing tracking paradigms, which often perform well on specific datasets but lack generalizability, MCTrack offers a unified solution. Additionally, we have standardized the format of perceptual results across various datasets, termed BaseVersion, facilitating researchers in the field of multi-object tracking (MOT) to concentrate on the core algorithmic development without the undue burden of data preprocessing. Finally, recognizing the limitations of current evaluation metrics, we propose a novel set that assesses motion information output, such as velocity and acceleration, crucial for downstream tasks. The source codes of the proposed method are available at this link: https://github.com/megvii-research/MCTrack}{https://github.com/megvii-research/MCTrack

cs.CV↗

arXiv2024

Analysis of a FEM-MCM Discretization for the 2D/3D stochastic closed-loop geothermal system

This paper develops a new 2D/3D stochastic closed-loop geothermal system with a random hydraulic conductivity tensor. We use the finite element method (FEM) and the Monte Carlo method (MCM) to discrete physical and probability spaces, respectively. This FEM-MCM method is effective. The stability for velocity and temperature is rigorously proved. Compared with the deterministic closed-loop geothermal system, a same optimal error estimate for approximate velocity and temperature is obtained. Furthermore, a series of numerical experiments were carried out to show this method has better stability and accuracy results.

math.NA↗

arXiv2024

Coresets for Constrained Clustering: General Assignment Constraints and Improved Size Bounds

Designing small-sized \emph{coresets}, which approximately preserve the costs of the solutions for large datasets, has been an important research direction for the past decade. We consider coreset construction for a variety of general constrained clustering problems. We introduce a general class of assignment constraints, including capacity constraints on cluster centers, and assignment structure constraints for data points (modeled by a convex body $\mathcal{B}$). We give coresets for clustering problems with such general assignment constraints that significantly generalize and improve known results. Notable implications include the first $\varepsilon$-coreset for capacitated and fair $k$-Median with $m$ outliers in Euclidean spaces whose size is $\tilde{O}(m + k^2 \varepsilon^{-4})$, generalizing and improving upon the prior bounds in [Braverman et al., FOCS' 22; Huang et al., ICLR' 23] (for capacitated $k$-Median, the coreset size bound obtained in [Braverman et al., FOCS' 22] is $\tilde{O}(k^3 \varepsilon^{-6})$, and for $k$-Median with $m$ outliers, the coreset size bound obtained in [Huang et al., ICLR' 23]} is $\tilde{O}(m + k^3 \varepsilon^{-5})$), and the first $ε$-coreset of size $\mathrm{poly}(k \varepsilon^{-1})$ for fault-tolerant clustering for various types of metric spaces.

cs.DS↗

arXiv2024

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model

Large Language Models (LLMs) have demonstrated exceptional proficiency in mathematical reasoning tasks due to their extensive parameter counts and training on vast datasets. Despite these capabilities, deploying LLMs is hindered by their computational demands. Distilling LLM mathematical reasoning into Smaller Language Models (SLMs) has emerged as a solution to this challenge, although these smaller models often suffer from errors in calculation and semantic understanding. Prior work has proposed Program-of-Thought Distillation (PoTD) to avoid calculation error. To further address semantic understanding errors, we propose Key-Point-Driven Mathematical Reasoning Distillation (KPDD). KPDD enhances the reasoning performance of SLMs by breaking down the problem-solving process into three stages: Core Question Extraction, Problem-Solving Information Extraction, and Step-by-Step Solution. This method is further divided into KPDD-CoT, which generates Chain-of-Thought rationales, and KPDD-PoT, which creates Program-of-Thought rationales. The experiment results show that KPDD-CoT significantly improves reasoning abilities, while KPDD-PoT achieves state-of-the-art performance in mathematical reasoning tasks. Our approach effectively mitigates misunderstanding errors, advancing the deployment of efficient and capable SLMs.

cs.CL↗

arXiv2024

Chaotic dynamics of granules-beam coupled vibration: Route and threshold

Although granular materials are the second most processed in industry after water, the theoretical study of granules-structure interactions is not as advanced as that of fluid-structure interactions due to the lack of a unified view of the constitutive relation of granular materials. In the previous work, the theoretical model of granules-beam coupled vibration was developed and verified by experiments. However, it was also found that the system exhibits significant stiffness-softening Duffing characteristics even under micro-vibration, which implies that chaotic responses may be reached under certain conditions. To reveal the route and critical conditions for the system to enter chaos, in the present work, the chaotic dynamics of the system are studied. In qualitative analysis, Melnikov method is applied to analyze the instability behavior of the perturbed heteroclinic orbit of the system, thus the critical condition for the system to enter Smale horseshoe chaos, i.e., the Melnikov criterion, is obtained. The validity of the criterion is verified numerically. In experimental studies, the existence of chaotic responses and the route to chaos for granules-beam coupled vibrations is revealed. The experimental results suggest that the system response first experiences symmetry-breaking then undergoes a complete period-doubling cascade, and finally enters single-scroll chaos. In addition, although the Additional Dynamic Load (ADL) generated by granular media is highly complicated, a general and simple evolution pattern of the chaos threshold is found by parametric experiments, which is also supported by the Melnikov criterion. In short, the chaotic dynamics of granules-beam coupled vibration is revealed, which is a contribution to the engineering vibration aspect. On the theoretical side, an impressive result lies in that for the first time, the Melnikov criterion of such fractional-order...

nlin.CD↗

arXiv2024

On the Melnikov method for fractional-order systems

This paper is dedicated to clarifying and introducing the correct application of Melnikov method in fractional dynamics. Attention to the complex dynamics of hyperbolic orbits and to fractional calculus can be, respectively, traced back to Poincarés attack on the three-body problem a century ago and to the early days of calculus three centuries ago. Nowadays, fractional calculus has been widely applied in modeling dynamic problems across various fields due to its advantages in describing problems with non-locality. Some of these models have also been confirmed to exhibit hyperbolic orbit dynamics, and recently, they have been extensively studied based on Melnikov method, an analytical approach for homoclinic and heteroclinic orbit dynamics. Despite its decade-long application in fractional dynamics, there is a universal problem in these applications that remains to be clarified, i.e., defining fractional-order systems within finite memory boundaries leads to the neglect of perturbation calculation for parts of the stable and unstable manifolds in Melnikov analysis. After clarifying and redefining the problem, a rigorous analytical case is provided for reference. Unlike existing results, the Melnikov criterion here is derived in a globally closed form, which was previously considered unobtainable due to difficulties in the analysis of fractional-order perturbations characterized by convolution integrals with power-law type singular kernels. Finally, numerical methods are employed to verify the derived Melnikov criterion. Overall, the clarification for the problem and the presented case are expected to provide insights for future research in this topic.

nlin.CD↗

arXiv2024

Resonant amplitude distribution of the Hilda asteroids and the free-floating planet flyby scenario

In some recent work, we provided a quantitative explanation for the number asymmetry of Jupiter Trojans by hypothesizing a free-floating planet (FFP) flyby into the Solar System. In support of that explanation, this paper examines the influence of the same FFP flyby on the Hilda asteroids, which orbit stably in the 3:2 mean motion resonance with Jupiter. The observed Hilda population exhibits two distinct resonant patterns: (1) a lack of Hildas with resonant amplitudes < 40 deg. at eccentricities < 0.1; (2) a nearly complete absence of Hildas with amplitudes < 20 deg., regardless of eccentricity. Previous models of Jupiter migration and resonance capture could account for the eccentricity distribution of Hildas but have failed to replicate the unusual absence of those with the smallest resonant amplitudes, which theoretically should be the most stable. Here we report that the FFP flyby can trigger an extremely rapid outward migration of Jupiter, causing a sudden shift in the 3:2 Jovian resonance. Consequently, Hildas with varying eccentricities would have their resonant amplitudes changed by different degrees, leading to the observed resonant patterns. We additionally show that, in our FFP flyby scenario, these patterns are consistently present across different resonant amplitude distributions of primordial Hildas arising from various formation models. We also place constraints on the potential parameters of the FFP, suggesting it should have an eccentricity of 1-1.3 or larger, an inclination up to 30 deg. or higher, and a minimum mass of about 50 Earth masses.

astro-ph.EP↗

arXiv2024

S-RAN: Semantic-Aware Radio Access Networks

Semantic communication (SemCom) has been a transformative paradigm, emphasizing the precise exchange of meaningful information over traditional bit-level transmissions. However, existing SemCom research, primarily centered on simplified scenarios like single-pair transmissions with direct wireless links, faces significant challenges when applied to real-world radio access networks (RANs). This article introduces a Semantic-aware Radio Access Network (S-RAN), offering a holistic systematic view of SemCom beyond single-pair transmissions. We begin by outlining the S-RAN architecture, introducing new physical components and logical functions along with key design challenges. We then present transceiver design for end-to-end transmission to overcome conventional SemCom transceiver limitations, including static channel conditions, oversimplified background knowledge models, and hardware constraints. Later, we delve into the discussion on radio resource management for multiple users, covering semantic channel modeling, performance metrics, resource management algorithms, and a case study, to elaborate distinctions from resource management for legacy RANs. Finally, we highlight open research challenges and potential solutions. The objective of this article is to serve as a basis for advancing SemCom research into practical wireless systems.

eess.SP↗

arXiv2024

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness

Recently, there has been significant interest in replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs), such as Direct Preference Optimization (DPO) and its variants. These approaches commonly use a binary cross-entropy mechanism on pairwise samples, i.e., minimizing and maximizing the loss based on preferred or dis-preferred responses, respectively. However, while this training strategy omits the reward model, it also overlooks the varying preference degrees within different responses. We hypothesize that this is a key factor hindering LLMs from sufficiently understanding human preferences. To address this problem, we propose a novel Self-supervised Preference Optimization (SPO) framework, which constructs a self-supervised preference degree loss combined with the alignment loss, thereby helping LLMs improve their ability to understand the degree of preference. Extensive experiments are conducted on two widely used datasets of different tasks. The results demonstrate that SPO can be seamlessly integrated with existing preference optimization methods and significantly boost their performance to achieve state-of-the-art performance. We also conduct detailed analyses to offer comprehensive insights into SPO, which verifies its effectiveness. The code is available at https://github.com/lijian16/SPO.

cs.CL↗

arXiv2024

Cylindrical Cavity Expansion: A Novel Method for Characterizing the Mechanical Properties of Soft Materials

The low elastic modulus of soft materials, combined with geometric nonlinearity and rate dependence, presents significant challenges in the characterization of their mechanical response. We introduce a novel method for measuring the mechanical properties of soft materials under large deformations via cylindrical cavity expansion. In this method, a cylindrical cavity is fabricated in the material and expanded by volume-controlled injection of an incompressible fluid with simultaneous measurement of the applied pressure at the cavity wall. The relationship between applied pressure and deformation at the cavity wall is then employed to characterize the nonlinear mechanical properties. We demonstrate the feasibility of the proposed method and validate it by measuring the mechanical properties of synthetic polydimethylsiloxane (PDMS) and comparing with reported values in the literature. Results indicate that the cylindrical cavitation method effectively captures the response of PDMS over a wide range of stiffness (shear modulus ranging from 5 kPa to 300 kPa) and exhibit high repeatability. The proposed method overcomes limitations in characterization of ultra-soft materials using traditional testing methods, such as challenges with fabrication and clamping in unaxial tension testing and friction and adhesion effects in compression and indentation testing, thus enabling accurate and precise characterization. It also offers improved accuracy and repeatability over other needle induced cavity expansion methods due to precise control over the initial cavity dimension and shape at the cost of increased invasiveness of testing.

cond-mat.mtrl-sci↗

arXiv2024

VI3DRM:Towards meticulous 3D Reconstruction from Sparse Views via Photo-Realistic Novel View Synthesis

Recently, methods like Zero-1-2-3 have focused on single-view based 3D reconstruction and have achieved remarkable success. However, their predictions for unseen areas heavily rely on the inductive bias of large-scale pretrained diffusion models. Although subsequent work, such as DreamComposer, attempts to make predictions more controllable by incorporating additional views, the results remain unrealistic due to feature entanglement in the vanilla latent space, including factors such as lighting, material, and structure. To address these issues, we introduce the Visual Isotropy 3D Reconstruction Model (VI3DRM), a diffusion-based sparse views 3D reconstruction model that operates within an ID consistent and perspective-disentangled 3D latent space. By facilitating the disentanglement of semantic information, color, material properties and lighting, VI3DRM is capable of generating highly realistic images that are indistinguishable from real photographs. By leveraging both real and synthesized images, our approach enables the accurate construction of pointmaps, ultimately producing finely textured meshes or point clouds. On the NVS task, tested on the GSO dataset, VI3DRM significantly outperforms state-of-the-art method DreamComposer, achieving a PSNR of 38.61, an SSIM of 0.929, and an LPIPS of 0.027. Code will be made available upon publication.

cs.CV↗

arXiv2024

X-ray hardening preceding the onset of SGR 1935+2154's radio pulsar phase

Magnetars are neutron stars with extremely strong magnetic fields, frequently powering high-energy activity in X-rays. Pulsed radio emission following some X-ray outbursts have been detected (\citealt{Camilo2006,camilo2007a}), albeit its physical origin is unclear. It has long been speculated that the origin of magnetars' radio signals is different from those from canonical pulsars, although convincing evidence is still lacking. Five months after magnetar SGR 1935+2154's X-ray outburst and its associated Fast Radio Burst (FRB) 20200428, a radio pulsar phase was discovered. Here we report the discovery of X-ray spectral hardening associated with the emergence of periodic radio pulsations from SGR 1935+2154 and a detailed analysis of the properties of the radio pulses. The observations suggest that radio emission originates from the outer magnetosphere of the magnetar, and the surface heating due to the bombardment of inward-going particles from the radio emission region is responsible for the observed X-ray spectral hardening.

astro-ph.HE↗

arXiv2024

Large-scale road network partitioning: a deep learning method based on convolutional autoencoder model

With the development of urbanization, the scale of urban road network continues to expand, especially in some Asian countries. Short-term traffic state prediction is one of the bases of traffic management and control. Constrained by the space-time cost of computation, the short-term traffic state prediction of large-scale urban road network is difficult. One way to solve this problem is to partition the whole network into multiple sub-networks to predict traffic state separately. In this study, a deep learning method is proposed for road network partitioning. The method mainly includes three steps. First, the daily speed series for roads are encoded into the matrix. Second, a convolutional autoencoder (AE) is built to extract series features and compress data. Third, the spatial hierarchical clustering method with adjacency relationship is applied in the road network. The proposed method was verified by the road network of Shenzhen which contains more than 5000 links. The results show that AE-hierarchical clustering distinguishes the tidal traffic characteristics and reflects the process of congestion propagation. Furthermore, two indicators are designed to make a quantitative comparison with spectral clustering: intra homogeneity increases by about 9% on average while inter heterogeneity about 9.5%. Compared with past methods, the time cost also decreases. The results may suggest ways to improve the management and control of urban road network in other metropolitan cities. The proposed method is expected to be extended to other problems related to large-scale networks.

eess.SY↗

arXiv2024

A Survey on Benchmarks of Multimodal Large Language Models

Multimodal Large Language Models (MLLMs) are gaining increasing popularity in both academia and industry due to their remarkable performance in various applications such as visual question answering, visual perception, understanding, and reasoning. Over the past few years, significant efforts have been made to examine MLLMs from multiple perspectives. This paper presents a comprehensive review of 200 benchmarks and evaluations for MLLMs, focusing on (1)perception and understanding, (2)cognition and reasoning, (3)specific domains, (4)key capabilities, and (5)other modalities. Finally, we discuss the limitations of the current evaluation methods for MLLMs and explore promising future directions. Our key argument is that evaluation should be regarded as a crucial discipline to support the development of MLLMs better. For more details, please visit our GitHub repository: https://github.com/swordlidev/Evaluation-Multimodal-LLMs-Survey.

cs.CL↗