arXiv subjects

Lei Li

Publications and source records attributed to Lei Li.

At least 307 records · Page 17Linked to original sources

arXiv2025

Propagation of chaos in path spaces via information theory

Propagation of chaos for interacting particle systems has been an active research topic over decades. We propose an alternative approach to study the mean-field limit of the stochastic interacting particle systems via tools from information theory. In our framework, the propagation of chaos is reduced to the space for driving processes with possible lower dimension. Indeed, after applying the data processing inequality, one only needs to estimate the difference between the drifts of the particle system and the mean-field Mckean stochastic differential equation. This point is particularly useful in situations where the discrepancy in the driving processes is more apparent than the investigated processes. We will take the second order system as well as other examples for the illustration of how our framework could be used. This approach allows us to focus on probability measures in path spaces for the driving processes, avoiding using the usual hypocoercivity technique or taking the pseudo-inverse of the diffusion matrix, which might be more stable for numerical computation. Our framework is different from current approaches in literature and could provide new insight into the study of interacting particle systems.

math.PR↗

arXiv2025

Longtime behaviors of a reducible cooperative system with nonlocal diffusions and free boundaries

This paper aims at understanding the longtime behaviors of a reducible cooperative system with nonlocal diffusions and different free boundaries, describing the interactions of two mutually beneficial species. Compared with the irreducible and monostable cooperative system, the system we care about here has many nonnegative steady states, leading to much different and rich longtime behaviors. Moreover, since the possible nonnegative steady states on half space are non-constant, we need to employ more detailed analysis to understand the corresponding steady state problems which in turn helps us to derive a complete classification for the longtime behaviors of our problem. The spreading speeds of free boundaries and more accurate limits of $(u,v)$ as $t\to\infty$ are also discussed, and accelerated spreading can happen if some threshold conditions are violated by kernel functions.

math.AP↗

arXiv2024

A structure-preserving collisional particle method for the Landau kinetic equation

In this paper, we propose and implement a structure-preserving stochastic particle method for the Landau equation. The method is based on a particle system for the Landau equation, where pairwise grazing collisions are modeled as diffusion processes. By exploiting the unique structure of the particle system and a spherical Brownian motion sampling, the method avoids additional temporal discretization of the particle system, ensuring that the discrete-time particle distributions exactly match their continuous-time counterparts. The method achieves $O(N)$ complexity per time step and preserves fundamental physical properties, including the conservation of mass, momentum and energy, as well as entropy dissipation. It demonstrates strong long-time accuracy and stability in numerical experiments. Furthermore, we also apply the method to the spatially non-homogeneous equations through a case study of the Vlasov--Poisson--Landau equation.

math.NA↗

arXiv2024

Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey

Building on the foundations of language modeling in natural language processing, Next Token Prediction (NTP) has evolved into a versatile training objective for machine learning tasks across various modalities, achieving considerable success. As Large Language Models (LLMs) have advanced to unify understanding and generation tasks within the textual modality, recent research has shown that tasks from different modalities can also be effectively encapsulated within the NTP framework, transforming the multimodal information into tokens and predict the next one given the context. This survey introduces a comprehensive taxonomy that unifies both understanding and generation within multimodal learning through the lens of NTP. The proposed taxonomy covers five key aspects: Multimodal tokenization, MMNTP model architectures, unified task representation, datasets \& evaluation, and open challenges. This new taxonomy aims to aid researchers in their exploration of multimodal intelligence. An associated GitHub repository collecting the latest papers and repos is available at https://github.com/LMM101/Awesome-Multimodal-Next-Token-Prediction

cs.CL↗

arXiv2024

Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models

Recent advancements in large language models (LLMs) have led to significant breakthroughs in mathematical reasoning capabilities. However, existing benchmarks like GSM8K or MATH are now being solved with high accuracy (e.g., OpenAI o1 achieves 94.8\% on MATH dataset), indicating their inadequacy for truly challenging these models. To bridge this gap, we propose a comprehensive and challenging benchmark specifically designed to assess LLMs' mathematical reasoning at the Olympiad level. Unlike existing Olympiad-related benchmarks, our dataset focuses exclusively on mathematics and comprises a vast collection of 4428 competition-level problems with rigorous human annotation. These problems are meticulously categorized into over 33 sub-domains and span more than 10 distinct difficulty levels, enabling a holistic assessment of model performance in Olympiad-mathematical reasoning. Furthermore, we conducted an in-depth analysis based on this benchmark. Our experimental results show that even the most advanced models, OpenAI o1-mini and OpenAI o1-preview, struggle with highly challenging Olympiad-level problems, with 60.54\% and 52.55\% accuracy, highlighting significant challenges in Olympiad-level mathematical reasoning.

cs.CL↗

arXiv2024

A second order Langevin sampler preserving positive volume for isothermal isobaric ensemble

We propose in this work a second-order Langevin sampler for the isothermal-isobaric ensemble (the NPT ensemble), preserving a positive volume for the simulation box. We first derive the suitable equations of motion for particles to be coupled with the overdamped Langevin equation of volume by sending the artificial mass of the periodic box to zero in the work of Liang et. al. [J. Chem. Phys. 157(14)]. We prove the well-posedness of the new system of equations and show that its invariant measure is the desired ensemble. The new continuous time equations not only justify the previous cell-rescaling methods, but also allow us to choose a suitable friction coefficient so that one has additive noise after a change of variable by taking logarithm of the volume. This observation allows us to propose a second order weak scheme that guarantees the positivity of the volume. Various numerical experiments have been performed to demonstrate the efficacy of our method.

math.NA↗

arXiv2024

Seamless Detection: Unifying Salient Object Detection and Camouflaged Object Detection

Achieving joint learning of Salient Object Detection (SOD) and Camouflaged Object Detection (COD) is extremely challenging due to their distinct object characteristics, i.e., saliency and camouflage. The only preliminary research treats them as two contradictory tasks, training models on large-scale labeled data alternately for each task and assessing them independently. However, such task-specific mechanisms fail to meet real-world demands for addressing unknown tasks effectively. To address this issue, in this paper, we pioneer a task-agnostic framework to unify SOD and COD. To this end, inspired by the agreeable nature of binary segmentation for SOD and COD, we propose a Contrastive Distillation Paradigm (CDP) to distil the foreground from the background, facilitating the identification of salient and camouflaged objects amidst their surroundings. To probe into the contribution of our CDP, we design a simple yet effective contextual decoder involving the interval-layer and global context, which achieves an inference speed of 67 fps. Besides the supervised setting, our CDP can be seamlessly integrated into unsupervised settings, eliminating the reliance on extensive human annotations. Experiments on public SOD and COD datasets demonstrate the superiority of our proposed framework in both supervised and unsupervised settings, compared with existing state-of-the-art approaches. Code is available on https://github.com/liuyi1989/Seamless-Detection.

cs.CV↗

arXiv2024

Characterization of minimal tripotents via annihilators and its application to the study of additive preservers of truncations

The contributions in this note begin with a new characterization of (positive) scalar multiples of minimal tripotents in a general JB$^*$-triple $E$, proving that a non-zero element $a\in E$ is a positive scalar multiple of a minimal tripotent in $E$ if, and only if, its inner quadratic annihilator (that is, the set $^{\perp_{q}}\!\{a\} = \{ b\in E: \{a,b,a\} =0\}$) is maximal among all inner quadratic annihilators of single elements in $E$. We subsequently apply this characterization to the study of surjective additive maps between atomic JBW$^*$-triples preserving truncations in both directions. Let $A: E\to F$ be a surjective additive mapping between atomic JBW$^*$-triples, where $E$ contains no one-dimensional Cartan factors as direct summands. We show that $A$ preserves truncations in both directions if, and only if, there exists a bijection $σ: Γ_1\to Γ_2$, a bounded family $(γ_k)_{k\in Γ_1}\subseteq \mathbb{R}^+$, and a family $(Φ_k)_{k\in Γ_1},$ where each $Φ_k$ is a (complex) linear or a conjugate-linear (isometric) triple isomorphism from $C_k$ onto $\widetilde{C}_{σ(k)}$ satisfying $\inf_{k} \{γ_k \} >0,$ and $$A(x) = \Big( γ_{k} Φ_k \left(π_k(x)\right) \Big)_{k\inΓ_1},\ \hbox{ for all } x\in E,$$ where $π_k$ denotes the canonical projection of $E$ onto $C_k.$

math.OA↗

arXiv2024

GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models

Visual Language Models (VLMs) demonstrate impressive capabilities in processing multimodal inputs, yet applications such as visual agents, which require handling multiple images and high-resolution videos, demand enhanced long-range modeling. Moreover, existing open-source VLMs lack systematic exploration into extending their context length, and commercial models often provide limited details. To tackle this, we aim to establish an effective solution that enhances long context performance of VLMs while preserving their capacities in short context scenarios. Towards this goal, we make the best design choice through extensive experiment settings from data curation to context window extending and utilizing: (1) we analyze data sources and length distributions to construct ETVLM - a data recipe to balance the performance across scenarios; (2) we examine existing position extending methods, identify their limitations and propose M-RoPE++ as an enhanced approach; we also choose to solely instruction-tune the backbone with mixed-source data; (3) we discuss how to better utilize extended context windows and propose hybrid-resolution training. Built on the Qwen-VL series model, we propose Giraffe, which is effectively extended to 128K lengths. Evaluated on extensive long context VLM benchmarks such as VideoMME and Viusal Haystacks, our Giraffe achieves state-of-the-art performance among similarly sized open-source long VLMs and is competitive with commercial model GPT-4V. We will open-source the code, data, and models.

cs.CV↗

arXiv2024

CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applications

Vision Transformers (ViTs) mark a revolutionary advance in neural networks with their token mixer's powerful global context capability. However, the pairwise token affinity and complex matrix operations limit its deployment on resource-constrained scenarios and real-time applications, such as mobile devices, although considerable efforts have been made in previous works. In this paper, we introduce CAS-ViT: Convolutional Additive Self-attention Vision Transformers, to achieve a balance between efficiency and performance in mobile applications. Firstly, we argue that the capability of token mixers to obtain global contextual information hinges on multiple information interactions, such as spatial and channel domains. Subsequently, we propose Convolutional Additive Token Mixer (CATM) employing underlying spatial and channel attention as novel interaction forms. This module eliminates troublesome complex operations such as matrix multiplication and Softmax. We introduce Convolutional Additive Self-attention(CAS) block hybrid architecture and utilize CATM for each block. And further, we build a family of lightweight networks, which can be easily extended to various downstream tasks. Finally, we evaluate CAS-ViT across a variety of vision tasks, including image classification, object detection, instance segmentation, and semantic segmentation. Our M and T model achieves 83.0\%/84.1\% top-1 with only 12M/21M parameters on ImageNet-1K. Meanwhile, throughput evaluations on GPUs, ONNX, and iPhones also demonstrate superior results compared to other state-of-the-art backbones. Extensive experiments demonstrate that our approach achieves a better balance of performance, efficient inference and easy-to-deploy. Our code and model are available at: \url{https://github.com/Tianfang-Zhang/CAS-ViT}

cs.CV↗

arXiv2024

Position-aware Guided Point Cloud Completion with CLIP Model

Point cloud completion aims to recover partial geometric and topological shapes caused by equipment defects or limited viewpoints. Current methods either solely rely on the 3D coordinates of the point cloud to complete it or incorporate additional images with well-calibrated intrinsic parameters to guide the geometric estimation of the missing parts. Although these methods have achieved excellent performance by directly predicting the location of complete points, the extracted features lack fine-grained information regarding the location of the missing area. To address this issue, we propose a rapid and efficient method to expand an unimodal framework into a multimodal framework. This approach incorporates a position-aware module designed to enhance the spatial information of the missing parts through a weighted map learning mechanism. In addition, we establish a Point-Text-Image triplet corpus PCI-TI and MVP-TI based on the existing unimodal point cloud completion dataset and use the pre-trained vision-language model CLIP to provide richer detail information for 3D shapes, thereby enhancing performance. Extensive quantitative and qualitative experiments demonstrate that our method outperforms state-of-the-art point cloud completion methods.

cs.CV↗

arXiv2024

Graph Canvas for Controllable 3D Scene Generation

Spatial intelligence is foundational to AI systems that interact with the physical world, particularly in 3D scene generation and spatial comprehension. Current methodologies for 3D scene generation often rely heavily on predefined datasets, and struggle to adapt dynamically to changing spatial relationships. In this paper, we introduce GraphCanvas3D, a programmable, extensible, and adaptable framework for controllable 3D scene generation. Leveraging in-context learning, GraphCanvas3D enables dynamic adaptability without the need for retraining, supporting flexible and customizable scene creation. Our framework employs hierarchical, graph-driven scene descriptions, representing spatial elements as graph nodes and establishing coherent relationships among objects in 3D environments. Unlike conventional approaches, which are constrained in adaptability and often require predefined input masks or retraining for modifications, GraphCanvas3D allows for seamless object manipulation and scene adjustments on the fly. Additionally, GraphCanvas3D supports 4D scene generation, incorporating temporal dynamics to model changes over time. Experimental results and user studies demonstrate that GraphCanvas3D enhances usability, flexibility, and adaptability for scene generation. Our code and models are available on the project website: https://github.com/ILGLJ/Graph-Canvas.

cs.CV↗

arXiv2024

Longtime behaviors of an epidemic model with nonlocal diffusions and a free boundary: rate of accelerated spreading

This is the third part of our series of work devoted to the dynamics of an epidemic model with nonlocal diffusions and free boundary. This part is concerned with the rate of accelerated spreading for three types of kernel functions when spreading happens. By constructing the suitable upper and lower solutions, we get the rate of the accelerated spreading of free boundary, which is closely related to the behavior of kernel functions near infinity. Our results indicate that the heavier the tail of the kernel functions are, the faster the rate of accelerated spreading is. Moreover, more accurate spreading profiles for solution component $(u,v)$ are also obtained.

math.AP↗

arXiv2024

Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation

Chain-of-Thought prompting has significantly enhanced the reasoning capabilities of large language models, with numerous studies exploring factors influencing its performance. However, the underlying mechanisms remain poorly understood. To further demystify the operational principles, this work examines three key aspects: decoding, projection, and activation, aiming to elucidate the changes that occur within models when employing Chainof-Thought. Our findings reveal that LLMs effectively imitate exemplar formats while integrating them with their understanding of the question, exhibiting fluctuations in token logits during generation but ultimately producing a more concentrated logits distribution, and activating a broader set of neurons in the final layers, indicating more extensive knowledge retrieval compared to standard prompts. Our code and data will be publicly avialable when the paper is accepted.

cs.AI↗

arXiv2024

Advancing global aerosol forecasting with artificial intelligence

Aerosol forecasting is essential for air quality warnings, health risk assessment, and climate change mitigation. However, it is more complex than weather forecasting due to the intricate interactions between aerosol physicochemical processes and atmospheric dynamics, resulting in significant uncertainty and high computational costs. Here, we develop an artificial intelligence-driven global aerosol-meteorology forecasting system (AI-GAMFS), which provides reliable 5-day, 3-hourly forecasts of aerosol optical components and surface concentrations at a 0.5° x 0.625° resolution. AI-GAMFS combines Vision Transformer and U-Net in a backbone network, robustly capturing the complex aerosol-meteorology interactions via global attention and spatiotemporal encoding. Trained on 42 years of advanced aerosol reanalysis data and initialized with GEOS Forward Processing (GEOS-FP) analyses, AI-GAMFS delivers operational 5-day forecasts in one minute. It outperforms the Copernicus Atmosphere Monitoring Service (CAMS) global forecasting system, GEOS-FP forecasts, and several regional dust forecasting systems in forecasting most aerosol variables including aerosol optical depth and dust components. Our results mark a significant step forward in leveraging AI to refine physics-based aerosol forecasting, facilitating more accurate global warnings for aerosol pollution events, such as dust storms and wildfires.

physics.ao-ph↗

arXiv2024

Importance Weighted Expectation-Maximization for Protein Sequence Design

Designing protein sequences with desired biological function is crucial in biology and chemistry. Recent machine learning methods use a surrogate sequence-function model to replace the expensive wet-lab validation. How can we efficiently generate diverse and novel protein sequences with high fitness? In this paper, we propose IsEM-Pro, an approach to generate protein sequences towards a given fitness criterion. At its core, IsEM-Pro is a latent generative model, augmented by combinatorial structure features from a separately learned Markov random fields (MRFs). We develop an Monte Carlo Expectation-Maximization method (MCEM) to learn the model. During inference, sampling from its latent space enhances diversity while its MRFs features guide the exploration in high fitness regions. Experiments on eight protein sequence design tasks show that our IsEM-Pro outperforms the previous best methods by at least 55% on average fitness score and generates more diverse and novel protein sequences.

q-bio.BM↗

arXiv2024

Trade, Trees, and Lives

This paper shows a cascading mechanism through which international trade-induced deforestation results in a decline of health outcomes in cities distant from where trade activities occur. We examine Brazil, which has ramped up agricultural export over the last two decades to meet rising global demand. Using a shift-share research design, we first show that export shocks cause substantial local agricultural expansion and a virtual one-for-one decline in forest cover. We then construct a dynamic area-of-effect model that predicts where atmospheric changes should be felt - due to loss of forests that would otherwise serve to filter out and absorb air pollutants as they travel - downwind of the deforestation areas. Leveraging quasi-random variation in these atmospheric connections, we establish a causal link between deforestation upstream and subsequent rises in air pollution and premature deaths downstream, with the mortality effects predominantly driven by cardiovascular and respiratory causes. Our estimates reveal a large telecoupled health externality of trade deforestation: over 700,000 premature deaths in Brazil over the past two decades. This equates to $0.18 loss in statistical life value per $1 agricultural exports over the study period.

econ.GN↗

arXiv2024

Hire a Linguist!: Learning Endangered Languages with In-Context Linguistic Descriptions

How can large language models (LLMs) process and translate endangered languages? Many languages lack a large corpus to train a decent LLM; therefore existing LLMs rarely perform well in unseen, endangered languages. On the contrary, we observe that 2000 endangered languages, though without a large corpus, have a grammar book or a dictionary. We propose LINGOLLM, a training-free approach to enable an LLM to process unseen languages that hardly occur in its pre-training. Our key insight is to demonstrate linguistic knowledge of an unseen language in an LLM's prompt, including a dictionary, a grammar book, and morphologically analyzed input text. We implement LINGOLLM on top of two models, GPT-4 and Mixtral, and evaluate their performance on 5 tasks across 8 endangered or low-resource languages. Our results show that LINGOLLM elevates translation capability from GPT-4's 0 to 10.5 BLEU for 10 language directions. Our findings demonstrate the tremendous value of linguistic knowledge in the age of LLMs for endangered languages. Our data, code, and model generations can be found at https://github.com/LLiLab/llm4endangeredlang.

cs.CL↗