arXiv subjects

Hong Li

Publications and source records attributed to Hong Li.

At least 109 records · Page 6Linked to original sources

arXiv2025

The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs

Multi-modal Large Language Models (MLLMs) have exhibited impressive capability. However, recently many deficiencies of MLLMs have been found compared to human intelligence, $\textit{e.g.}$, hallucination. To drive the MLLMs study, the community dedicated efforts to building larger benchmarks with complex tasks. In this paper, we propose benchmarking an essential but usually overlooked intelligence: $\textbf{association}$, a human's basic capability to link observation and prior practice memory. To comprehensively investigate MLLM's performance on the association, we formulate the association task and devise a standard benchmark based on adjective and verb semantic concepts. Instead of costly data annotation and curation, we propose a convenient $\textbf{annotation-free}$ construction method transforming the general dataset for our association tasks. Simultaneously, we devise a rigorous data refinement process to eliminate confusion in the raw dataset. Building on this database, we establish three levels of association tasks: single-step, synchronous, and asynchronous associations. Moreover, we conduct a comprehensive investigation into the MLLMs' zero-shot association capabilities, addressing multiple dimensions, including three distinct memory strategies, both open-source and closed-source MLLMs, cutting-edge Mixture-of-Experts (MoE) models, and the involvement of human experts. Our systematic investigation shows that current open-source MLLMs consistently exhibit poor capability in our association tasks, even the currently state-of-the-art GPT-4V(vision) also has a significant gap compared to humans. We believe our benchmark would pave the way for future MLLM studies. $\textit{Our data and code are available at:}$ https://mvig-rhos.com/llm_inception.

cs.CV↗

arXiv2025

Dynamic Tuning of Single-Photon Emission in Monolayer WSe2 via Localized Strain Engineering

Two-dimensional (2D) materials have emerged as promising candidates for next-generation integrated single-photon emitters (SPEs). However, significant variability in the emission energies of 2D SPEs presents a major challenge in producing identical single photons from different SPEs, which may become crucial for various quantum applications including quantum information processing. Although various approaches to dynamically tuning the emission energies of 2D SPEs have been developed to address the issue, the practical solution to matching multiple individual SPEs in a single 2D flake is still scarce. In this work, we demonstrate a precise emission energy tuning of individual SPEs in a WSe2 monolayer. Our approach utilizes localized strain fields near individual SPEs, which we control independently by adjusting the physical volume of an SU-8-based stressor layer via focused laser annealing. This technique allows continuous emission energy tuning of up to 15 meV while maintaining the qualities of SPEs. Additionally, we showcase the precise spectral alignment of three distinct SPEs in a single WSe2 monolayer to the same wavelength. The tunability of 2D SPEs represents a solid step towards the on-chip integrated photonics with 2D materials for quantum technologies.

cond-mat.mes-hall↗

arXiv2025

Enhanced dynamo drive for the sawtooth relaxation process due to non-uniform resistivity distribution in a reversed field pinch

In this work, we use the three-dimensional resistive MHD code NIMROD to investigate the impact of resistivity inhomogeneity on the sawtooth process of an reversed field pinch (RFP) plasma. The simulation employs a non-uniform resistivity profile similar to experiments, which monotonically increases from the core to the edge as the temperature decreases. The resistivity inhomogeneity introduces an additional electric field in the plasma, which accelerates the inward diffusion of magnetic flux and changing the self sustained reversal state, hence significantly enhances the dynamo effect and the sawtooth process in the RFP plasma.

physics.plasm-ph↗

arXiv2025

Discovery of Rapid Polarization Angle Variation During the 2022 Outburst of XTE J1701-462

The geometry of the Comptonization corona in neutron star low-mass X-ray binaries is still unclear. We conducted time-resolved polarimetric analysis of the archival observations of XTE J1701--462 obtained with the \textit{Imaging X-ray Polarimeter Explorer} during its 2022 outburst, and found that the polarization angle (PA) varied significantly with time when the source was in the normal branch (NB), with $67 \pm 8^{\circ}$ in the first epoch, $-34 \pm 8^{\circ}$ in the second, and $-58 \pm 8^{\circ}$ in the third, last epoch. Meanwhile, the polarization degree remained constant at around 2\%, above the minimum detectable polarization at the 99\% confidence level (MDP$_{99}$). The rapid PA variation causes depolarization in the time-averaged data, resulting in a nondetection as reported in the literature. The rapid (intra-day) PA variation may suggest that there is a fast transformation of the corona geometry, likely switching from a slab geometry with enhanced disk emission and reflection, to a more vertically extended spreading layer geometry.

astro-ph.HE↗

arXiv2025

Mid-infrared group-IV nanowire laser

Semiconductor nanowires have shown great potential for enabling ultra-compact lasers for integrated photonics platforms. Despite the impressive progress in developing nanowire lasers, their integration into Si photonics platforms remains challenging largely due to the use of III-V and II-VI semiconductors as gain media. These materials not only have high material costs, but also require inherently complex integration with Si-based fabrication processing, increasing overall costs and thereby limiting their large-scale adoption. Furthermore, these material-based nanowire lasers rarely emit above 2 um, which is a technologically important wavelength regime for various applications in imaging and quantum sensing. Recently, group-IV nanowires, particularly direct bandgap GeSn nanowires capable of emitting above 2 um, have emerged as promising cost-effective gain media for Si-compatible nanowire lasers, but there has been no successful demonstration of lasing from this seemingly promising nanowire platform. Herein, we report the experimental observation of lasing above 2 um from a single bottom-up grown GeSn nanowire. By harnessing strain engineering and optimized cavity designs simultaneously, the single GeSn nanowire achieves an amplified material gain that can sufficiently overcome minimized optical losses, resulting in a single-mode lasing with an ultra-low threshold of ~5.3 kW cm-2. Our finding paves the way for all-group IV mid-infrared photonic-integrated circuits with compact Si-compatible lasers for on-chip classical and quantum sensing and free-space communication.

physics.optics↗

arXiv2025

Fine-Grained 1-Day Vulnerability Detection in Binaries via Patch Code Localization

1-day vulnerabilities in binaries have become a major threat to software security. Patch presence test is one of the effective ways to detect the vulnerability. However, existing patch presence test works do not perform well in practical scenarios due to the interference from the various compilers and optimizations, patch-similar code blocks, and irrelevant functions in stripped binaries. In this paper, we propose a novel approach named PLocator, which leverages stable values from both the patch code and its context, extracted from the control flow graph, to accurately locate the real patch code in the target function, offering a practical solution for real-world vulnerability detection scenarios. To evaluate the effectiveness of PLocator, we collected 73 CVEs and constructed two comprehensive datasets ($Dataset_{-irr}$ and $Dataset_{+irr}$), comprising 1,090 and 27,250 test cases at four compilation optimization levels and two compilers with three different experiments, i.e., Same, XO (cross-optimizations), and XC (cross-compilers). The results demonstrate that PLocator achieves an average TPR of 88.2% and FPR of 12.9% in a short amount of time, outperforming state-of-the-art approaches by 26.7% and 63.5%, respectively, indicating that PLocator is more practical for the 1-day vulnerability detection task.

cs.CR↗

arXiv2025

Symmetry-Driven Bulk-Edge Correspondence in Electron Magnetofluids at Finite Temperature

We present a theoretical framework connecting the pseudo-Chern number in momentum space to the spectral flow index in phase space for continuous media, with specific applications to topological Langmuir-cyclotron waves (TLCWs) in magnetized plasmas at uniform finite temperatures. By deriving a rigorous correspondence between these two topological invariants, we provide a solid justification for previous studies that applied this relationship heuristically across various continuous media. For magnetized plasmas with finite-temperature effects, we confirm the existence of TLCWs through numerical computation of bulk Chern number differences and analytical calculation of the spectral flow index. These findings advance the understanding of topological phenomena in continuous media.

physics.plasm-ph↗

arXiv2025

Tackling Small Sample Survival Analysis via Transfer Learning: A Study of Colorectal Cancer Prognosis

Survival prognosis is crucial for medical informatics. Practitioners often confront small-sized clinical data, especially cancer patient cases, which can be insufficient to induce useful patterns for survival predictions. This study deals with small sample survival analysis by leveraging transfer learning, a useful machine learning technique that can enhance the target analysis with related knowledge pre-learned from other data. We propose and develop various transfer learning methods designed for common survival models. For parametric models such as DeepSurv, Cox-CC (Cox-based neural networks), and DeepHit (end-to-end deep learning model), we apply standard transfer learning techniques like pretraining and fine-tuning. For non-parametric models such as Random Survival Forest, we propose a new transfer survival forest (TSF) model that transfers tree structures from source tasks and fine-tunes them with target data. We evaluated the transfer learning methods on colorectal cancer (CRC) prognosis. The source data are 27,379 SEER CRC stage I patients, and the target data are 728 CRC stage I patients from the West China Hospital. When enhanced by transfer learning, Cox-CC's $C^{td}$ value was boosted from 0.7868 to 0.8111, DeepHit's from 0.8085 to 0.8135, DeepSurv's from 0.7722 to 0.8043, and RSF's from 0.7940 to 0.8297 (the highest performance). All models trained with data as small as 50 demonstrated even more significant improvement. Conclusions: Therefore, the current survival models used for cancer prognosis can be enhanced and improved by properly designed transfer learning techniques. The source code used in this study is available at https://github.com/YonghaoZhao722/TSF.

cs.LG↗

arXiv2024

Generating Unseen Nonlinear Evolution in Sea Surface Temperature Using a Deep Learning-Based Latent Space Data Assimilation Framework

Advances in data assimilation (DA) methods have greatly improved the accuracy of Earth system predictions. To fuse multi-source data and reconstruct the nonlinear evolution missing from observations, geoscientists are developing future-oriented DA methods. In this paper, we redesign a purely data-driven latent space DA framework (DeepDA) that employs a generative artificial intelligence model to capture the nonlinear evolution in sea surface temperature. Under variational constraints, DeepDA embedded with nonlinear features can effectively fuse heterogeneous data. The results show that DeepDA remains highly stable in capturing and generating nonlinear evolutions even when a large amount of observational information is missing. It can be found that when only 10% of the observation information is available, the error increase of DeepDA does not exceed 40%. Furthermore, DeepDA has been shown to be robust in the fusion of real observations and ensemble simulations. In particular, this paper provides a mechanism analysis of the nonlinear evolution generated by DeepDA from the perspective of physical patterns, which reveals the inherent explainability of our DL model in capturing multi-scale ocean signals.

physics.ao-ph↗

arXiv2024

TransferFuzz: Fuzzing with Historical Trace for Verifying Propagated Vulnerability Code

Code reuse in software development frequently facilitates the spread of vulnerabilities, making the scope of affected software in CVE reports imprecise. Traditional methods primarily focus on identifying reused vulnerability code within target software, yet they cannot verify if these vulnerabilities can be triggered in new software contexts. This limitation often results in false positives. In this paper, we introduce TransferFuzz, a novel vulnerability verification framework, to verify whether vulnerabilities propagated through code reuse can be triggered in new software. Innovatively, we collected runtime information during the execution or fuzzing of the basic binary (the vulnerable binary detailed in CVE reports). This process allowed us to extract historical traces, which proved instrumental in guiding the fuzzing process for the target binary (the new binary that reused the vulnerable function). TransferFuzz introduces a unique Key Bytes Guided Mutation strategy and a Nested Simulated Annealing algorithm, which transfers these historical traces to implement trace-guided fuzzing on the target binary, facilitating the accurate and efficient verification of the propagated vulnerability. Our evaluation, conducted on widely recognized datasets, shows that TransferFuzz can quickly validate vulnerabilities previously unverifiable with existing techniques. Its verification speed is 2.5 to 26.2 times faster than existing methods. Moreover, TransferFuzz has proven its effectiveness by expanding the impacted software scope for 15 vulnerabilities listed in CVE reports, increasing the number of affected binaries from 15 to 53. The datasets and source code used in this article are available at https://github.com/Siyuan-Li201/TransferFuzz.

cs.SE↗

arXiv2024

BinEnhance: An Enhancement Framework Based on External Environment Semantics for Binary Code Search

Binary code search plays a crucial role in applications like software reuse detection. Currently, existing models are typically based on either internal code semantics or a combination of function call graphs (CG) and internal code semantics. However, these models have limitations. Internal code semantic models only consider the semantics within the function, ignoring the inter-function semantics, making it difficult to handle situations such as function inlining. The combination of CG and internal code semantics is insufficient for addressing complex real-world scenarios. To address these limitations, we propose BinEnhance, a novel framework designed to leverage the inter-function semantics to enhance the expression of internal code semantics for binary code search. Specifically, BinEnhance constructs an External Environment Semantic Graph (EESG), which establishes a stable and analogous external environment for homologous functions by using different inter-function semantic relations (e.g., call, location, data-co-use). After the construction of EESG, we utilize the embeddings generated by existing internal code semantic models to initialize nodes of EESG. Finally, we design a Semantic Enhancement Model (SEM) that uses Relational Graph Convolutional Networks (RGCNs) and a residual block to learn valuable external semantics on the EESG for generating the enhanced semantics embedding. In addition, BinEnhance utilizes data feature similarity to refine the cosine similarity of semantic embeddings. We conduct experiments under six different tasks (e.g., under function inlining scenario) and the results illustrate the performance and robustness of BinEnhance. The application of BinEnhance to HermesSim, Asm2vec, TREX, Gemini, and Asteria on two public datasets results in an improvement of Mean Average Precision (MAP) from 53.6% to 69.7%. Moreover, the efficiency increases fourfold.

cs.SE↗

arXiv2024

AnimateAnything: Consistent and Controllable Animation for Video Generation

We present a unified controllable video generation approach AnimateAnything that facilitates precise and consistent video manipulation across various conditions, including camera trajectories, text prompts, and user motion annotations. Specifically, we carefully design a multi-scale control feature fusion network to construct a common motion representation for different conditions. It explicitly converts all control information into frame-by-frame optical flows. Then we incorporate the optical flows as motion priors to guide final video generation. In addition, to reduce the flickering issues caused by large-scale motion, we propose a frequency-based stabilization module. It can enhance temporal coherence by ensuring the video's frequency domain consistency. Experiments demonstrate that our method outperforms the state-of-the-art approaches. For more details and videos, please refer to the webpage: https://yu-shaonian.github.io/Animate_Anything/.

cs.CV↗

arXiv2024

Linear Chain Transformation: Expanding Optimization Dynamics for Fine-Tuning Large Language Models

Fine-tuning large language models (LLMs) has become essential for adapting pretrained models to specific downstream tasks. In this paper, we propose Linear Chain Transformation (LinChain), a novel approach that introduces a sequence of linear transformations during fine-tuning to enrich optimization dynamics. By incorporating multiple linear transformations into the parameter update process, LinChain expands the effective rank of updates and enhances the model's ability to learn complex task-specific representations. We demonstrate that this method significantly improves the performance of LLM fine-tuning over state-of-the-art methods by providing more flexible optimization paths during training, while maintaining the inference efficiency of the resulting model. Our experiments on various benchmark tasks show that LinChain leads to better generalization, fewer learnable parameters, and improved task adaptation, making it a compelling strategy for LLM fine-tuning.

cs.CL↗

arXiv2024

Deterministic formation of carbon-functionalized quantum emitters in hexagonal boron nitride

Forming single-photon emitters (SPEs) in insulating hexagonal boron nitride (hBN) has sparked wide interests in the quantum photonics. Despite significant progress, it remains challenging to deterministically create SPEs at precise locations with a specific type of element for creating defects. In this study, we present a straightforward approach to generate site-deterministic carbon-functionalized quantum emitters in hBN by harnessing ultrasonic nanoindentation. The obtained SPEs are high-quality and can be scaled up to large arrays in a single fabrication step. Comprehensive experimental analyses reveal that the insertion of carbon atoms into the hBN lattice is the source of the robust quantum emission. Complementary theoretical studies suggest possible candidates for the structural origin of the defects based on our experimental results. This rapid and scalable nanoindentation method provides a new way to create SPE arrays with specific types of atoms, enabling the comprehensive investigation of the origins and mechanics of SPE formations in two-dimensional (2D) materials and beyond.

physics.app-ph↗

arXiv2024

IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts

Recent advances in 3D generation have been remarkable, with methods such as DreamFusion leveraging large-scale text-to-image diffusion-based models to guide 3D object generation. These methods enable the synthesis of detailed and photorealistic textured objects. However, the appearance of 3D objects produced by such text-to-3D models is often unpredictable, and it is hard for single-image-to-3D methods to deal with images lacking a clear subject, complicating the generation of appearance-controllable 3D objects from complex images. To address these challenges, we present IPDreamer, a novel method that captures intricate appearance features from complex $\textbf{I}$mage $\textbf{P}$rompts and aligns the synthesized 3D object with these extracted features, enabling high-fidelity, appearance-controllable 3D object generation. Our experiments demonstrate that IPDreamer consistently generates high-quality 3D objects that align with both the textual and complex image prompts, highlighting its promising capability in appearance-controlled, complex 3D object generation. Our code is available at https://github.com/zengbohan0217/IPDreamer.

cs.CV↗

arXiv2024

Li$_{14}$Mn$_{2}$S$_{9}$ and Li$_{10}$Si$_{2}$S$_{9}$ as a pair of all-electrochem-active electrode and solid-state electrolyte with chemical compatibility and low interface resistance

In solid-state batteries (SSBs), improving the physical contact at the electrode-electrolyte interface is essential for achieving better performance and durability. On the one hand, it is necessary to look for solid-state electrolytes (SSEs) with high ionic conductivity and no reaction with the electrode, on the other hand, to design the all-electrochem-active (AEA) electrodes that contain no SSEs and other non-active substances. In this work, we proposed a pair of AEA-electrode and SSE with the same structural framework and excellent interface compatibility, Li$_{14}$Mn$_{2}$S$_{9}$ and Li$_{10}$Si$_{2}$S$_{9}$, and confirmed the feasibility by ab-initio molecular dynamics (AIMD) simulations and machine learning interatomic potential based molecular dynamics (MLIP-based MD) simulations, providing a new approach to promote interfacial stability in SSBs.

cond-mat.mtrl-sci↗

arXiv2024

ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy

While vision-and-language models significantly advance in many fields, the challenge of continual learning is unsolved. Parameter-efficient modules like adapters and prompts present a promising way to alleviate catastrophic forgetting. However, existing works usually learn individual adapters for each task, which may result in redundant knowledge among adapters. Moreover, they continue to use the original pre-trained model to initialize the downstream model, leading to negligible changes in the model's generalization compared to the original model. In addition, there is still a lack of research investigating the consequences of integrating a multi-modal model into the updating procedure for both uni-modal and multi-modal tasks and the subsequent impacts it has on downstream tasks. In this paper, we propose an adapter-based two-stage learning paradigm, a multi-modal continual learning scheme that consists of experience-based learning and novel knowledge expansion, which helps the model fully use experience knowledge and compensate for novel knowledge. Extensive experiments demonstrate that our method is proficient for continual learning. It expands the distribution of representation upstream while also minimizing the negative impact of forgetting previous tasks. Additionally, it enhances the generalization capability for downstream tasks. Furthermore, we incorporate both multi-modal and uni-modal tasks into upstream continual learning. We observe that learning from upstream tasks can help with downstream tasks. Our code will be available at: https://github.com/lihong2303/ATLAS.

cs.LG↗

arXiv2024

Enhancing Performance and Scalability of Large-Scale Recommendation Systems with Jagged Flash Attention

The integration of hardware accelerators has significantly advanced the capabilities of modern recommendation systems, enabling the exploration of complex ranking paradigms previously deemed impractical. However, the GPU-based computational costs present substantial challenges. In this paper, we demonstrate our development of an efficiency-driven approach to explore these paradigms, moving beyond traditional reliance on native PyTorch modules. We address the specific challenges posed by ranking models' dependence on categorical features, which vary in length and complicate GPU utilization. We introduce Jagged Feature Interaction Kernels, a novel method designed to extract fine-grained insights from long categorical features through efficient handling of dynamically sized tensors. We further enhance the performance of attention mechanisms by integrating Jagged tensors with Flash Attention. Our novel Jagged Flash Attention achieves up to 9x speedup and 22x memory reduction compared to dense attention. Notably, it also outperforms dense flash attention, with up to 3x speedup and 53% more memory efficiency. In production models, we observe 10% QPS improvement and 18% memory savings, enabling us to scale our recommendation systems with longer features and more complex architectures.

cs.LG↗