arXiv subjects

Jian Zhang

Publications and source records attributed to Jian Zhang.

At least 361 records · Page 20Linked to original sources

arXiv2024

Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation

Code large language models (LLMs) face limitations in repository-level code generation due to their lack of awareness of repository-level dependencies (e.g., user-defined attributes), resulting in dependency errors such as undefined-variable and no-member errors. In this work, we introduce ToolGen, an approach that integrates autocompletion tools into the code LLM generation process to address these dependencies. ToolGen comprises two main phases: Trigger Insertion and Model Fine-tuning (Offline), and Tool-integrated Code Generation (Online). During the offline phase, ToolGen augments functions within a given code corpus with a special mark token, indicating positions to trigger autocompletion tools. These augmented functions, along with their corresponding docstrings, are then used to fine-tune a selected code LLM. In the online phase, ToolGen iteratively generates functions by predicting tokens step-by-step using the fine-tuned LLM. Whenever a mark token is encountered, ToolGen invokes the autocompletion tool to suggest code completions and selects the most appropriate one. We conduct comprehensive experiments to evaluate ToolGen's effectiveness in repository-level code generation. To facilitate this evaluation, we create a benchmark comprising 671 real-world code repositories and introduce two new dependency-based metrics: Dependency Coverage and Static Validity Rate. The results demonstrate that ToolGen significantly improves Dependency Coverage by 31.4% to 39.1% and Static Validity Rate by 44.9% to 57.7% across the three LLMs, while maintaining competitive or improved performance in widely recognized similarity metrics such as BLEU-4, CodeBLEU, Edit Similarity, and Exact Match. On the CoderEval dataset, ToolGen achieves improvements of 40.0% and 25.0% in Pass@1 for CodeT5 and CodeLlama, respectively.

cs.SE↗

arXiv2024

Improving AlphaFlow for Efficient Protein Ensembles Generation

Investigating conformational landscapes of proteins is a crucial way to understand their biological functions and properties. AlphaFlow stands out as a sequence-conditioned generative model that introduces flexibility into structure prediction models by fine-tuning AlphaFold under the flow-matching framework. Despite the advantages of efficient sampling afforded by flow-matching, AlphaFlow still requires multiple runs of AlphaFold to finally generate one single conformation. Due to the heavy consumption of AlphaFold, its applicability is limited in sampling larger set of protein ensembles or the longer chains within a constrained timeframe. In this work, we propose a feature-conditioned generative model called AlphaFlow-Lit to realize efficient protein ensembles generation. In contrast to the full fine-tuning on the entire structure, we focus solely on the light-weight structure module to reconstruct the conformation. AlphaFlow-Lit performs on-par with AlphaFlow and surpasses its distilled version without pretraining, all while achieving a significant sampling acceleration of around 47 times. The advancement in efficiency showcases the potential of AlphaFlow-Lit in enabling faster and more scalable generation of protein ensembles.

cs.LG↗

arXiv2024

A multi-field decomposed model order reduction approach for thermo-mechanically coupled gradient-extended damage simulations

Numerical simulations are crucial for comprehending how engineering structures behave under extreme conditions, particularly when dealing with thermo-mechanically coupled issues compounded by damage-induced material softening. However, such simulations often entail substantial computational expenses. To mitigate this, the focus has shifted towards employing model order reduction (MOR) techniques, which hold promise for accelerating computations. Yet, applying MOR to highly nonlinear, multi-physical problems influenced by material softening remains a relatively new area of research, with numerous unanswered questions. Addressing this gap, this study proposes and investigates a novel multi-field decomposed MOR technique, rooted in a snapshot-based Proper Orthogonal Decomposition-Galerkin (POD-G) projection approach. Utilizing a recently developed thermo-mechanically coupled gradient-extended damage-plasticity model as a case study, this work demonstrates that splitting snapshot vectors into distinct physical fields (displacements, damage, temperature) and projecting them onto separate lower-dimensional subspaces can yield more precise and stable outcomes compared to conventional methods. Through a series of numerical benchmark tests, our multi-field decomposed MOR technique demonstrates its capacity to significantly reduce computational expenses in simulations involving severe damage, while maintaining a high level of accuracy.

math.AP↗

arXiv2024

Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs

Large Language Models (LLMs) are increasingly adopted in financial analysis for interpreting complex market data and trends. However, their use is challenged by intrinsic biases (e.g., risk-preference bias) and a superficial understanding of market intricacies, necessitating a thorough assessment of their financial insight. To address these issues, we introduce Financial Bias Indicators (FBI), a framework with components like Bias Unveiler, Bias Detective, Bias Tracker, and Bias Antidote to identify, detect, analyze, and eliminate irrational biases in LLMs. By combining behavioral finance principles with bias examination, we evaluate 23 leading LLMs and propose a de-biasing method based on financial causal knowledge. Results show varying degrees of financial irrationality among models, influenced by their design and training. Models trained specifically on financial datasets may exhibit more irrationality, and even larger financial language models (FinLLMs) can show more bias than smaller, general models. We utilize four prompt-based methods incorporating causal debiasing, effectively reducing financial biases in these models. This work enhances the understanding of LLMs' bias in financial applications, laying the foundation for developing more reliable and rational financial analysis tools.

cs.CL↗

arXiv2024

Double-tough and ultra-strong ceramics: leveraging multiscale toughening mechanisms through Bayesian Optimization

We present an optimization-driven approach to creating a double-tough ceramic material with a brick-and-mortar microstructure, where the mortar is itself transformation-toughened, engineered with the goal of simultaneously achieving high strength and fracture toughness levels. Specifically, we design a material where high-strength alumina bricks are interconnected via a ceria-stabilized zirconia mortar. As the design of such a material, driven by multiscale toughening mechanisms, requires a laborious trial-and-error approach, we propose a Bayesian optimization framework as an integral part of our methodology to streamline and accelerate the design process. We use a Gaussian process to emulate the material's mechanical response and implement a cost-aware batch Bayesian optimization to efficiently identify optimal design process parameters, accounting for the cost of experimentally varying them. This approach expedites the optimization of the material's mechanical properties. As a result, we develop a bio-inspired all-ceramic composite that exhibits an exceptional balance between bending strength (704 MPa), and fracture toughness (13.6 MPa m^0.5).

physics.app-ph↗

arXiv2024

Self-reconfigurable Multifunctional Memristive Nociceptor for Intelligent Robotics

Artificial nociceptors, mimicking human-like stimuli perception, are of significance for intelligent robotics to work in hazardous and dynamic scenarios. One of the most essential characteristics of the human nociceptor is its self-adjustable attribute, which indicates that the threshold of determination of a potentially hazardous stimulus relies on environmental knowledge. This critical attribute has been currently omitted, but it is highly desired for artificial nociceptors. Inspired by these shortcomings, this article presents, for the first time, a Self-Directed Channel (SDC) memristor-based self-reconfigurable nociceptor, capable of perceiving hazardous pressure stimuli under different temperatures and demonstrates key features of tactile nociceptors, including 'threshold,' 'no-adaptation,' and 'sensitization.' The maximum amplification of hazardous external stimuli is 1000%, and its response characteristics dynamically adapt to current temperature conditions by automatically altering the generated modulation schemes for the memristor. The maximum difference ratio of the response of memristors at different temperatures is 500%, and this adaptability closely mimics the functions of biological tactile nociceptors, resulting in accurate danger perception in various conditions. Beyond temperature adaptation, this memristor-based nociceptor has the potential to integrate different sensory modalities by applying various sensors, thereby achieving human-like perception capabilities in real-world environments.

physics.app-ph↗

arXiv2024

GraphStorm: all-in-one graph machine learning framework for industry applications

Graph machine learning (GML) is effective in many business applications. However, making GML easy to use and applicable to industry applications with massive datasets remain challenging. We developed GraphStorm, which provides an end-to-end solution for scalable graph construction, graph model training and inference. GraphStorm has the following desirable properties: (a) Easy to use: it can perform graph construction and model training and inference with just a single command; (b) Expert-friendly: GraphStorm contains many advanced GML modeling techniques to handle complex graph data and improve model performance; (c) Scalable: every component in GraphStorm can operate on graphs with billions of nodes and can scale model training and inference to different hardware without changing any code. GraphStorm has been used and deployed for over a dozen billion-scale industry applications after its release in May 2023. It is open-sourced in Github: https://github.com/awslabs/graphstorm.

cs.LG↗

arXiv2024

Real-Time State Modulation and Acquisition Circuit in Neuromorphic Memristive Systems

Memristive neuromorphic systems are designed to emulate human perception and cognition, where the memristor states represent essential historical information to perform both low-level and high-level tasks. However, current systems face challenges with the separation of state modulation and acquisition, leading to undesired time delays that impact real-time performance. To overcome this issue, we introduce a dual-function circuit that concurrently modulates and acquires memristor state information. This is achieved through two key features: 1) a feedback operational amplifier (op-amp) based circuit that ensures precise voltage application on the memristor while converting the passing current into a voltage signal; 2) a division calculation circuit that acquires state information from the modulation voltage and the converted voltage, improving stability by leveraging the intrinsic threshold characteristics of memristors. This circuit has been evaluated in a memristor-based nociceptor and a memristor crossbar, demonstrating exceptional performance. For instance, it achieves mean absolute acquisition errors below 1 Ω during the modulation process in the nociceptor application. These results demonstrate that the proposed circuit can operate at different scales, holding the potential to enhance a wide range of neuromorphic applications.

physics.app-ph↗

arXiv2024

Fracture metamaterials with on-demand crack paths enabled by bending

In many scenarios -- when we bite food or during a crash -- fracture is inevitable. Finding solutions to steer fracture to mitigate its impact or turn it into a purposeful functionality, is therefore crucial. Strategies using composites, changes in chemical composition or crystal orientation, have proven to be very efficient, but the crack path control remains limited and has not been achieved in load-bearing structures. Here, we introduce fracture metamaterials consisting of slender elements whose bending enables large elastic deformation as fracture propagates. This interplay between bending and fracture enables tunable energy dissipation and the design of on-demand crack paths of arbitrary complexity. To this end, we use topology optimisation to create unit cells with anisotropic fracture energy, which we then tile up to realize fracture metamaterials with uniform density that we 3D-print. The thin ligaments that constitute the unit cells confer them a strikingly distinct response in tension and shear, and we show that by controlling the orientation and layout of the unit cells the sequential progress of the crack can be controlled, making the fracture path arbitrarily tortuous. This tortuosity increases the energy dissipation of the metamaterial without changing its stiffness. Using bespoke arrangements of unit cells, metamaterials can have on-demand fracture paths of arbitrary complexity. Our findings bring a new perspective on inelastic deformations in mechanical metamaterials, with potential applications in areas as diverse as the food industry, structural design, and for shock and impact damping.

cond-mat.soft↗

arXiv2024

Rigorous Simulation-based Testing for Autonomous Driving Systems -- Targeting the Achilles' Heel of Four Open Autopilots

Simulation-based testing remains the main approach for validating Autonomous Driving Systems. We propose a rigorous test method based on breaking down scenarios into simple ones, taking into account the fact that autopilots make decisions according to traffic rules whose application depends on local knowledge and context. This leads us to consider the autopilot as a dynamic system receiving three different types of vistas as input, each characterizing a specific driving operation and a corresponding control policy. The test method for the considered vista types generates test cases for critical configurations that place the vehicle under test in critical situations characterized by the transition from cautious behavior to progression in order to clear an obstacle. The test cases thus generated are realistic, i.e., they determine the initial conditions from which safe control policies are possible, based on knowledge of the vehicle's dynamic characteristics. Constraint analysis identifies the most critical test cases, whose success implies the validity of less critical ones. Test coverage can therefore be greatly simplified. Critical test cases reveal major defects in Apollo, Autoware, and the Carla and LGSVL autopilots. Defects include accidents, software failures, and traffic rule violations that would be difficult to detect by random simulation, as the test cases lead to situations characterized by finely-tuned parameters of the vehicles involved, such as their relative position and speed. Our results corroborate real-life observations and confirm that autonomous driving systems still have a long way to go before offering acceptable safety guarantees.

cs.SE↗

arXiv2024

Asymptotic behavior of solutions for a critical heat equation with nonlocal reaction

In this paper, we consider the following nonlocal parabolic equation \begin{equation*} u_{t}-Δu=\left( \int_Ω\frac{|u(y,t)|^{2^{\ast}_μ}}{|x-y|^μ}dy\right) |u|^{2^{\ast}_μ-2}u,\ \text{in}\ Ω\times(0,\infty), \end{equation*} where $Ω$ is a bounded domain in $\mathbb{R}^{N}$, $0<μ<N$ and $2^{\ast}_μ=(2N-μ)/(N-2)$ denotes the critical exponent in the sense of the Hardy-Littlewood-Sobolev inequality. We first introduce the stable and unstable sets for the equation and prove that the problem has a potential well structure. Next, we investigate the global asymptotic behavior of the solutions. In particular, we study the behavior of the global solutions that intersect neither with the stable set nor the unstable set. Finally, we prove that global solutions have $L^{\infty}$-uniform bound under some natural conditions.

math.AP↗

arXiv2024

IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

Infrared and visible image fusion (IVIF) is used to generate fusion images with comprehensive features of both images, which is beneficial for downstream vision tasks. However, current methods rarely consider the illumination condition in low-light environments, and the targets in the fused images are often not prominent. To address the above issues, we propose an Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet. In our framework, an illumination enhancement network first estimates the incident illumination maps of input images. Afterwards, with the help of proposed adaptive differential fusion module (ADFM) and salient target aware module (STAM), an image fusion network effectively integrates the salient features of the illumination-enhanced infrared and visible images into a fusion image of high visual quality. Extensive experimental results verify that our method outperforms five state-of-the-art methods of fusing infrared and visible images.

cs.CV↗

arXiv2024

Protect-Your-IP: Scalable Source-Tracing and Attribution against Personalized Generation

With the advent of personalized generation models, users can more readily create images resembling existing content, heightening the risk of violating portrait rights and intellectual property (IP). Traditional post-hoc detection and source-tracing methods for AI-generated content (AIGC) employ proactive watermark approaches; however, these are less effective against personalized generation models. Moreover, attribution techniques for AIGC rely on passive detection but often struggle to differentiate AIGC from authentic images, presenting a substantial challenge. Integrating these two processes into a cohesive framework not only meets the practical demands for protection and forensics but also improves the effectiveness of attribution tasks. Inspired by this insight, we propose a unified approach for image copyright source-tracing and attribution, introducing an innovative watermarking-attribution method that blends proactive and passive strategies. We embed copyright watermarks into protected images and train a watermark decoder to retrieve copyright information from the outputs of personalized models, using this watermark as an initial step for confirming if an image is AIGC-generated. To pinpoint specific generation techniques, we utilize powerful visual backbone networks for classification. Additionally, we implement an incremental learning strategy to adeptly attribute new personalized models without losing prior knowledge, thereby enhancing the model's adaptability to novel generation methods. We have conducted experiments using various celebrity portrait series sourced online, and the results affirm the efficacy of our method in source-tracing and attribution tasks, as well as its robustness against knowledge forgetting.

cs.CV↗

arXiv2024

ReVideo: Remake a Video with Motion and Content Control

Despite significant advancements in video generation and editing using diffusion models, achieving accurate and localized video editing remains a substantial challenge. Additionally, most existing video editing methods primarily focus on altering visual content, with limited research dedicated to motion editing. In this paper, we present a novel attempt to Remake a Video (ReVideo) which stands out from existing methods by allowing precise video editing in specific areas through the specification of both content and motion. Content editing is facilitated by modifying the first frame, while the trajectory-based motion control offers an intuitive user interaction experience. ReVideo addresses a new task involving the coupling and training imbalance between content and motion control. To tackle this, we develop a three-stage training strategy that progressively decouples these two aspects from coarse to fine. Furthermore, we propose a spatiotemporal adaptive fusion module to integrate content and motion control across various sampling steps and spatial locations. Extensive experiments demonstrate that our ReVideo has promising performance on several accurate video editing applications, i.e., (1) locally changing video content while keeping the motion constant, (2) keeping content unchanged and customizing new motion trajectories, (3) modifying both content and motion trajectories. Our method can also seamlessly extend these applications to multi-area editing without specific training, demonstrating its flexibility and robustness.

cs.CV↗

arXiv2024

KPConvX: Modernizing Kernel Point Convolution with Kernel Attention

In the field of deep point cloud understanding, KPConv is a unique architecture that uses kernel points to locate convolutional weights in space, instead of relying on Multi-Layer Perceptron (MLP) encodings. While it initially achieved success, it has since been surpassed by recent MLP networks that employ updated designs and training strategies. Building upon the kernel point principle, we present two novel designs: KPConvD (depthwise KPConv), a lighter design that enables the use of deeper architectures, and KPConvX, an innovative design that scales the depthwise convolutional weights of KPConvD with kernel attention values. Using KPConvX with a modern architecture and training strategy, we are able to outperform current state-of-the-art approaches on the ScanObjectNN, Scannetv2, and S3DIS datasets. We validate our design choices through ablation studies and release our code and models.

cs.CV↗

arXiv2024

Diffusion-Based Hierarchical Image Steganography

This paper introduces Hierarchical Image Steganography, a novel method that enhances the security and capacity of embedding multiple images into a single container using diffusion models. HIS assigns varying levels of robustness to images based on their importance, ensuring enhanced protection against manipulation. It adaptively exploits the robustness of the Diffusion Model alongside the reversibility of the Flow Model. The integration of Embed-Flow and Enhance-Flow improves embedding efficiency and image recovery quality, respectively, setting HIS apart from conventional multi-image steganography techniques. This innovative structure can autonomously generate a container image, thereby securely and efficiently concealing multiple images and text. Rigorous subjective and objective evaluations underscore our advantage in analytical resistance, robustness, and capacity, illustrating its expansive applicability in content safeguarding and privacy fortification.

cs.CV↗

arXiv2024

3D Shape Augmentation with Content-Aware Shape Resizing

Recent advancements in deep learning for 3D models have propelled breakthroughs in generation, detection, and scene understanding. However, the effectiveness of these algorithms hinges on large training datasets. We address the challenge by introducing Efficient 3D Seam Carving (E3SC), a novel 3D model augmentation method based on seam carving, which progressively deforms only part of the input model while ensuring the overall semantics are unchanged. Experiments show that our approach is capable of producing diverse and high-quality augmented 3D shapes across various types and styles of input models, achieving considerable improvements over previous methods. Quantitative evaluations demonstrate that our method effectively enhances the novelty and quality of shapes generated by other subsequent 3D generation algorithms.

cs.CV↗

arXiv2024

360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model

Panorama video recently attracts more interest in both study and application, courtesy of its immersive experience. Due to the expensive cost of capturing 360-degree panoramic videos, generating desirable panorama videos by prompts is urgently required. Lately, the emerging text-to-video (T2V) diffusion methods demonstrate notable effectiveness in standard video generation. However, due to the significant gap in content and motion patterns between panoramic and standard videos, these methods encounter challenges in yielding satisfactory 360-degree panoramic videos. In this paper, we propose a pipeline named 360-Degree Video Diffusion model (360DVD) for generating 360-degree panoramic videos based on the given prompts and motion conditions. Specifically, we introduce a lightweight 360-Adapter accompanied by 360 Enhancement Techniques to transform pre-trained T2V models for panorama video generation. We further propose a new panorama dataset named WEB360 consisting of panoramic video-text pairs for training 360DVD, addressing the absence of captioned panoramic video datasets. Extensive experiments demonstrate the superiority and effectiveness of 360DVD for panorama video generation. Our project page is at https://akaneqwq.github.io/360DVD/.

cs.CV↗