arXiv Science⌕ Search

arXiv subjects

Le Chen

Publications and source records attributed to Le Chen.

At least 73 records · Page 4Linked to original sources

Link Node: A Method to Characterize the Chain Topology of Intrinsically Disordered Proteins

It is well-known that intrinsically disordered proteins (IDP) are highly dynamic, which is related to their functionality in various biological processes. However, the characterization of the intricate structures of IDP has been a challenge. Here, we analyze the chain topology of IDPs to characterize their conformations, in combination with molecular dynamics simulation (MD). We systematically compute the Gauss Linking Number ($GLN$) between segments in IDP, and show that the resulting GLN Map can effectively depict an unconventional structure -- physical link, i.e., the entanglement between two segments. The crossing points of physical links are further identified and denoted as Link Nodes. We show that the probability distribution of Link Nodes is highly heterogeneous and there are certain residues that largely affect the chain topology of IDP. Moreover, the structural fluctuations of the vicinity of these residues are largely suppressed, i.e., Link Node provides useful information about the topological constraint imposed on the residues during the conformation fluctuations of IDP. We further reveal that the evolution of the chain topology is considerably slow (with a timescale of hundreds of nanoseconds), which is distinct from the flipping of residue contact.

physics.bio-ph↗

Identifying Policy Gradient Subspaces

Policy gradient methods hold great potential for solving complex continuous control tasks. Still, their training efficiency can be improved by exploiting structure within the optimization problem. Recent work indicates that supervised learning can be accelerated by leveraging the fact that gradients lie in a low-dimensional and slowly-changing subspace. In this paper, we conduct a thorough evaluation of this phenomenon for two popular deep policy gradient methods on various simulated benchmark tasks. Our results demonstrate the existence of such gradient subspaces despite the continuously changing data distribution inherent to reinforcement learning. These findings reveal promising directions for future work on more efficient reinforcement learning, e.g., through improving parameter-space exploration or enabling second-order optimization.

cs.LG↗

The Landscape and Challenges of HPC Research and LLMs

Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning. Both encoder-decoder models and prompt-based techniques have shown immense potential for natural language processing and code-based tasks. Over the past several years, many research labs and institutions have invested heavily in high-performance computing, approaching or breaching exascale performance levels. In this paper, we posit that adapting and utilizing such language model-based techniques for tasks in high-performance computing (HPC) would be very beneficial. This study presents our reasoning behind the aforementioned position and highlights how existing ideas can be improved and adapted for HPC tasks.

cs.LG↗

PERFOGRAPH: A Numerical Aware Program Graph Representation for Performance Optimization and Program Analysis

The remarkable growth and significant success of machine learning have expanded its applications into programming languages and program analysis. However, a key challenge in adopting the latest machine learning methods is the representation of programming languages, which directly impacts the ability of machine learning methods to reason about programs. The absence of numerical awareness, aggregate data structure information, and improper way of presenting variables in previous representation works have limited their performances. To overcome the limitations and challenges of current program representations, we propose a graph-based program representation called PERFOGRAPH. PERFOGRAPH can capture numerical information and the aggregate data structure by introducing new nodes and edges. Furthermore, we propose an adapted embedding method to incorporate numerical awareness. These enhancements make PERFOGRAPH a highly flexible and scalable representation that effectively captures programs intricate dependencies and semantics. Consequently, it serves as a powerful tool for various applications such as program analysis, performance optimization, and parallelism discovery. Our experimental results demonstrate that PERFOGRAPH outperforms existing representations and sets new state-of-the-art results by reducing the error rate by 7.4% (AMD dataset) and 10% (NVIDIA dataset) in the well-known Device Mapping challenge. It also sets new state-of-the-art results in various performance optimization tasks like Parallelism Discovery and NUMA and Prefetchers Configuration prediction.

cs.PL↗

On the radius of self-repellent fractional Brownian motion

We study the radius $R_T$ of a self-repellent fractional Brownian motion $\left\{B^H_t\right\}_{0\le t\le T}$ taking values in $\mathbb{R}^d$. Our sharpest result is for $d=1$, where we find that with high probability, \begin{equation*} R_T \asymp T^ν, \quad \text{with $ν=\frac{2}{3}\left(1+H\right)$.} \end{equation*} For $d>1$, we provide upper and lower bounds for the exponent $ν$, but these bounds do not match.

math.PR↗

CompCodeVet: A Compiler-guided Validation and Enhancement Approach for Code Dataset

Large language models (LLMs) have become increasingly prominent in academia and industry due to their remarkable performance in diverse applications. As these models evolve with increasing parameters, they excel in tasks like sentiment analysis and machine translation. However, even models with billions of parameters face challenges in tasks demanding multi-step reasoning. Code generation and comprehension, especially in C and C++, emerge as significant challenges. While LLMs trained on code datasets demonstrate competence in many tasks, they struggle with rectifying non-compilable C and C++ code. Our investigation attributes this subpar performance to two primary factors: the quality of the training dataset and the inherent complexity of the problem which demands intricate reasoning. Existing "Chain of Thought" (CoT) prompting techniques aim to enhance multi-step reasoning. This approach, however, retains the limitations associated with the latent drawbacks of LLMs. In this work, we propose CompCodeVet, a compiler-guided CoT approach to produce compilable code from non-compilable ones. Diverging from the conventional approach of utilizing larger LLMs, we employ compilers as a teacher to establish a more robust zero-shot thought process. The evaluation of CompCodeVet on two open-source code datasets shows that CompCodeVet has the ability to improve the training dataset quality for LLMs.

cs.LG↗

Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization

As a promising fashion for visual localization, scene coordinate regression (SCR) has seen tremendous progress in the past decade. Most recent methods usually adopt neural networks to learn the mapping from image pixels to 3D scene coordinates, which requires a vast amount of annotated training data. We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for SCR. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain, which can hinder the regression accuracy or bring unnecessary computational costs with redundant data. These challenges are addressed in three folds in this paper: (1) A NeRF is designed to separately predict uncertainties for the rendered color and depth images, which reveal data reliability at the pixel level. (2) SCR is formulated as deep evidential learning with epistemic uncertainty, which is used to evaluate information gain and scene coordinate quality. (3) Based on the three arts of uncertainties, a novel view selection policy is formed that significantly improves data efficiency. Experiments on public datasets demonstrate that our method could select the samples that bring the most information gain and promote the performance with the highest efficiency.

cs.CV↗

Global solution for superlinear stochastic heat equation on $\mathbb{R}^d$ under Osgood-type conditions

We study the \textit{stochastic heat equation} (SHE) on $\R^d$ subject to a centered Gaussian noise that is white in time and colored in space.The drift term is assumed to satisfy an Osgood-type condition and the diffusion coefficient may have certain related growth. We show that there exists random field solution which do not explode in finite time. This complements and improves upon recent results on blow-up of solutions to stochastic partial differential equations.

math.PR↗

Data Race Detection Using Large Language Models

Large language models (LLMs) are demonstrating significant promise as an alternate strategy to facilitate analyses and optimizations of high-performance computing programs, circumventing the need for resource-intensive manual tool creation. In this paper, we explore a novel LLM-based data race detection approach combining prompting engineering and fine-tuning techniques. We create a dedicated dataset named DRB-ML, which is derived from DataRaceBench, with fine-grain labels showing the presence of data race pairs and their associated variables, line numbers, and read/write information. DRB-ML is then used to evaluate representative LLMs and fine-tune open-source ones. Our experiment shows that LLMs can be a viable approach to data race detection. However, they still cannot compete with traditional data race detection tools when we need detailed information about variable pairs causing data races.

cs.LG↗

HPC-GPT: Integrating Large Language Model for High-Performance Computing

Large Language Models (LLMs), including the LLaMA model, have exhibited their efficacy across various general-domain natural language processing (NLP) tasks. However, their performance in high-performance computing (HPC) domain tasks has been less than optimal due to the specialized expertise required to interpret the model responses. In response to this challenge, we propose HPC-GPT, a novel LLaMA-based model that has been supervised fine-tuning using generated QA (Question-Answer) instances for the HPC domain. To evaluate its effectiveness, we concentrate on two HPC tasks: managing AI models and datasets for HPC, and data race detection. By employing HPC-GPT, we demonstrate comparable performance with existing methods on both tasks, exemplifying its excellence in HPC-related scenarios. Our experiments on open-source benchmarks yield extensive results, underscoring HPC-GPT's potential to bridge the performance gap between LLMs and HPC-specific tasks. With HPC-GPT, we aim to pave the way for LLMs to excel in HPC domains, simplifying the utilization of language models in complex computing applications.

cs.DC↗

Creating a Dataset for High-Performance Computing Code Translation using LLMs: A Bridge Between OpenMP Fortran and C++

In this study, we present a novel dataset for training machine learning models translating between OpenMP Fortran and C++ code. To ensure reliability and applicability, the dataset is created from a range of representative open-source OpenMP benchmarks. It is also refined using a meticulous code similarity test. The effectiveness of our dataset is assessed using both quantitative (CodeBLEU) and qualitative (human evaluation) methods. We showcase how this dataset significantly elevates the translation competencies of large language models (LLMs). Specifically, models without prior coding knowledge experienced a boost of $\mathbf{\times~5.1}$ in their CodeBLEU scores, while models with some coding familiarity saw an impressive $\mathbf{\times~9.9}$-fold increase. The best fine-tuned model using our dataset outperforms GPT-4. It is also reaching human-level accuracy. This work underscores the immense potential of our dataset in propelling advancements in the domain of code translation for high-performance computing. The dataset is accessible at \href{https://github.com/bin123apple/Fortran-CPP-HPC-code-translation-dataset}{OpenMP-Fortran-CPP-Translation}.

cs.SE↗

Parabolic Anderson model with colored noise on torus

We construct an intrinsic family of Gaussian noises on $d$-dimensional flat torus $\mathbb{T}^d$. It is the analogue of the colored noise on $\mathbb{R}^d$, and allows us to study stochastic PDEs on torus in the Itô sense in high dimensions. With this noise, we consider the parabolic Anderson model (PAM) with measure-valued initial conditions and establish some basic properties of the solution, including a sharp upper and lower bound for the moments and Hölder continuity in space and time. The study of the toy model of $\mathbb{T}^d$ in the present paper is a first step towards our effort in understanding how geometry and topology play an role in the behavior of stochastic PDEs on general (compact) manifolds.

math.PR↗

DataRaceBench V1.4.1 and DataRaceBench-ML V0.1: Benchmark Suites for Data Race Detection

Data races pose a significant threat in multi-threaded parallel applications due to their negative impact on program correctness. DataRaceBench, an open-source benchmark suite, is specifically crafted to assess these data race detection tools in a systematic and measurable manner. Machine learning techniques have recently demonstrated considerable potential in high-performance computing (HPC) program analysis and optimization. However, these techniques require specialized data formats for training and refinement. This paper presents the latest update to DataRaceBench, incorporating new data race contributions from Wu et al. \cite{wu2023model}, and introduces a derived dataset named DataRaceBench-ML (DRB-ML) \cite{drbml}. DRB-ML aligns with the emerging trend of machine learning and large language models. Originating from DataRaceBench, this dataset includes detailed labels that denote the presence of a data race and provides comprehensive details of associated variables, such as variable names, line numbers, and the operation (read/write). Unique to DRB-ML, we have also integrated a series of tailored prompt-response pairs specifically designed for LLM fine-tuning.

cs.SE↗

Parabolic stochastic PDEs on bounded domains with rough initial conditions: moment and correlation bounds

We consider nonlinear parabolic stochastic PDEs on a bounded Lipschitz domain driven by a Gaussian noise that is white in time and colored in space, with Dirichlet or Neumann boundary condition. We establish existence, uniqueness and moment bounds of the random field solution under measure-valued initial data $ν$. We also study the two-point correlation function of the solution and obtain explicit upper and lower bounds. For $C^{1, α}$-domains with Dirichlet condition, the initial data $ν$ is not required to be a finite measure and the moment bounds can be improved under the weaker condition that the leading eigenfunction of the differential operator is integrable with respect to $|ν|$. As an application, we show that the solution is fully intermittent for sufficiently high level $λ$ of noise under the Dirichlet condition, and for all $λ> 0$ under the Neumann condition.

math.PR↗

LM4HPC: Towards Effective Language Model Application in High-Performance Computing

In recent years, language models (LMs), such as GPT-4, have been widely used in multiple domains, including natural language processing, visualization, and so on. However, applying them for analyzing and optimizing high-performance computing (HPC) software is still challenging due to the lack of HPC-specific support. In this paper, we design the LM4HPC framework to facilitate the research and development of HPC software analyses and optimizations using LMs. Tailored for supporting HPC datasets, AI models, and pipelines, our framework is built on top of a range of components from different levels of the machine learning software stack, with Hugging Face-compatible APIs. Using three representative tasks, we evaluated the prototype of our framework. The results show that LM4HPC can help users quickly evaluate a set of state-of-the-art models and generate insightful leaderboards.

cs.LG↗

Asymptotic properties of stochastic partial differential equations in the sublinear regime

In this paper, we investigate stochastic heat equation with sublinear diffusion coefficients. By assuming certain concavity of the diffusion coefficient, we establish non-trivial moment upper bounds and almost sure spatial asymptotic properties for the solutions. These results shed light on the smoothing intermittency effect under weak diffusion (i.e., sublinear growth) previously observed by Zeldovich et al. [Zel+87]. The sample-path spatial asymptotics obtained in this paper partially bridge a gap in earlier works of Conus et al. [CJK13; Con+13], which focused on two extreme scenarios: a linear diffusion coefficient and a bounded diffusion coefficient. Our approach is highly robust and applicable to a variety of stochastic partial differential equations, including the one-dimensional stochastic wave equation and the stochastic fractional diffusion equations.

math.PR↗

Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation

Detecting parallelizable code regions is a challenging task, even for experienced developers. Numerous recent studies have explored the use of machine learning for code analysis and program synthesis, including parallelization, in light of the success of machine learning in natural language processing. However, applying machine learning techniques to parallelism detection presents several challenges, such as the lack of an adequate dataset for training, an effective code representation with rich information, and a suitable machine learning model to learn the latent features of code for diverse analyses. To address these challenges, we propose a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code. The proposed approach primarily focused on loop-level parallelization with OpenMP. Moreover, we create an OMP\_Serial dataset with 18598 parallelizable and 13972 non-parallelizable loops to train the machine learning models. Our results show that our proposed approach achieves the accuracy of parallelizable code region detection with 85\% accuracy and outperforms the state-of-the-art token-based machine learning approach. These results indicate that our approach is competitive with state-of-the-art tools and capable of handling loops with complex structures that other tools may overlook.

cs.LG↗

Spatial ergodicity and central limit theorems for parabolic Anderson model with delta initial condition

Let $\{u(t\,, x)\}_{t >0, x \in\mathbb{R}}$ denote the solution to the parabolic Anderson model with initial condition $δ_0$ and driven by space-time white noise on $\mathbb{R}_+\times\mathbb{R}$, and let $p_t(x):= (2πt)^{-1/2}\exp\{-x^2/(2t)\}$ denote the standard Gaussian heat kernel on the line. We use a non-trivial adaptation of the methods in our companion papers \cite{CKNP,CKNP_b} in order to prove that the random field $x\mapsto u(t\,,x)/p_t(x)$ is ergodic for every $t >0$. And we establish an associated quantitative central limit theorem following the approach based on the Malliavin-Stein method introduced in Huang, Nualart, and Viitasaari \cite{HNV2018}.

math.PR↗