arXiv subjects

Chi Chen

Publications and source records attributed to Chi Chen.

At least 73 records · Page 4Linked to original sources

arXiv2024

Goldfish: An Efficient Federated Unlearning Framework

With recent legislation on the right to be forgotten, machine unlearning has emerged as a crucial research area. It facilitates the removal of a user's data from federated trained machine learning models without the necessity for retraining from scratch. However, current machine unlearning algorithms are confronted with challenges of efficiency and validity. To address the above issues, we propose a new framework, named Goldfish. It comprises four modules: basic model, loss function, optimization, and extension. To address the challenge of low validity in existing machine unlearning algorithms, we propose a novel loss function. It takes into account the loss arising from the discrepancy between predictions and actual labels in the remaining dataset. Simultaneously, it takes into consideration the bias of predicted results on the removed dataset. Moreover, it accounts for the confidence level of predicted results. Additionally, to enhance efficiency, we adopt knowledge a distillation technique in the basic model and introduce an optimization module that encompasses the early termination mechanism guided by empirical risk and the data partition mechanism. Furthermore, to bolster the robustness of the aggregated model, we propose an extension module that incorporates a mechanism using adaptive distillation temperature to address the heterogeneity of user local data and a mechanism using adaptive weight to handle the variety in the quality of uploaded models. Finally, we conduct comprehensive experiments to illustrate the effectiveness of proposed approach.

cs.LG↗

arXiv2024

Accelerating computational materials discovery with artificial intelligence and cloud high-performance computing: from large-scale screening to experimental validation

High-throughput computational materials discovery has promised significant acceleration of the design and discovery of new materials for many years. Despite a surge in interest and activity, the constraints imposed by large-scale computational resources present a significant bottleneck. Furthermore, examples of large-scale computational discovery carried through experimental validation remain scarce, especially for materials with product applicability. Here we demonstrate how this vision became reality by first combining state-of-the-art artificial intelligence (AI) models and traditional physics-based models on cloud high-performance computing (HPC) resources to quickly navigate through more than 32 million candidates and predict around half a million potentially stable materials. By focusing on solid-state electrolytes for battery applications, our discovery pipeline further identified 18 promising candidates with new compositions and rediscovered a decade's worth of collective knowledge in the field as a byproduct. By employing around one thousand virtual machines (VMs) in the cloud, this process took less than 80 hours. We then synthesized and experimentally characterized the structures and conductivities of our top candidates, the Na$_x$Li$_{3-x}$YCl$_6$ ($0 < x < 3$) series, demonstrating the potential of these compounds to serve as solid electrolytes. Additional candidate materials that are currently under experimental investigation could offer more examples of the computational discovery of new phases of Li- and Na-conducting solid electrolytes. We believe that this unprecedented approach of synergistically integrating AI models and cloud HPC not only accelerates materials discovery but also showcases the potency of AI-guided experimentation in unlocking transformative scientific breakthroughs with real-world applications.

cond-mat.mtrl-sci↗

arXiv2023

Filling the Image Information Gap for VQA: Prompting Large Language Models to Proactively Ask Questions

Large Language Models (LLMs) demonstrate impressive reasoning ability and the maintenance of world knowledge not only in natural language tasks, but also in some vision-language tasks such as open-domain knowledge-based visual question answering (OK-VQA). As images are invisible to LLMs, researchers convert images to text to engage LLMs into the visual question reasoning procedure. This leads to discrepancies between images and their textual representations presented to LLMs, which consequently impedes final reasoning performance. To fill the information gap and better leverage the reasoning capability, we design a framework that enables LLMs to proactively ask relevant questions to unveil more details in the image, along with filters for refining the generated information. We validate our idea on OK-VQA and A-OKVQA. Our method continuously boosts the performance of baselines methods by an average gain of 2.15% on OK-VQA, and achieves consistent improvements across different LLMs.

cs.CL↗

arXiv2023

Overcoming the Size Limit of First Principles Molecular Dynamics Simulations with an In-Distribution Substructure Embedding Active Learner

Large-scale first principles molecular dynamics are crucial for simulating complex processes in chemical, biomedical, and materials sciences. However, the unfavorable time complexity with respect to system sizes leads to prohibitive computational costs when the simulation contains over a few hundred atoms in practice. We present an In-Distribution substructure Embedding Active Learner (IDEAL) to enable efficient simulation of large complex systems with quantum accuracy by maintaining a machine learning force field (MLFF) as an accurate surrogate to the first principles methods. By extracting high-uncertainty substructures into low-uncertainty atom environments, the active learner is allowed to concentrate on and learn from small substructures of interest rather than carrying out intractable quantum chemical computations on large structures. IDEAL is benchmarked on various systems and shows sub-linear complexity, accelerating the simulation thousands of times compared with conventional active learning and millions of times compared with pure first principles simulations. To demonstrate the capability of IDEAL in practical applications, we simulated a polycrystalline lithium system composed of one million atoms and the full ammonia formation process in a Haber-Bosch reaction on a 3-nm Iridium nanoparticle catalyst on a computing node comprising one single A100 GPU and 24 CPU cores.

cond-mat.mtrl-sci↗

arXiv2023

Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models

Recently, Multimodal Large Language Models (MLLMs) that enable Large Language Models (LLMs) to interpret images through visual instruction tuning have achieved significant success. However, existing visual instruction tuning methods only utilize image-language instruction data to align the language and image modalities, lacking a more fine-grained cross-modal alignment. In this paper, we propose Position-enhanced Visual Instruction Tuning (PVIT), which extends the functionality of MLLMs by integrating an additional region-level vision encoder. This integration promotes a more detailed comprehension of images for the MLLM. In addition, to efficiently achieve a fine-grained alignment between the vision modules and the LLM, we design multiple data generation strategies to construct an image-region-language instruction dataset. Finally, we present both quantitative experiments and qualitative analysis that demonstrate the superiority of the proposed model. Code and data will be released at https://github.com/PVIT-official/PVIT.

cs.CV↗

arXiv2023

Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning

Advances in deep learning have greatly improved structure prediction of molecules. However, many macroscopic observations that are important for real-world applications are not functions of a single molecular structure, but rather determined from the equilibrium distribution of structures. Traditional methods for obtaining these distributions, such as molecular dynamics simulation, are computationally expensive and often intractable. In this paper, we introduce a novel deep learning framework, called Distributional Graphormer (DiG), in an attempt to predict the equilibrium distribution of molecular systems. Inspired by the annealing process in thermodynamics, DiG employs deep neural networks to transform a simple distribution towards the equilibrium distribution, conditioned on a descriptor of a molecular system, such as a chemical graph or a protein sequence. This framework enables efficient generation of diverse conformations and provides estimations of state densities. We demonstrate the performance of DiG on several molecular tasks, including protein conformation sampling, ligand structure sampling, catalyst-adsorbate sampling, and property-guided structure generation. DiG presents a significant advancement in methodology for statistically understanding molecular systems, opening up new research opportunities in molecular science.

physics.chem-ph↗

arXiv2023

Weakly Supervised Vision-and-Language Pre-training with Relative Representations

Weakly supervised vision-and-language pre-training (WVLP), which learns cross-modal representations with limited cross-modal supervision, has been shown to effectively reduce the data cost of pre-training while maintaining decent performance on downstream tasks. However, current WVLP methods use only local descriptions of images, i.e., object tags, as cross-modal anchors to construct weakly-aligned image-text pairs for pre-training. This affects the data quality and thus the effectiveness of pre-training. In this paper, we propose to directly take a small number of aligned image-text pairs as anchors, and represent each unaligned image and text by its similarities to these anchors, i.e., relative representations. We build a WVLP framework based on the relative representations, namely RELIT, which collects high-quality weakly-aligned image-text pairs from large-scale image-only and text-only data for pre-training through relative representation-based retrieval and generation. Experiments on four downstream tasks show that RELIT achieves new state-of-the-art results under the weakly supervised setting.

cs.CV↗

arXiv2022

Observation of Gapped Topological Surface States and Isolated Surface Resonances in PdTe$_2$ Ultrathin Films

The superconductor PdTe$_2$ is known to host bulk Dirac bands and topological surface states. The coexistence of superconductivity and topological surface states makes PdTe$_2$ a promising platform for exploring topological superconductivity and Majorana bound states. In this work, we report the layer-by-layer molecular beam epitaxy growth and spectroscopic characterization of high quality PdTe$_2$ films with thickness down to 3 monolayers (ML). In the 3 ML PdTe$_2$ film, we observed spin-polarized surface resonance states, which are isolated from the bulk bands due to the quantum size effects. In addition, the hybridization of surface states on opposite faces leads to a thickness-dependent gap in the topological surface Dirac bands. Our photoemission results show clearly that the size of the hybridization gap increases as the film thickness is reduced. The success in growing high quality PdTe$_2$ films by state-of-art molecular beam epitaxy technique and the observation of surface resonances and gaped topological surface states sheds light on the applications of PdTe$_2$ quantum films in spintronics and topological computation.

cond-mat.mes-hall↗

arXiv2022

The Intercalation Chemistry of the Disordered RockSalt Li3V2O5 Anode from Cluster Expansions and Machine Learning Interatomic Potentials

Disordered rocksalt (DRX) Li3V2O5 is a promising candidate for anode in rechargeable lithium-ion batteries because of its ideal low voltage, high rate capability, and superior cycling stability. Herein, we presents a comprehensive study of intercalation chemistry of the DRX-Li3V2O5 anode using density functional theory calculations combined with machine learning cluster expansions and interatomic potentials. The predicted voltage profile of the disordered Li3V2O5 anode at room temperature based on Monte Carlo simulations with a fitted cluster expansion model is in excellent agreement with experiments. In contrast to previous DFT results, we find that Li ions predominately intercalate into tetrahedral sites during charging, while the majority of Li and V ions at octahedral sites remain stable. In addition, MD simulations with a fitted moment tensor potential attribute the fast-charging capability of DRX-Li3V2O5 to the facile diffusivity of Li+ via tetrahedral - octahedral - tetrahedral pathway. We further suggest tuning the Li:V ratio as a means to trade off increased lithiation capacity and decreased anode voltage in this system. This work provides in-depth insights into the high-performance DRX-Li3V2O5 anode, and paves the way to the discovery of other disordered anode materials.

cond-mat.mtrl-sci↗

arXiv2022

A Universal Graph Deep Learning Interatomic Potential for the Periodic Table

Interatomic potentials (IAPs), which describe the potential energy surface of atoms, are a fundamental input for atomistic simulations. However, existing IAPs are either fitted to narrow chemistries or too inaccurate for general applications. Here, we report a universal IAP for materials based on graph neural networks with three-body interactions (M3GNet). The M3GNet IAP was trained on the massive database of structural relaxations performed by the Materials Project over the past 10 years and has broad applications in structural relaxation, dynamic simulations and property prediction of materials across diverse chemical spaces. About 1.8 million materials were identified from a screening of 31 million hypothetical crystal structures to be potentially stable against existing Materials Project crystals based on M3GNet energies. Of the top 2000 materials with the lowest energies above hull, 1578 were verified to be stable using DFT calculations. These results demonstrate a machine learning-accelerated pathway to the discovery of synthesizable materials with exceptional properties.

cond-mat.mtrl-sci↗

arXiv2022

Multi-scale Investigation of Chemical Short-Range Order and Dislocation Glide in the MoNbTi and TaNbTi Refractory Multi-Principal Element Alloys

Refractory multi-principal element alloys (RMPEAs) are promising materials for high-temperature structural applications. Here, we investigate the role of chemical short-range ordering (CSRO) on dislocation glide in two model RMPEAs - TaNbTi and MoNbTi - using a multi-scale modeling approach. A highly accurate machine learning interatomic potential was developed for the Mo-Ta-Nb-Ti system and used to demonstrate that MoNbTi exhibits a much greater degree of SRO than TaNbTi and the local composition has a direct effect on the unstable stacking fault energies (USFE). From mesoscale phase-field dislocation dynamics simulations, we find that increasing SRO leads to higher mean USFEs, thereby increasing the stress required for dislocation glide. The gliding dislocations experience significant hardening due to pinning and depinning caused by random compositional fluctuations, with higher SRO decreasing the degree of USFE dispersion and hence, amount of hardening. Finally, we show how the morphology of an expanding dislocation loop is affected by the applied stress, with higher SRO requiring higher applied stresses to achieve smooth screw dislocation glide.

cond-mat.mtrl-sci↗

arXiv2022

CG-SSD: Corner Guided Single Stage 3D Object Detection from LiDAR Point Cloud

At present, the anchor-based or anchor-free models that use LiDAR point clouds for 3D object detection use the center assigner strategy to infer the 3D bounding boxes. However, in a real world scene, the LiDAR can only acquire a limited object surface point clouds, but the center point of the object does not exist. Obtaining the object by aggregating the incomplete surface point clouds will bring a loss of accuracy in direction and dimension estimation. To address this problem, we propose a corner-guided anchor-free single-stage 3D object detection model (CG-SSD ).Firstly, 3D sparse convolution backbone network composed of residual layers and sub-manifold sparse convolutional layers are used to construct bird's eye view (BEV) features for further deeper feature mining by a lite U-shaped network; Secondly, a novel corner-guided auxiliary module (CGAM) is proposed to incorporate corner supervision signals into the neural network. CGAM is explicitly designed and trained to detect partially visible and invisible corners to obtains a more accurate object feature representation, especially for small or partial occluded objects; Finally, the deep features from both the backbone networks and CGAM module are concatenated and fed into the head module to predict the classification and 3D bounding boxes of the objects in the scene. The experiments demonstrate CG-SSD achieves the state-of-art performance on the ONCE benchmark for supervised 3D object detection using single frame point cloud data, with 62.77%mAP. Additionally, the experiments on ONCE and Waymo Open Dataset show that CGAM can be extended to most anchor-based models which use the BEV feature to detect objects, as a plug-in and bring +1.17%-+14.27%AP improvement.

cs.CV↗

arXiv2022

A Universal Machine Learning Model for Elemental Grain Boundary Energies

The grain boundary (GB) energy has a profound influence on the grain growth and properties of polycrystalline metals. Here, we show that the energy of a GB, normalized by the bulk cohesive energy, can be described purely by four geometric features. By machine learning on a large computed database of 361 small $Σ$ ($Σ< 10$) GBs of more than 50 metals, we develop a model that can predict the grain boundary energies to within a mean absolute error of 0.13 J m$^{-2}$. More importantly, this universal GB energy model can be extrapolated to the energies of high $Σ$ GBs without loss in accuracy. These results highlight the importance of capturing fundamental scaling physics and domain knowledge in the design of interpretable, extrapolatable machine learning models for materials science.

cond-mat.mtrl-sci↗

arXiv2022

Visualization of Band Shifting and Interlayer Coupling in WxMo1-xS2 Alloys using Near-Field Broadband Absorption Microscopy

Beyond-diffraction-limit optical absorption spectroscopy provides profound information on the graded band structures of composition-spread and stacked two-dimensional materials, in which direct/indirect bandgap, interlayer coupling, sliding, and possible defects significantly modify their optoelectronic functionalities such as photoluminescence efficiency. We here visualize the spatially-varying band structure of monolayer and bilayer transition metal dichalcogenide alloys for the first time by using near-field broadband absorption microscopy. The near-field-spectral and -spatial diagrams manifest the excitonic band shift that results from the interplay of composition spreading and interlayer coupling. These results enable us to identify the top layer of the bilayer alloy as pure WS2. We also use the aberration-free near-field transmittance images to demarcate the exact boundaries of alloyed and pure transition metal dichalcogenides. This technology can offer new insights on various layered structures in the era of stacking science in quest of novel quantum optoelectronic devices.

cond-mat.mtrl-sci↗

arXiv2021

Recent Advances and Applications of Deep Learning Methods in Materials Science

Deep learning (DL) is one of the fastest growing topics in materials data science, with rapidly emerging applications spanning atomistic, image-based, spectral, and textual data modalities. DL allows analysis of unstructured data and automated identification of features. Recent development of large materials databases has fueled the application of DL methods in atomistic prediction in particular. In contrast, advances in image and spectral data have largely leveraged synthetic data enabled by high quality forward models as well as by generative unsupervised DL methods. In this article, we present a high-level overview of deep-learning methods followed by a detailed discussion of recent developments of deep learning in atomistic simulation, materials imaging, spectral analysis, and natural language processing. For each modality we discuss applications involving both theoretical and experimental data, typical modeling approaches with their strengths and limitations, and relevant publicly available software and datasets. We conclude the review with a discussion of recent cross-cutting work related to uncertainty quantification in this field and a brief perspective on limitations, challenges, and potential growth areas for DL methods in materials science. The application of DL methods in materials science presents an exciting avenue for future materials discovery and design.

cond-mat.mtrl-sci↗

arXiv2021

A Topic Guided Pointer-Generator Model for Generating Natural Language Code Summaries

Code summarization is the task of generating natural language description of source code, which is important for program understanding and maintenance. Existing approaches treat the task as a machine translation problem (e.g., from Java to English) and applied Neural Machine Translation models to solve the problem. These approaches only consider a given code unit (e.g., a method) without its broader context. The lacking of context may hinder the NMT model from gathering sufficient information for code summarization. Furthermore, existing approaches use a fixed vocabulary and do not fully consider the words in code, while many words in the code summary may come from the code. In this work, we present a neural network model named ToPNN for code summarization, which uses the topics in a broader context (e.g., class) to guide the neural networks that combine the generation of new words and the copy of existing words in code. Based on the model we present an approach for generating natural language code summaries at the method level (i.e., method comments). We evaluate our approach using a dataset with 4,203,565 commented Java methods. The results show significant improvement over state-of-the-art approaches and confirm the positive effect of class topics and the copy mechanism.

cs.SE↗

arXiv2021

Bridging the Gap Between Simulated and Experimental Ionic Conductivities in Lithium Superionic Conductors

Lithium superionic conductors (LSCs) are of major importance as solid electrolytes for next-generation all-solid-state lithium-ion batteries. While $ab$ $initio$ molecular dynamics have been extensively applied to study these materials, there are often large discrepancies between predicted and experimentally measured ionic conductivities and activation energies due to the high temperatures and short time scales of such simulations. Here, we present a strategy to bridge this gap using moment tensor potentials (MTPs). We show that MTPs trained on energies and forces computed using the van der Waals optB88 functional yield much more accurate lattice parameters, which in turn leads to accurate prediction of ionic conductivities and activation energies for the Li$_{0.33}$La$_{0.56}$TiO$_3$, Li$_3$YCl$_6$ and Li$_7$P$_3$S$_{11}$ LSCs. NPT MD simulations using the optB88 MTPs also reveal that all three LSCs undergo a transition between two quasi-linear Arrhenius regimes at relatively low temperatures. This transition can be traced to an expansion in the number and diversity of diffusion pathways, in some cases with a change in the dimensionality of diffusion. This work presents not only an approach to develop high accuracy MTPs, but also outlines the diffusion characteristics for LSCs which is otherwise inaccessible through $ab$ $initio$ computation.

cond-mat.mtrl-sci↗

arXiv2021

Mask-Align: Self-Supervised Neural Word Alignment

Word alignment, which aims to align translationally equivalent words between source and target sentences, plays an important role in many natural language processing tasks. Current unsupervised neural alignment methods focus on inducing alignments from neural machine translation models, which does not leverage the full context in the target sequence. In this paper, we propose Mask-Align, a self-supervised word alignment model that takes advantage of the full context on the target side. Our model masks out each target token and predicts it conditioned on both source and the remaining target tokens. This two-step process is based on the assumption that the source token contributing most to recovering the masked target token should be aligned. We also introduce an attention variant called leaky attention, which alleviates the problem of unexpected high cross-attention weights on special tokens such as periods. Experiments on four language pairs show that our model outperforms previous unsupervised neural aligners and obtains new state-of-the-art results.

cs.CL↗