arXiv subjects

Xu Chu

Publications and source records attributed to Xu Chu.

At least 91 records · Page 5Linked to original sources

arXiv2024

TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation

A practical dialogue system requires the capacity for ongoing skill acquisition and adaptability to new tasks while preserving prior knowledge. However, current methods for Continual Dialogue State Tracking (DST), a crucial function of dialogue systems, struggle with the catastrophic forgetting issue and knowledge transfer between tasks. We present TaSL, a novel framework for task skill localization and consolidation that enables effective knowledge transfer without relying on memory replay. TaSL uses a novel group-wise technique to pinpoint task-specific and task-shared areas. Additionally, a fine-grained skill consolidation strategy protects task-specific knowledge from being forgotten while updating shared knowledge for bi-directional knowledge transfer. As a result, TaSL strikes a balance between preserving previous knowledge and excelling at new tasks. Comprehensive experiments on various backbones highlight the significant performance improvements of TaSL over existing state-of-the-art methods. The source code is provided for reproducibility.

cs.CL↗

arXiv2024

Microscale physics and macroscale convective heat transfer in supercritical fluids

Driven by fundamental thermodynamic efficiency considerations, an emerging trend in the energy and propulsion systems is that the working fluid operates at a pressure above the critical pressure. Energy transport is thus accompanied by dramatic and strongly nonlinear variations of thermophysical properties, which cause abnormal heat transfer behavior and non-ideal gas effects. This situation raises a crucial challenge for the heat exchanger and turbomachinery design, overall energy and exergy efficiency. We aim to provide a multi-scale overview of the flow and thermal behavior of fluid above the critical point: microscopic physics and macroscopic transport. Microscopic physics, i.e. near-critical thermophysical properties, phase transition and fluid dynamics, are introduced. A particular focus will be on the supercritical pseudo boiling process, which is a generalization of classical liquid-vapor phase transitions to non-equilibrium supercritical states. These new perspectives lead to a revised view of the state space. Further, recent results demonstrated the possibility of stable supercritical fluid interfaces without surface tension. On the macroscale, recent progress on the modeling of turbulent flow and convective heat transfer of supercritical fluids are summarized. Direct numerical simulation is able to offer insights into the physics of thermal fluids. We start with a description of fundamental fluid mechanics problems related to supercritical fluids. In addition, the heat transfer deterioration in supercritical fluids is found to be closely connected to the flow relaminarization by the non-uniform body-force. Finally, various modeling approaches such as recently developed advanced Reynolds-averaged turbulence modeling as well as machine-learning methods are summarized.

physics.flu-dyn↗

arXiv2024

SketchQL Demonstration: Zero-shot Video Moment Querying with Sketches

In this paper, we will present SketchQL, a video database management system (VDBMS) for retrieving video moments with a sketch-based query interface. This novel interface allows users to specify object trajectory events with simple mouse drag-and-drop operations. Users can use trajectories of single objects as building blocks to compose complex events. Using a pre-trained model that encodes trajectory similarity, SketchQL achieves zero-shot video moments retrieval by performing similarity searches over the video to identify clips that are the most similar to the visual query. In this demonstration, we introduce the graphic user interface of SketchQL and detail its functionalities and interaction mechanisms. We also demonstrate the end-to-end usage of SketchQL from query composition to video moments retrieval using real-world scenarios.

cs.DB↗

arXiv2024

Data-driven methods for flow and transport in porous media: a review

This review examined the current advancements in data-driven methods for analyzing flow and transport in porous media, which has various applications in energy, chemical engineering, environmental science, and beyond. Although there has been progress in recent years, the challenges of current experimental and high-fidelity numerical simulations, such as high computational costs and difficulties in accurately representing complex, heterogeneous structures, can still potentially be addressed by state-of-the-art data-driven methods. We analyzed the synergistic potential of these methods, addressed their limitations, and suggested how they can be effectively integrated to improve both the fidelity and efficiency of current research. A discussion on future research directions in this field was conducted, emphasizing the need for collaborative efforts that combine domain expertise in physics and advanced computationald and data-driven methodologies.

physics.flu-dyn↗

arXiv2024

JAX-based differentiable fluid dynamics on GPU and end-to-end optimization

This project aims to advance differentiable fluid dynamics for hypersonic coupled flow over porous media, demonstrating the potential of automatic differentiation (AD)-based optimization for end-to-end solutions. Leveraging AD efficiently handles high-dimensional optimization problems, offering a flexible alternative to traditional methods. We utilized JAX-Fluids, a newly developed solver based on the JAX framework, which combines autograd and TensorFlow's XLA. Compiled on a HAWK-AI node with NVIDIA A100 GPU, JAX-Fluids showed computational performance comparable to other high-order codes like FLEXI. Validation with a compressible turbulent channel flow DNS case showed excellent agreement, and a new boundary condition for modeling porous media was successfully tested on a laminar boundary layer case. Future steps in our research are anticipated.

physics.flu-dyn↗

arXiv2024

Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation

With the increasingly powerful performances and enormous scales of pretrained models, promoting parameter efficiency in fine-tuning has become a crucial need for effective and efficient adaptation to various downstream tasks. One representative line of fine-tuning methods is Orthogonal Fine-tuning (OFT), which rigorously preserves the angular distances within the parameter space to preserve the pretrained knowledge. Despite the empirical effectiveness, OFT still suffers low parameter efficiency at $\mathcal{O}(d^2)$ and limited capability of downstream adaptation. Inspired by Givens rotation, in this paper, we proposed quasi-Givens Orthogonal Fine-Tuning (qGOFT) to address the problems. We first use $\mathcal{O}(d)$ Givens rotations to accomplish arbitrary orthogonal transformation in $SO(d)$ with provable equivalence, reducing parameter complexity from $\mathcal{O}(d^2)$ to $\mathcal{O}(d)$. Then we introduce flexible norm and relative angular adjustments under soft orthogonality regularization to enhance the adaptation capability of downstream semantic deviations. Extensive experiments on various tasks and pretrained models validate the effectiveness of our methods.

cs.LG↗

arXiv2024

An investigation of anisotropy in the bubbly turbulent flow via direct numerical simulations

This study explores the dynamics of dispersed bubbly turbulent flow in a channel using interface-resolved direct numerical simulation (DNS) with an efficient Coupled Level-Set Volume-of-Fluid (CLSVOF) solver. The influence of number of bubbles (96 and 192), flow direction, and Eotvos number was examined across eight distinct cases. The results indicate that in upward flows, bubbles tend to accumulate near the wall, with smaller Eotvos numbers bringing them closer to the wall and enhancing energy dissipation through increased turbulence and vorticity. This proximity causes the liquid phase velocity to attenuate, and the bubbles, being more spherical, induce more isotropic turbulence. Conversely, in downward flows, bubbles cluster in the middle of the channel and induce additional pseudo-turbulence in the channel center, which induce additional turbulent kinetic energy in the channel center. The study further examines budget of Turbulent Kinetic Energy (TKE) and the exact balance equation for the Reynolds stresses, revealing that near-wall bubble motion generates substantial velocity gradients, particularly in the wall-normal direction, significantly impacting the turbulence structure.

physics.flu-dyn↗

arXiv2024

SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction

Electronic health record (EHR) data has emerged as a valuable resource for analyzing patient health status. However, the prevalence of missing data in EHR poses significant challenges to existing methods, leading to spurious correlations and suboptimal predictions. While various imputation techniques have been developed to address this issue, they often obsess unnecessary details and may introduce additional noise when making clinical predictions. To tackle this problem, we propose SMART, a Self-Supervised Missing-Aware RepresenTation Learning approach for patient health status prediction, which encodes missing information via elaborated attentions and learns to impute missing values through a novel self-supervised pre-training approach that reconstructs missing data representations in the latent space. By adopting missing-aware attentions and focusing on learning higher-order representations, SMART promotes better generalization and robustness to missing data. We validate the effectiveness of SMART through extensive experiments on six EHR tasks, demonstrating its superiority over state-of-the-art methods.

cs.LG↗

arXiv2024

HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses

In this paper, we investigate the retrieval-augmented generation (RAG) based on Knowledge Graphs (KGs) to improve the accuracy and reliability of Large Language Models (LLMs). Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization. To this end, we develop a Hypothesis Knowledge Graph Enhanced (HyKGE) framework, which leverages LLMs' powerful reasoning capacity to compensate for the incompleteness of user queries, optimizes the interaction process with LLMs, and provides diverse retrieved knowledge. Specifically, HyKGE explores the zero-shot capability and the rich knowledge of LLMs with Hypothesis Outputs to extend feasible exploration directions in the KGs, as well as the carefully curated prompt to enhance the density and efficiency of LLMs' responses. Furthermore, we introduce the HO Fragment Granularity-aware Rerank Module to filter out noise while ensuring the balance between diversity and relevance in retrieved knowledge. Experiments on two Chinese medical multiple-choice question datasets and one Chinese open-domain medical Q&A dataset with two LLM turbos demonstrate the superiority of HyKGE in terms of accuracy and explainability.

cs.CL↗

arXiv2024

Infinite-Horizon Graph Filters: Leveraging Power Series to Enhance Sparse Information Aggregation

Graph Neural Networks (GNNs) have shown considerable effectiveness in a variety of graph learning tasks, particularly those based on the message-passing approach in recent years. However, their performance is often constrained by a limited receptive field, a challenge that becomes more acute in the presence of sparse graphs. In light of the power series, which possesses infinite expansion capabilities, we propose a novel Graph Power Filter Neural Network (GPFN) that enhances node classification by employing a power series graph filter to augment the receptive field. Concretely, our GPFN designs a new way to build a graph filter with an infinite receptive field based on the convergence power series, which can be analyzed in the spectral and spatial domains. Besides, we theoretically prove that our GPFN is a general framework that can integrate any power series and capture long-range dependencies. Finally, experimental results on three datasets demonstrate the superiority of our GPFN over state-of-the-art baselines.

cs.LG↗

arXiv2024

LoRA Dropout as a Sparsity Regularizer for Overfitting Control

Parameter-efficient fine-tuning methods, represented by LoRA, play an essential role in adapting large-scale pre-trained models to downstream tasks. However, fine-tuning LoRA-series models also faces the risk of overfitting on the training dataset, and yet there's still a lack of theoretical guidance and practical mechanism to control overfitting on LoRA-based PEFT methods. In this paper, we propose a LoRA Dropout mechanism for the LoRA-based methods by introducing random noises to the learnable low-rank matrices and increasing parameter sparsity. We then demonstrate the theoretical mechanism of our LoRA Dropout mechanism from the perspective of sparsity regularization by providing a generalization error bound under this framework. Theoretical results show that appropriate sparsity would help tighten the gap between empirical and generalization risks and thereby control overfitting. Furthermore, based on the LoRA Dropout framework, we introduce a test-time ensemble strategy and provide theoretical evidence demonstrating that the ensemble method can further compress the error bound, and lead to better performance during inference time. Extensive experiments on various NLP tasks provide practical validations of the effectiveness of our LoRA Dropout framework in improving model accuracy and calibration.

cs.LG↗

arXiv2024

Exploring the Potential of Large Language Models in Graph Generation

Large language models (LLMs) have achieved great success in many fields, and recent works have studied exploring LLMs for graph discriminative tasks such as node classification. However, the abilities of LLMs for graph generation remain unexplored in the literature. Graph generation requires the LLM to generate graphs with given properties, which has valuable real-world applications such as drug discovery, while tends to be more challenging. In this paper, we propose LLM4GraphGen to explore the ability of LLMs for graph generation with systematical task designs and extensive experiments. Specifically, we propose several tasks tailored with comprehensive experiments to address key questions regarding LLMs' understanding of different graph structure rules, their ability to capture structural type distributions, and their utilization of domain knowledge for property-based graph generation. Our evaluations demonstrate that LLMs, particularly GPT-4, exhibit preliminary abilities in graph generation tasks, including rule-based and distribution-based generation. We also observe that popular prompting methods, such as few-shot and chain-of-thought prompting, do not consistently enhance performance. Besides, LLMs show potential in generating molecules with specific properties. These findings may serve as foundations for designing good LLMs based models for graph generation and provide valuable insights and further research.

cs.LG↗

arXiv2024

Imputation with Inter-Series Information from Prototypes for Irregular Sampled Time Series

Irregularly sampled time series are ubiquitous, presenting significant challenges for analysis due to missing values. Despite existing methods address imputation, they predominantly focus on leveraging intra-series information, neglecting the potential benefits that inter-series information could provide, such as reducing uncertainty and memorization effect. To bridge this gap, we propose PRIME, a Prototype Recurrent Imputation ModEl, which integrates both intra-series and inter-series information for imputing missing values in irregularly sampled time series. Our framework comprises a prototype memory module for learning inter-series information, a bidirectional gated recurrent unit utilizing prototype information for imputation, and an attentive prototypical refinement module for adjusting imputations. We conducted extensive experiments on three datasets, and the results underscore PRIME's superiority over the state-of-the-art models by up to 26% relative improvement on mean square error.

cs.LG↗

arXiv2023

Investigation on the Performance of a Torque-driven Undulatory Swimmer with Distributed Flexibility

The current study presents a systematic investigation of the locomotion performance of a swimmer with a wide range of parameter settings. Two-dimensional simulations with the immersed boundary method are employed for the fluid-structure interaction analysis. Unlike most previous studies where the kinematics of the swimmer is predetermined, the locomotion of the current swimmer is the response of a single periodic torque applied on the anterior part. The effect of the distribution of body stiffness on swimming performance and propulsion generation is discussed with different pitch frequencies and amplitudes. An analysis of the phase-averaged vorticity field and thrust sequence is given to clarify the change of performance due to the variation of flexibility. This study demonstrates that body stiffness is a key factor that influences the performance of undulatory swimming when the pitch angle is low or moderate. The simple torque input of the current simulations provides a more direct and engineering-related insight for the future design of microrobotic swimmers.

physics.flu-dyn↗

arXiv2023

An investigation of information flux between turbulent boundary layer and porous medium

The interaction between boundary layer turbulence and a porous layer is the cornerstone to the interface engineering. In this study, the spatial resolved transfer entropy is used to assess the asymmetry of the causal interaction next to a permeable wall. The analysis was based on pore-resolved direct numerical simulation of turbulent channel flow over a cylinder array. The spatial map of transfer entropy reveals the information flux between the porous medium and arbitrary nearby position.

physics.flu-dyn↗

arXiv2023

Modeling of Coupled Turbulent Channel Porous Media Flow through a Deep Autoencoder Echo State Network Framework

In this study, we propose a novel approach, namely the combined Convolutional Deep Autoencoder Echo State Network (CDAE ESN) model, for the analysis and forecasting of dynamics and low order statistics in coupled turbulent channel porous media flows. Such systems find wide applications in industrial settings, including transpiration cooling and smart interface engineering. However, the complex geometry of coupled flow systems presents additional challenges for purely data-driven models. Our results demonstrate that the integration of deep autoencoder and echo state network techniques enables effective modeling and prediction of dominant flow behaviors, particularly within the porous domain exhibiting laminar regimes. To enhance the model s applicability across a broader range of data domains, we further employ fine-tuning on a dataset encompassing varying porosities. The achieved average statistics exhibit a reasonable agreement, underscoring the efficacy of our proposed approach.

physics.flu-dyn↗

arXiv2023

Non-intrusive, transferable model for coupled turbulent channel-porous media flow based upon neural networks

Turbulent flow over permeable interface is omnipresent featuring complex flow topology. In this work, a data driven, end to end machine learning model has been developed to model the turbulent flow in porous media. For the same, we have derived a non linear reduced order model with a deep convolution autoencoder network. This model can reduce highly resolved spatial dimensions, which is a prerequisite for direct numerical simulation. A downstream recurrent neural network has been trained to capture the temporal trend of reduced modes, thus it is able to provide future evolution of modes. We further evaluate the trained model s capability on a newer dataset with a different porosity. In such cases, fine tuning could reduce the efforts (up to two order of magnitude) to train a model with limited dataset and knowledge and still show a good agreement on the mean velocity profile. Leveraging the current model, we find that even quick fine tuning achieving an impressive order of magnitude reduction in training time by approximately still results in effective flow predictions. This promising discovery encourages the fast development of a substantial amount of data-driven models tailored for various types of porous media. The diminished training time substantially lowers the computational cost when dealing with changing porous topologies, making it feasible to systematically explore interface engineering with different types of porous media. Overall, the data driven model shows a good agreement, especially for the porous media which can aid the DNS and reduce the burden to resolve this complex domain during the simulations. The fine tuning is able to reduce the training cost significantly and maintain an acceptable accuracy when a new flow condition comes into play.

physics.flu-dyn↗

arXiv2023

Simulation and analytical modeling of high-speed droplet impact onto a surface

The fluid dynamics of liquid droplet impact on surfaces hold significant relevance to various industrial applications. However, high impact velocities introduce compressible effects, leading to material erosion. A gap in understanding and modeling these effects has motivated this study. We simulated droplet impacts on surfaces and proposed a new analytical model for impact pressure and droplet turning line, targeting at predictions for enhanced cavitation. The highly compressed liquid behind the droplet expands sideways, causing lateral jetting. As the droplet encounters a shock wave, it reflects as a rarefaction wave, leading to low-pressure zones within the droplet. These zones converge at the droplet's center, causing cavitation, which, upon collapse, induces another shock wave, contributing to erosion. Using the well-established model for the low-velocity impact shows a significant discrepancy. Hence, an analytical model for the turning line radius is introduced, incorporating the lateral jetting's characteristic length scale. Comparing our model with existing ones, our new model exhibits superior predictive accuracy.

physics.flu-dyn↗