arXiv subjects

Qiang Liu

Publications and source records attributed to Qiang Liu.

At least 181 records · Page 10Linked to original sources

arXiv2025

AdaSlicing: Adaptive Online Network Slicing under Continual Network Dynamics in Open Radio Access Networks

Open radio access networks (e.g., O-RAN) facilitate fine-grained control (e.g., near-RT RIC) in next-generation networks, necessitating advanced AI/ML techniques in handling online resource orchestration in real-time. However, existing approaches can hardly adapt to time-evolving network dynamics in network slicing, leading to significant online performance degradation. In this paper, we propose AdaSlicing, a new adaptive network slicing system, to online learn to orchestrate virtual resources while efficiently adapting to continual network dynamics. The AdaSlicing system includes a new soft-isolated RAN virtualization framework and a novel AdaOrch algorithm. We design the AdaOrch algorithm by integrating AI/ML techniques (i.e., Bayesian learning agents) and optimization methods (i.e., the ADMM coordinator). We design the soft-isolated RAN virtualization to improve the virtual resource utilization of slices while assuring the isolation among virtual resources at runtime. We implement AdaSlicing on an O-RAN compliant network testbed by using OpenAirInterface RAN, Open5GS Core, and FlexRIC near-RT RIC, with Ettus USRP B210 SDR. With extensive network experiments, we demonstrate that AdaSlicing substantially outperforms state-of-the-art works with 64.2% cost reduction and 45.5% normalized performance improvement, which verifies its high adaptability, scalability, and assurance.

cs.NI↗

arXiv2024

Fairness without Demographics through Learning Graph of Gradients

Machine learning systems are notoriously prone to biased predictions about certain demographic groups, leading to algorithmic fairness issues. Due to privacy concerns and data quality problems, some demographic information may not be available in the training data and the complex interaction of different demographics can lead to a lot of unknown minority subpopulations, which all limit the applicability of group fairness. Many existing works on fairness without demographics assume the correlation between groups and features. However, we argue that the model gradients are also valuable for fairness without demographics. In this paper, we show that the correlation between gradients and groups can help identify and improve group fairness. With an adversarial weighting architecture, we construct a graph where samples with similar gradients are connected and learn the weights of different samples from it. Unlike the surrogate grouping methods that cluster groups from features and labels as proxy sensitive attribute, our method leverages the graph structure as a soft grouping mechanism, which is much more robust to noises. The results show that our method is robust to noise and can improve fairness significantly without decreasing the overall accuracy too much.

cs.LG↗

arXiv2024

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming

Multimodal large language models (MLLMs) enhance their perceptual capabilities by integrating visual and textual information. However, processing the massive number of visual tokens incurs a significant computational cost. Existing analysis of the MLLM attention mechanisms remains shallow, leading to coarse-grain token pruning strategies that fail to effectively balance speed and accuracy. In this paper, we conduct a comprehensive investigation of MLLM attention mechanisms with LLaVA. We find that numerous visual tokens and partial attention computations are redundant during the decoding process. Based on this insight, we propose Spatial-Temporal Visual Token Trimming ($\textbf{ST}^{3}$), a framework designed to accelerate MLLM inference without retraining. $\textbf{ST}^{3}$ consists of two primary components: 1) Progressive Visual Token Pruning (\textbf{PVTP}), which eliminates inattentive visual tokens across layers, and 2) Visual Token Annealing (\textbf{VTA}), which dynamically reduces the number of visual tokens in each layer as the generated tokens grow. Together, these techniques deliver around $\mathbf{2\times}$ faster inference with only about $\mathbf{30\%}$ KV cache memory compared to the original LLaVA, while maintaining consistent performance across various datasets. Crucially, $\textbf{ST}^{3}$ can be seamlessly integrated into existing pre-trained MLLMs, providing a plug-and-play solution for efficient inference.

cs.CV↗

arXiv2024

Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agents

In industrial control systems, the generation and verification of Programmable Logic Controller (PLC) code are critical for ensuring operational efficiency and safety. While Large Language Models (LLMs) have made strides in automated code generation, they often fall short in providing correctness guarantees and specialized support for PLC programming. To address these challenges, this paper introduces Agents4PLC, a novel framework that not only automates PLC code generation but also includes code-level verification through an LLM-based multi-agent system. We first establish a comprehensive benchmark for verifiable PLC code generation area, transitioning from natural language requirements to human-written-verified formal specifications and reference PLC code. We further enhance our `agents' specifically for industrial control systems by incorporating Retrieval-Augmented Generation (RAG), advanced prompt engineering techniques, and Chain-of-Thought strategies. Evaluation against the benchmark demonstrates that Agents4PLC significantly outperforms previous methods, achieving superior results across a series of increasingly rigorous metrics. This research not only addresses the critical challenges in PLC programming but also highlights the potential of our framework to generate verifiable code applicable to real-world industrial applications.

cs.SE↗

arXiv2024

EGEAN: An Exposure-Guided Embedding Alignment Network for Post-Click Conversion Estimation

Accurate post-click conversion rate (CVR) estimation is crucial for online advertising systems. Despite significant advances in causal approaches designed to address the Sample Selection Bias problem, CVR estimation still faces challenges due to Covariate Shift. Given the intrinsic connection between the distribution of covariates in the click and non-click spaces, this study proposes an Exposure-Guided Embedding Alignment Network (EGEAN) to address estimation bias caused by covariate shift. Additionally, we propose a Parameter Varying Doubly Robust Estimator with steady-state control to handle small propensities better. Online A/B tests conducted on the Meituan advertising system demonstrate that our method significantly outperforms baseline models with respect to CVR and GMV, validating its effectiveness. Code is available: https://github.com/hydrogen-maker/EGEAN.

cs.LG↗

arXiv2024

SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization

Large Language Models (LLMs) exhibit impressive performance across various tasks, but deploying them for inference poses challenges. Their high resource demands often necessitate complex, costly multi-GPU pipelines, or the use of smaller, less capable models. While quantization offers a promising solution utilizing lower precision for model storage, existing methods frequently experience significant performance drops at lower precision levels. Additionally, they typically provide only a limited set of solutions at specific bit levels, many of which are extensively manually tuned. To address these challenges, we propose a new method called SKIM: Scaled K-means clustering wIth Mixed precision. Our approach introduces two novel techniques: 1. A greedy algorithm to solve approximately optimal bit allocation across weight channels, and 2. A trainable scaling vector for non-differentiable K-means clustering. These techniques substantially improve performance and can be adapted to any given bit. Notably, in terms of model perplexity, our method narrows the gap between 3-bit quantized LLaMA models and their full precision counterparts by 16.3% on average.

cs.LG↗

arXiv2024

Breaking Event Rumor Detection via Stance-Separated Multi-Agent Debate

The rapid spread of rumors on social media platforms during breaking events severely hinders the dissemination of the truth. Previous studies reveal that the lack of annotated resources hinders the direct detection of unforeseen breaking events not covered in yesterday's news. Leveraging large language models (LLMs) for rumor detection holds significant promise. However, it is challenging for LLMs to provide comprehensive responses to complex or controversial issues due to limited diversity. In this work, we propose the Stance Separated Multi-Agent Debate (S2MAD) to address this issue. Specifically, we firstly introduce Stance Separation, categorizing comments as either supporting or opposing the original claim. Subsequently, claims are classified as subjective or objective, enabling agents to generate reasonable initial viewpoints with different prompt strategies for each type of claim. Debaters then follow specific instructions through multiple rounds of debate to reach a consensus. If a consensus is not reached, a judge agent evaluates the opinions and delivers a final verdict on the claim's veracity. Extensive experiments conducted on two real-world datasets demonstrate that our proposed model outperforms state-of-the-art methods in terms of performance and effectively improves the performance of LLMs in breaking event rumor detection.

cs.CL↗

arXiv2024

Taurus Database: How to be Fast, Available, and Frugal in the Cloud

Using cloud Database as a Service (DBaaS) offerings instead of on-premise deployments is increasingly common. Key advantages include improved availability and scalability at a lower cost than on-premise alternatives. In this paper, we describe the design of Taurus, a new multi-tenant cloud database system. Taurus separates the compute and storage layers in a similar manner to Amazon Aurora and Microsoft Socrates and provides similar benefits, such as read replica support, low network utilization, hardware sharing and scalability. However, the Taurus architecture has several unique advantages. Taurus offers novel replication and recovery algorithms providing better availability than existing approaches using the same or fewer replicas. Also, Taurus is highly optimized for performance, using no more than one network hop on critical paths and exclusively using append-only storage, delivering faster writes, reduced device wear, and constant-time snapshots. This paper describes Taurus and provides a detailed description and analysis of the storage node architecture, which has not been previously available from the published literature.

cs.DB↗

arXiv2024

Rectified Flow For Structure Based Drug Design

Deep generative models have achieved tremendous success in structure-based drug design in recent years, especially for generating 3D ligand molecules that bind to specific protein pocket. Notably, diffusion models have transformed ligand generation by providing exceptional quality and creativity. However, traditional diffusion models are restricted by their conventional learning objectives, which limit their broader applicability. In this work, we propose a new framework FlowSBDD, which is based on rectified flow model, allows us to flexibly incorporate additional loss to optimize specific target and introduce additional condition either as an extra input condition or replacing the initial Gaussian distribution. Extensive experiments on CrossDocked2020 show that our approach could achieve state-of-the-art performance on generating high-affinity molecules while maintaining proper molecular properties without specifically designing binding site, with up to -8.50 Avg. Vina Dock score and 75.0% Diversity.

cs.LG↗

arXiv2024

InsightEdit: Towards Better Instruction Following for Image Editing

In this paper, we focus on the task of instruction-based image editing. Previous works like InstructPix2Pix, InstructDiffusion, and SmartEdit have explored end-to-end editing. However, two limitations still remain: First, existing datasets suffer from low resolution, poor background consistency, and overly simplistic instructions. Second, current approaches mainly condition on the text while the rich image information is underexplored, therefore inferior in complex instruction following and maintaining background consistency. Targeting these issues, we first curated the AdvancedEdit dataset using a novel data construction pipeline, formulating a large-scale dataset with high visual quality, complex instructions, and good background consistency. Then, to further inject the rich image information, we introduce a two-stream bridging mechanism utilizing both the textual and visual features reasoned by the powerful Multimodal Large Language Models (MLLM) to guide the image editing process more precisely. Extensive results demonstrate that our approach, InsightEdit, achieves state-of-the-art performance, excelling in complex instruction following and maintaining high background consistency with the original image.

cs.CV↗

arXiv2024

A rotational ellipsoid model for solid Earth tide with high precision

Solid Earth tide represents the response of solid Earth to the lunar (solar) gravitational force. The yielding solid Earth due to the force has been thought to be a prolate ellipsoid since the time of Lord Kelvin, yet the ellipsoid's geometry such as major semi-axis's length, minor semi-axis's length, and flattening remains unresolved. Additionally, the tidal displacement of reference point is conventionally resolved through a combination of expanded potential equations and given Earth model. Here we present a geometric model in which both the ellipsoid's geometry and the tidal displacement of reference point can be resolved through a rotating ellipse with respect to the Moon (Sun). We test the geometric model using 23-year gravity data from 22 superconducting gravimeter (SG) stations and compare it with the current model recommended by the IERS (International Earth Rotation System) conventions (2010), the average Root Mean Square (RMS) deviation of the gravity change yielded by the geometric model against observation is 6.47 μGal (equivalent to 2.07 cm), while that yielded by the current model is 30.77 μGal (equivalent to 9.85 cm). The geometric model will greatly contribute to many application fields such as geodesy, geophysics, astronomy, and oceanography.

physics.geo-ph↗

arXiv2024

Investigating the Behavior and Spatiotemporal Variations of Green Line Emission in the Solar Corona

Understanding coronal structure and dynamics can be facilitated by analyzing green-line emission, which enables the investigation of diverse coronal structures such as coronal loops, streamers, coronal holes, and various eruptions in the solar atmosphere. In this study, we investigated the spatiotemporal behaviors of green-line emissions in both low and high latitudes across nine solar cycles, ranging from cycle 17 to the current cycle 25, using the Modified Homogeneous Data Set (MHDS). We employed methodologies such as cross-correlation, power spectral density (PSD), and wavelet transform techniques for this analysis. We found distinct behaviors in green line energy across various latitudinal distributions in the solar atmosphere. The trends observed at higher latitudes differ from those at lower latitudes. The emission behaviors show a close association with other solar phenomena like solar flares, sunspots, and coronal mass ejections (CMEs) throughout the solar cycles. The observed variations exhibit harmonic periods. The emission activity is significantly higher in the low latitudes, accounting for over 70 percent of the emissions, while the higher latitudes contribute less than 30 percent. The emissions exhibit asymmetric behavior between the northern and southern hemispheres, leading to a 44-year cycle of solar hemispheric dominance shifts. Various factors, such as Alfvén waves, solar magnetic fields, sunspots, differential rotation, and reconnection events, influence the observed differences in behavior between lower and higher latitudes, suggesting the existence of potential underlying phenomena contributing to deviations in properties, intensity, temporal dynamics, and spatiotemporal lifetime.

astro-ph.SR↗

arXiv2024

AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies

Diffusion-based imitation learning improves Behavioral Cloning (BC) on multi-modal decision-making, but comes at the cost of significantly slower inference due to the recursion in the diffusion process. It urges us to design efficient policy generators while keeping the ability to generate diverse actions. To address this challenge, we propose AdaFlow, an imitation learning framework based on flow-based generative modeling. AdaFlow represents the policy with state-conditioned ordinary differential equations (ODEs), which are known as probability flows. We reveal an intriguing connection between the conditional variance of their training loss and the discretization error of the ODEs. With this insight, we propose a variance-adaptive ODE solver that can adjust its step size in the inference stage, making AdaFlow an adaptive decision-maker, offering rapid inference without sacrificing diversity. Interestingly, it automatically reduces to a one-step generator when the action distribution is uni-modal. Our comprehensive empirical evaluation shows that AdaFlow achieves high performance with fast inference speed.

cs.LG↗

arXiv2024

A Multi-Server Information-Sharing Environment for Cross-Party Collaboration on A Private Cloud

Interoperability remains the key problem in multi-discipline collaboration based on building information modeling (BIM). Although various methods have been proposed to solve the technical issues of interoperability, such as data sharing and data consistency; organizational issues, including data ownership and data privacy, remain unresolved to date. These organizational issues prevent different stakeholders from sharing their data due to concerns regarding losing control of the data. This study proposes a multi-server information-sharing approach on a private cloud after analyzing the requirements for cross-party collaboration to address the aforementioned issues and prepare for massive data handling in the near future. This approach adopts a global controller to track the location, ownership and privacy of the data, which are stored in different servers that are controlled by different parties. Furthermore, data consistency conventions, parallel sub-model extraction, and sub-model integration with model verification are investigated in depth to support information sharing in a distributed environment and to maintain data consistency. Thus, with this approach, the ownership and privacy of the data can be controlled by its owner while still enabling certain required data to be shared with other parties. Application of the multi-server approach for information interoperability and cross-party collaboration is illustrated using a real construction project of an airport terminal. Validation shows that the proposed approach is feasible for maintaining the ownership and privacy of the data while supporting cross-party data sharing and collaboration at the same time, thus avoiding possible legal problems regarding data copyrights or other legal issues.

cs.CR↗

arXiv2024

Evolving to the Future: Unseen Event Adaptive Fake News Detection on Social Media

With the rapid development of social media, the wide dissemination of fake news on social media is increasingly threatening both individuals and society. One of the unique challenges for fake news detection on social media is how to detect fake news on future events. Recently, numerous fake news detection models that utilize textual information and the propagation structure of posts have been proposed. Unfortunately, most of the existing approaches can hardly handle this challenge since they rely heavily on event-specific features for prediction and cannot generalize to unseen events. To address this, we introduce \textbf{F}uture \textbf{AD}aptive \textbf{E}vent-based Fake news Detection (FADE) framework. Specifically, we train a target predictor through an adaptive augmentation strategy and graph contrastive learning to obtain higher-quality features and make more accurate overall predictions. Simultaneously, we independently train an event-only predictor to obtain biased predictions. We further mitigate event bias by subtracting the event-only predictor's output from the target predictor's output to obtain the final prediction. Encouraging results from experiments designed to emulate real-world social media conditions validate the effectiveness of our method in comparison to existing state-of-the-art approaches.

cs.SI↗

arXiv2024

Is the annualized compounded return of Medallion over 35%?

It is a challenge to estimate fund performance by compounded returns. Arguably, it is incorrect to use yearly returns directly for compounding, with reported annualized return of above 60% for Medallion for the 31 years up to 2018. We propose an estimation based on fund sizes and trading profits and obtain a compounded return of 31.8% before fees. Alternatively, we suggest using the manager's wealth as a proxy and arriving at a compounded growth rate of 25.6% for Simons for the 33 years up to 2020. We conclude that the annualized compounded return of Medallion before fees is probably under 35%. Our findings have implications for correctly estimating fund performance.

q-fin.PM↗

arXiv2024

Risk-neutral valuation of options under arithmetic Brownian motions

On April 22, 2020, the CME Group switched to Bachelier pricing for a group of oil futures options. The Bachelier model, or more generally the arithmetic Brownian motion (ABM), is not so widely used in finance, though. This paper provides the first comprehensive survey of options pricing under ABM. Using the risk-neutral valuation, we derive formulas for European options for three underlying types, namely an underlying that does not pay dividends, an underlying that pays a continuous dividend yield, and futures. Further, we derive Black-Scholes-Merton-like partial differential equations, which can in principle be utilized to price American options numerically via finite difference.

q-fin.PR↗

arXiv2024

Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Prediction

Molecular property prediction (MPP) is integral to drug discovery and material science, but often faces the challenge of data scarcity in real-world scenarios. Addressing this, few-shot molecular property prediction (FSMPP) has been developed. Unlike other few-shot tasks, FSMPP typically employs a pre-trained molecular encoder and a context-aware classifier, benefiting from molecular pre-training and molecular context information. Despite these advancements, existing methods struggle with the ineffective fine-tuning of pre-trained encoders. We attribute this issue to the imbalance between the abundance of tunable parameters and the scarcity of labeled molecules, and the lack of contextual perceptiveness in the encoders. To overcome this hurdle, we propose a parameter-efficient in-context tuning method, named Pin-Tuning. Specifically, we propose a lightweight adapter for pre-trained message passing layers (MP-Adapter) and Bayesian weight consolidation for pre-trained atom/bond embedding layers (Emb-BWC), to achieve parameter-efficient tuning while preventing over-fitting and catastrophic forgetting. Additionally, we enhance the MP-Adapters with contextual perceptiveness. This innovation allows for in-context tuning of the pre-trained encoder, thereby improving its adaptability for specific FSMPP tasks. When evaluated on public datasets, our method demonstrates superior tuning with fewer trainable parameters, improving few-shot predictive performance.

cs.LG↗