arXiv subjects

Yong Li

Publications and source records attributed to Yong Li.

At least 451 records · Page 25Linked to original sources

arXiv2024

Towards Personalized Federated Learning via Comprehensive Knowledge Distillation

Federated learning is a distributed machine learning paradigm designed to protect data privacy. However, data heterogeneity across various clients results in catastrophic forgetting, where the model rapidly forgets previous knowledge while acquiring new knowledge. To address this challenge, personalized federated learning has emerged to customize a personalized model for each client. However, the inherent limitation of this mechanism is its excessive focus on personalization, potentially hindering the generalization of those models. In this paper, we present a novel personalized federated learning method that uses global and historical models as teachers and the local model as the student to facilitate comprehensive knowledge distillation. The historical model represents the local model from the last round of client training, containing historical personalized knowledge, while the global model represents the aggregated model from the last round of server aggregation, containing global generalized knowledge. By applying knowledge distillation, we effectively transfer global generalized knowledge and historical personalized knowledge to the local model, thus mitigating catastrophic forgetting and enhancing the general performance of personalized models. Extensive experimental results demonstrate the significant advantages of our method.

cs.LG↗

arXiv2024

A Survey of Generative AI for Intelligent Transportation Systems: Road Transportation Perspective

Intelligent transportation systems are vital for modern traffic management and optimization, greatly improving traffic efficiency and safety. With the rapid development of generative artificial intelligence (Generative AI) technologies in areas like image generation and natural language processing, generative AI has also played a crucial role in addressing key issues in intelligent transportation systems (ITS), such as data sparsity, difficulty in observing abnormal scenarios, and in modeling data uncertainty. In this review, we systematically investigate the relevant literature on generative AI techniques in addressing key issues in different types of tasks in ITS tailored specifically for road transportation. First, we introduce the principles of different generative AI techniques. Then, we classify tasks in ITS into four types: traffic perception, traffic prediction, traffic simulation, and traffic decision-making. We systematically illustrate how generative AI techniques addresses key issues in these four different types of tasks. Finally, we summarize the challenges faced in applying generative AI to intelligent transportation systems, and discuss future research directions based on different application scenarios.

cs.AI↗

arXiv2024

Enhancing ID-based Recommendation with Large Language Models

Large Language Models (LLMs) have recently garnered significant attention in various domains, including recommendation systems. Recent research leverages the capabilities of LLMs to improve the performance and user modeling aspects of recommender systems. These studies primarily focus on utilizing LLMs to interpret textual data in recommendation tasks. However, it's worth noting that in ID-based recommendations, textual data is absent, and only ID data is available. The untapped potential of LLMs for ID data within the ID-based recommendation paradigm remains relatively unexplored. To this end, we introduce a pioneering approach called "LLM for ID-based Recommendation" (LLM4IDRec). This innovative approach integrates the capabilities of LLMs while exclusively relying on ID data, thus diverging from the previous reliance on textual data. The basic idea of LLM4IDRec is that by employing LLM to augment ID data, if augmented ID data can improve recommendation performance, it demonstrates the ability of LLM to interpret ID data effectively, exploring an innovative way for the integration of LLM in ID-based recommendation. We evaluate the effectiveness of our LLM4IDRec approach using three widely-used datasets. Our results demonstrate a notable improvement in recommendation performance, with our approach consistently outperforming existing methods in ID-based recommendation by solely augmenting input data.

cs.IR↗

arXiv2024

Seven-octave ultrabroadband metamaterial absorbers via Q-weighted mode density modulation

Absorption is a crucial parameter in shaping wave propagation dynamics, yet achieving ultra-broadband absorption remains highly challenging, particularly in balancing low-frequency and broad bandwidth. Here, we present a metamaterial absorber (MMA) capable of achieving simultaneous spectral coverage across a seven-octave range of near-perfect absorption from 100 Hz to 12,800 Hz by engineering the quality-factor-weighted (Q-weighted) mode density. The Q-weighted mode density considers mode density, resonant frequencies, radiative loss, and intrinsic loss of multiple resonant modes, providing a comprehensive approach to govern broadband absorption properties. By optimizing the number of resonant modes and managing intrinsic losses, our approach achieves an intensive Q-weighted mode density across an ultra-wide bandwidth, enabling ultra-broadband absorption with high efficiency. These findings significantly advance the bandwidth capabilities of state-of-the-art MMAs and pave the way for the development of ultra-broadband metamaterial devices across various wave systems.

physics.app-ph↗

arXiv2024

EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces

Effectively learning the temporal dynamics in electroencephalogram (EEG) signals is challenging yet essential for decoding brain activities using brain-computer interfaces (BCIs). Although Transformers are popular for their long-term sequential learning ability in the BCI field, most methods combining Transformers with convolutional neural networks (CNNs) fail to capture the coarse-to-fine temporal dynamics of EEG signals. To overcome this limitation, we introduce EEG-Deformer, which incorporates two main novel components into a CNN-Transformer: (1) a Hierarchical Coarse-to-Fine Transformer (HCT) block that integrates a Fine-grained Temporal Learning (FTL) branch into Transformers, effectively discerning coarse-to-fine temporal patterns; and (2) a Dense Information Purification (DIP) module, which utilizes multi-level, purified temporal information to enhance decoding accuracy. Comprehensive experiments on three representative cognitive tasks-cognitive attention, driving fatigue, and mental workload detection-consistently confirm the generalizability of our proposed EEG-Deformer, demonstrating that it either outperforms or performs comparably to existing state-of-the-art methods. Visualization results show that EEG-Deformer learns from neurophysiologically meaningful brain regions for the corresponding cognitive tasks. The source code can be found at https://github.com/yi-ding-cs/EEG-Deformer.

eess.SP↗

arXiv2024

Enantiomer-Specific Pumping of Chiral Molecules

Enantiomer-specific state transfer (ESST), which involves transferring enantiomers with different handedness of a chiral molecule into different-energy internal states, is a challenging yet significant task. Previous ESST methods are based on dynamic processes and thus require the preparation of initial states and precise control of microwave operation times. We propose a novel ESST approach, called enantiomer-specific pumping (ESP), which is based on a {\it dissipative} process, and thereby eliminates the need for these two technical requirements. This approach utilizes a special microwave-induced dark state that appears exclusively for the enantiomer with a specific handedness. Specifically, in ESP, the enantiomer lacking the dark state can be pumped out of the subspace of relevant internal states, while the enantiomer with the dark state maintains a finite probability within this subspace, offering high efficiency in ESST. Notably, ESP facilitates enantiodetection without the need for enantiopure samples as reference.

quant-ph↗

arXiv2024

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

Recent technology development brings the boom of numerous new Demand-Driven Services (DDS) into urban lives, including ridesharing, on-demand delivery, express systems and warehousing. In DDS, a service loop is an elemental structure, including its service worker, the service providers and corresponding service targets. The service workers should transport either people or parcels from the providers to the target locations. Various planning tasks within DDS can thus be classified into two individual stages: 1) Dispatching, which is to form service loops from demand/supply distributions, and 2) Routing, which is to decide specific serving orders within the constructed loops. Generating high-quality strategies in both stages is important to develop DDS but faces several challenges. Meanwhile, deep reinforcement learning (DRL) has been developed rapidly in recent years. It is a powerful tool to solve these problems since DRL can learn a parametric model without relying on too many problem-based assumptions and optimize long-term effects by learning sequential decisions. In this survey, we first define DDS, then highlight common applications and important decision/control problems within. For each problem, we comprehensively introduce the existing DRL solutions. We also introduce open simulation environments for development and evaluation of DDS applications. Finally, we analyze remaining challenges and discuss further research opportunities in DRL solutions for DDS.

cs.LG↗

arXiv2024

PSY: Posterior Sampling Based Privacy Enhancer in Large Language Models

Privacy vulnerabilities in LLMs, such as leakage from memorization, have been constantly identified, and various mitigation proposals have been proposed. LoRA is usually used in fine-tuning LLMs and a good entry point to insert privacy-enhancing modules. In this ongoing research, we introduce PSY, a Posterior Sampling based PrivacY enhancer that can be used in LoRA. We propose a simple yet effective realization of PSY using posterior sampling, which effectively prevents privacy leakage from intermediate information and, in turn, preserves the privacy of data owners. We evaluate LoRA extended with PSY against state-of-the-art membership inference and data extraction attacks. The experiments are executed on three different LLM architectures fine-tuned on three datasets with LoRA. In contrast to the commonly used differential privacy method, we find that our proposed modification consistently reduces the attack success rate. Meanwhile, our method has almost no negative impact on model fine-tuning or final performance. Most importantly, PSY reveals a promising path toward privacy enhancement with latent space extensions.

cs.CR↗

arXiv2024

Physics-driven AI for Channel Estimation in Cellular Network

In cellular mobile networks, wireless channel quality (CQ) is a crucial factor in determining communication performance and user's network experience. Accurately predicting CQ based on real environmental characteristics, specific base station configurations and user trajectories can help network operators optimize base station deployment, improving coverage and capacity. The Received Signal Reference Power (RSRP) and Signal-to-Interference-plus-Noise Ratio (SINR) of user equipment (UE) are key indicators of CQ in wireless communication. However, existing researches have limitations in terms of generation accuracy. Regression methods such as statistical inference and random forests fail to effectively capture the unique characteristics of wireless environments; theoretical derivations relying on specific communication protocols lack generalization capability; data-driven machine learning (ML) methods like Long Short-Term Memory (LSTM) Network often suffer from a lack of interpretability. To overcome these limitations, we propose physics-informed diffusion models, which accurately generate RSRP and SINR at UE based on the wireless environment, base station configurations, and user trajectories. The model adopts a modular and end-to-end design, employing a teacher-student framework to achieve knowledge distillation. This method integrates expert knowledge into the training of diffusion models, enhancing both the interpretability and accuracy, while also facilitating faster convergence of the model parameters. Furthermore, it allows for self-adaptation in various scenarios through few-shot learning. This approach provides valuable guidance for optimizing base station deployment, predicting user network experience, and building real-world simulators.

cs.NI↗

arXiv2024

UrbanWorld: An Urban World Model for 3D City Generation

Cities, as the essential environment of human life, encompass diverse physical elements such as buildings, roads and vegetation, which continuously interact with dynamic entities like people and vehicles. Crafting realistic, interactive 3D urban environments is essential for nurturing AGI systems and constructing AI agents capable of perceiving, decision-making, and acting like humans in real-world environments. However, creating high-fidelity 3D urban environments usually entails extensive manual labor from designers, involving intricate detailing and representation of complex urban elements. Therefore, accomplishing this automatically remains a longstanding challenge. Toward this problem, we propose UrbanWorld, the first generative urban world model that can automatically create a customized, realistic and interactive 3D urban world with flexible control conditions. UrbanWorld incorporates four key stages in the generation pipeline: flexible 3D layout generation from OSM data or urban layout with semantic and height maps, urban scene design with Urban MLLM, controllable urban asset rendering via progressive 3D diffusion, and MLLM-assisted scene refinement. We conduct extensive quantitative analysis on five visual metrics, demonstrating that UrbanWorld achieves SOTA generation realism. Next, we provide qualitative results about the controllable generation capabilities of UrbanWorld using both textual and image-based prompts. Lastly, we verify the interactive nature of these environments by showcasing the agent perception and navigation within the created environments. We contribute UrbanWorld as an open-source tool available at https://github.com/Urban-World/UrbanWorld.

cs.CV↗

arXiv2024

Shorter Is Different: Characterizing the Dynamics of Short-Form Video Platforms

The emerging short-form video platforms have been growing tremendously and become one of the leading social media recently. Although the expanded popularity of these platforms has attracted increasing research attention, there has been a lack of understanding of whether and how they deviate from traditional long-form video-sharing platforms such as YouTube and Bilibili. To address this, we conduct a large-scale data-driven analysis of Kuaishou, one of the largest short-form video platforms in China. Based on 248 million videos uploaded to the platform across all categories, we identify their notable differences from long-form video platforms through a comparison study with Bilibili, a leading long-form video platform in China. We find that videos are shortened by multiples on Kuaishou, with distinctive categorical distributions over-represented by life-related rather than interest-based videos. Users interact with videos less per view, but top videos can even more effectively acquire users' collective attention. More importantly, ordinary content creators have higher probabilities of producing hit videos. Our results shed light on the uniqueness of short-form video platforms and pave the way for future research and design for better short-form video ecology.

cs.MM↗

arXiv2024

Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructions

This paper considers a scenario in city navigation: an AI agent is provided with language descriptions of the goal location with respect to some well-known landmarks; By only observing the scene around, including recognizing landmarks and road network connections, the agent has to make decisions to navigate to the goal location without instructions. This problem is very challenging, because it requires agent to establish self-position and acquire spatial representation of complex urban environment, where landmarks are often invisible. In the absence of navigation instructions, such abilities are vital for the agent to make high-quality decisions in long-range city navigation. With the emergent reasoning ability of large language models (LLMs), a tempting baseline is to prompt LLMs to "react" on each observation and make decisions accordingly. However, this baseline has very poor performance that the agent often repeatedly visits same locations and make short-sighted, inconsistent decisions. To address these issues, this paper introduces a novel agentic workflow featured by its abilities to perceive, reflect and plan. Specifically, we find LLaVA-7B can be fine-tuned to perceive the direction and distance of landmarks with sufficient accuracy for city navigation. Moreover, reflection is achieved through a memory mechanism, where past experiences are stored and can be retrieved with current perception for effective decision argumentation. Planning uses reflection results to produce long-term plans, which can avoid short-sighted decisions in long-range navigation. We show the designed workflow significantly improves navigation ability of the LLM agent compared with the state-of-the-art baselines.

cs.AI↗

arXiv2024

EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment

Embodied artificial intelligence emphasizes the role of an agent's body in generating human-like behaviors. The recent efforts on EmbodiedAI pay a lot of attention to building up machine learning models to possess perceiving, planning, and acting abilities, thereby enabling real-time interaction with the world. However, most works focus on bounded indoor environments, such as navigation in a room or manipulating a device, with limited exploration of embodying the agents in open-world scenarios. That is, embodied intelligence in the open and outdoor environment is less explored, for which one potential reason is the lack of high-quality simulators, benchmarks, and datasets. To address it, in this paper, we construct a benchmark platform for embodied intelligence evaluation in real-world city environments. Specifically, we first construct a highly realistic 3D simulation environment based on the real buildings, roads, and other elements in a real city. In this environment, we combine historically collected data and simulation algorithms to conduct simulations of pedestrian and vehicle flows with high fidelity. Further, we designed a set of evaluation tasks covering different EmbodiedAI abilities. Moreover, we provide a complete set of input and output interfaces for access, enabling embodied agents to easily take task requirements and current environmental observations as input and then make decisions and obtain performance evaluations. On the one hand, it expands the capability of existing embodied intelligence to higher levels. On the other hand, it has a higher practical value in the real world and can support more potential applications for artificial general intelligence. Based on this platform, we evaluate some popular large language models for embodied intelligence capabilities of different dimensions and difficulties.

cs.AI↗

arXiv2024

DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization

Large language models achieve exceptional performance on various downstream tasks through supervised fine-tuning. However, the diversity of downstream tasks and practical requirements makes deploying multiple full-parameter fine-tuned models challenging. Current methods that compress the delta weight struggle to achieve ultra-high compression, failing to minimize the deployment overhead. To address the above issue, we propose a novel distribution-driven delta compression framework DeltaDQ, which utilizes Group-wise Dropout and Separate Quantization to achieve ultra-high compression for the delta weight. We have observed that the matrix-computed intermediate results for the delta weight exhibit extremely small variance and min-max range characteristics, referred to as Balanced Intermediate Results. Exploiting this phenomenon, we introduce Group-wise Dropout to perform dropout on the delta weight using an optimal group size. Furthermore, using Separate Quantization, sparse weights are quantized and decomposed to achieve a lower bit. Experimental results show that DeltaDQ achieves 16x compression with improved accuracy compared to baselines for WizardMath and WizardCoder models across different parameter scales. Moreover, DeltaDQ demonstrates the ability for ultra-high compression ratio, achieving 128x compression for the WizardMath-7B model and 512x compression for the WizardMath-70B model.

cs.LG↗

arXiv2024

OpenCity: A Scalable Platform to Simulate Urban Activities with Massive LLM Agents

Agent-based models (ABMs) have long been employed to explore how individual behaviors aggregate into complex societal phenomena in urban space. Unlike black-box predictive models, ABMs excel at explaining the micro-macro linkages that drive such emergent behaviors. The recent rise of Large Language Models (LLMs) has led to the development of LLM agents capable of simulating urban activities with unprecedented realism. However, the extreme high computational cost of LLMs presents significant challenges for scaling up the simulations of LLM agents. To address this problem, we propose OpenCity, a scalable simulation platform optimized for both system and prompt efficiencies. Specifically, we propose a LLM request scheduler to reduce communication overhead by parallelizing requests through IO multiplexing. Besides, we deisgn a "group-and-distill" prompt optimization strategy minimizes redundancy by clustering agents with similar static attributes. Through experiments on six global cities, OpenCity achieves a 600-fold acceleration in simulation time per agent, a 70% reduction in LLM requests, and a 50% reduction in token usage. These improvements enable the simulation of 10,000 agents' daily activities in 1 hour on commodity hardware. Besides, the substantial speedup of OpenCity allows us to establish a urban simulation benchmark for LLM agents for the first time, comparing simulated urban activities with real-world data in 6 major cities around the globe. We believe our OpenCity platform provides a critical infrastructure to harness the power of LLMs for interdisciplinary studies in urban space, fostering the collective efforts of broader research communities. Code repo is available at https://anonymous.4open.science/r/Anonymous-OpenCity-42BD.

cs.MA↗

arXiv2024

HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction

Citation networks are critical in modern science, and predicting which previous papers (candidates) will a new paper (query) cite is a critical problem. However, the roles of a paper's citations vary significantly, ranging from foundational knowledge basis to superficial contexts. Distinguishing these roles requires a deeper understanding of the logical relationships among papers, beyond simple edges in citation networks. The emergence of LLMs with textual reasoning capabilities offers new possibilities for discerning these relationships, but there are two major challenges. First, in practice, a new paper may select its citations from gigantic existing papers, where the texts exceed the context length of LLMs. Second, logical relationships between papers are implicit, and directly prompting an LLM to predict citations may result in surface-level textual similarities rather than the deeper logical reasoning. In this paper, we introduce the novel concept of core citation, which identifies the critical references that go beyond superficial mentions. Thereby, we elevate the citation prediction task from a simple binary classification to distinguishing core citations from both superficial citations and non-citations. To address this, we propose $\textbf{HLM-Cite}$, a $\textbf{H}$ybrid $\textbf{L}$anguage $\textbf{M}$odel workflow for citation prediction, which combines embedding and generative LMs. We design a curriculum finetune procedure to adapt a pretrained text embedding model to coarsely retrieve high-likelihood core citations from vast candidates and then design an LLM agentic workflow to rank the retrieved papers through one-shot reasoning, revealing the implicit relationships among papers. With the pipeline, we can scale the candidate sets to 100K papers. We evaluate HLM-Cite across 19 scientific fields, demonstrating a 17.6% performance improvement comparing SOTA methods.

cs.DL↗

arXiv2024

Continuous-wave amplitude control via the interference phenomenon in acoustic structures

We propose a strategy to continuously tune the amplitude of acoustic waves based on the interference among two mode-conversion paths in passive acoustic structures. The interference phenomenon is attributed to two conjugate acoustic geometric phases obtained with two mode-conversion processes in hybrid-type geometric-phase meta-atom (HGPM) pair. Notably, 100% modulation depth of the wave amplitude is achievable by simply varying the local orientation angle of meta-atom. We utilize the acoustic structure made of two cylindrical resonators to construct deep-subwavelength secondary source with designated initial phase delay, and HGPM supporting desired mode-conversion functionality is accordingly fabricated with four secondary sources. Both theory and experiment consistently verify the continuous amplitude modulation function of HGPM pair, which showcases a general scheme for reconfigurable amplitude-type acoustic meta-devices, i.e., those that require grayscale amplitude modulation for acoustic field engineering.

physics.app-ph↗

arXiv2024

An Interdisciplinary Survey on Origin-destination Flows Modeling: Theory and Techniques

Origin-destination (OD) flow modeling is an extensively researched subject across multiple disciplines, such as the investigation of travel demand in transportation and spatial interaction modeling in geography. However, researchers from different fields tend to employ their own unique research paradigms and lack interdisciplinary communication, preventing the cross-fertilization of knowledge and the development of novel solutions to challenges. This article presents a systematic interdisciplinary survey that comprehensively and holistically scrutinizes OD flows from utilizing fundamental theory to studying the mechanism of population mobility and solving practical problems with engineering techniques, such as computational models. Specifically, regional economics, urban geography, and sociophysics are adept at employing theoretical research methods to explore the underlying mechanisms of OD flows. They have developed three influential theoretical models: the gravity model, the intervening opportunities model, and the radiation model. These models specifically focus on examining the fundamental influences of distance, opportunities, and population on OD flows, respectively. In the meantime, fields such as transportation, urban planning, and computer science primarily focus on addressing four practical problems: OD prediction, OD construction, OD estimation, and OD forecasting. Advanced computational models, such as deep learning models, have gradually been introduced to address these problems more effectively. Finally, based on the existing research, this survey summarizes current challenges and outlines future directions for this topic. Through this survey, we aim to break down the barriers between disciplines in OD flow-related research, fostering interdisciplinary perspectives and modes of thinking.

cs.OH↗