arXiv subjects

Chao Feng

Publications and source records attributed to Chao Feng.

At least 109 records · Page 6Linked to original sources

arXiv2024

NeuroBind: Towards Unified Multimodal Representations for Neural Signals

Understanding neural activity and information representation is crucial for advancing knowledge of brain function and cognition. Neural activity, measured through techniques like electrophysiology and neuroimaging, reflects various aspects of information processing. Recent advances in deep neural networks offer new approaches to analyzing these signals using pre-trained models. However, challenges arise due to discrepancies between different neural signal modalities and the limited scale of high-quality neural data. To address these challenges, we present NeuroBind, a general representation that unifies multiple brain signal types, including EEG, fMRI, calcium imaging, and spiking data. To achieve this, we align neural signals in these image-paired neural datasets to pre-trained vision-language embeddings. Neurobind is the first model that studies different neural modalities interconnectedly and is able to leverage high-resource modality models for various neuroscience tasks. We also showed that by combining information from different neural signal modalities, NeuroBind enhances downstream performance, demonstrating the effectiveness of the complementary strengths of different neural modalities. As a result, we can leverage multiple types of neural signals mapped to the same space to improve downstream tasks, and demonstrate the complementary strengths of different neural modalities. This approach holds significant potential for advancing neuroscience research, improving AI systems, and developing neuroprosthetics and brain-computer interfaces.

q-bio.NC↗

arXiv2024

DART: A Solution for Decentralized Federated Learning Model Robustness Analysis

Federated Learning (FL) has emerged as a promising approach to address privacy concerns inherent in Machine Learning (ML) practices. However, conventional FL methods, particularly those following the Centralized FL (CFL) paradigm, utilize a central server for global aggregation, which exhibits limitations such as bottleneck and single point of failure. To address these issues, the Decentralized FL (DFL) paradigm has been proposed, which removes the client-server boundary and enables all participants to engage in model training and aggregation tasks. Nevertheless, as CFL, DFL remains vulnerable to adversarial attacks, notably poisoning attacks that undermine model performance. While existing research on model robustness has predominantly focused on CFL, there is a noteworthy gap in understanding the model robustness of the DFL paradigm. In this paper, a thorough review of poisoning attacks targeting the model robustness in DFL systems, as well as their corresponding countermeasures, are presented. Additionally, a solution called DART is proposed to evaluate the robustness of DFL models, which is implemented and integrated into a DFL platform. Through extensive experiments, this paper compares the behavior of CFL and DFL under diverse poisoning attacks, pinpointing key factors affecting attack spread and effectiveness within the DFL. It also evaluates the performance of different defense mechanisms and investigates whether defense mechanisms designed for CFL are compatible with DFL. The empirical results provide insights into research challenges and suggest ways to improve the robustness of DFL models for future research.

cs.DC↗

arXiv2024

IFA: Interaction Fidelity Attention for Entire Lifelong Behaviour Sequence Modeling

The lifelong user behavior sequence provides abundant information of user preference and gains impressive improvement in the recommendation task, however increases computational consumption significantly. To meet the severe latency requirement in online service, a short sub-sequence is sampled based on similarity to the target item. Unfortunately, items not in the sub-sequence are abandoned, leading to serious information loss. In this paper, we propose a new efficient paradigm to model the full lifelong sequence, which is named as \textbf{I}nteraction \textbf{F}idelity \textbf{A}ttention (\textbf{IFA}). In IFA, we input all target items in the candidate set into the model at once, and leverage linear transformer to reduce the time complexity of the cross attention between the candidate set and the sequence without any interaction information loss. We also additionally model the relationship of all target items for optimal set generation, and design loss function for better consistency of training and inference. We demonstrate the effectiveness and efficiency of our model by off-line and online experiments in the recommender system of Kuaishou.

cs.IR↗

arXiv2024

Asset-centric Threat Modeling for AI-based Systems

Threat modeling is a popular method to securely develop systems by achieving awareness of potential areas of future damage caused by adversaries. However, threat modeling for systems relying on Artificial Intelligence is still not well explored. While conventional threat modeling methods and tools did not address AI-related threats, research on this amalgamation still lacks solutions capable of guiding and automating the process, as well as providing evidence that the methods hold up in practice. Consequently, this paper presents ThreatFinderAI, an approach and tool providing guidance and automation to model AI-related assets, threats, countermeasures, and quantify residual risks. To evaluate the practicality of the approach, participants were tasked to recreate a threat model developed by cybersecurity experts of an AI-based healthcare platform. Secondly, the approach was used to identify and discuss strategic risks in an LLM-based application through a case study. Overall, the solution's usability was well-perceived and effectively supports threat identification and risk discussion.

cs.CR↗

arXiv2024

High-repetition-rate seeded free-electron laser with direct-amplification of an external coherent laser

Various scientific and industrial researches such as spectroscopy and advanced nano-technologies have been demanding high flux and fully coherent EUV and X-ray radiation. These demands can be commendably satisfied with a MHz-level repetition-rate seeded free-electron laser (FEL). Dictated by the seed laser system, seeded FELs have faced obstacles for the realization of MHz repetition rate. Reducing the required peak power of an external coherent seed laser can effectively increase the repetition rate of a seeded FEL. This paper presents a novel technique that employs a long modulator as a carrier for laser amplification and electron modulation, which is quite different from nominal seeded FELs. Applications of the proposed technique into high-gain harmonic generation (HGHG) and echo-enabled harmonic generation (EEHG) are investigated. Simulation results demonstrate that seed laser power is reduced by about three orders of magnitude and the FEL radiation possesses consistently high coherence and power stability with respect to the nominal HGHG or EEHG. The proposed technique paves the way for the realization of fully coherent EUV and soft X-ray FELs with a repetition rate of MHz and an average power of about 100 W.

physics.acc-ph↗

arXiv2024

Fedstellar: A Platform for Decentralized Federated Learning

In 2016, Google proposed Federated Learning (FL) as a novel paradigm to train Machine Learning (ML) models across the participants of a federation while preserving data privacy. Since its birth, Centralized FL (CFL) has been the most used approach, where a central entity aggregates participants' models to create a global one. However, CFL presents limitations such as communication bottlenecks, single point of failure, and reliance on a central server. Decentralized Federated Learning (DFL) addresses these issues by enabling decentralized model aggregation and minimizing dependency on a central entity. Despite these advances, current platforms training DFL models struggle with key issues such as managing heterogeneous federation network topologies. To overcome these challenges, this paper presents Fedstellar, a platform extended from p2pfl library and designed to train FL models in a decentralized, semi-decentralized, and centralized fashion across diverse federations of physical or virtualized devices. The Fedstellar implementation encompasses a web application with an interactive graphical interface, a controller for deploying federations of nodes using physical or virtual devices, and a core deployed on each device which provides the logic needed to train, aggregate, and communicate in the network. The effectiveness of the platform has been demonstrated in two scenarios: a physical deployment involving single-board devices such as Raspberry Pis for detecting cyberattacks, and a virtualized deployment comparing various FL approaches in a controlled environment using MNIST and CIFAR-10 datasets. In both scenarios, Fedstellar demonstrated consistent performance and adaptability, achieving F1 scores of 91%, 98%, and 91.2% using DFL for detecting cyberattacks and classifying MNIST and CIFAR-10, respectively, reducing training time by 32% compared to centralized approaches.

cs.LG↗

arXiv2024

Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning

Despite vision-language models' (VLMs) remarkable capabilities as versatile visual assistants, two substantial challenges persist within the existing VLM frameworks: (1) lacking task diversity in pretraining and visual instruction tuning, and (2) annotation error and bias in GPT-4 synthesized instruction tuning data. Both challenges lead to issues such as poor generalizability, hallucination, and catastrophic forgetting. To address these challenges, we construct Vision-Flan, the most diverse publicly available visual instruction tuning dataset to date, comprising 187 diverse tasks and 1,664,261 instances sourced from academic datasets, and each task is accompanied by an expert-written instruction. In addition, we propose a two-stage instruction tuning framework, in which VLMs are firstly finetuned on Vision-Flan and further tuned on GPT-4 synthesized data. We find this two-stage tuning framework significantly outperforms the traditional single-stage visual instruction tuning framework and achieves the state-of-the-art performance across a wide range of multi-modal evaluation benchmarks. Finally, we conduct in-depth analyses to understand visual instruction tuning and our findings reveal that: (1) GPT-4 synthesized data does not substantially enhance VLMs' capabilities but rather modulates the model's responses to human-preferred formats; (2) A minimal quantity (e.g., 1,000) of GPT-4 synthesized data can effectively align VLM responses with human-preference; (3) Visual instruction tuning mainly helps large-language models (LLMs) to understand visual features.

cs.CL↗

arXiv2024

Voyager: MTD-Based Aggregation Protocol for Mitigating Poisoning Attacks on DFL

The growing concern over malicious attacks targeting the robustness of both Centralized and Decentralized Federated Learning (FL) necessitates novel defensive strategies. In contrast to the centralized approach, Decentralized FL (DFL) has the advantage of utilizing network topology and local dataset information, enabling the exploration of Moving Target Defense (MTD) based approaches. This work presents a theoretical analysis of the influence of network topology on the robustness of DFL models. Drawing inspiration from these findings, a three-stage MTD-based aggregation protocol, called Voyager, is proposed to improve the robustness of DFL models against poisoning attacks by manipulating network topology connectivity. Voyager has three main components: an anomaly detector, a network topology explorer, and a connection deployer. When an abnormal model is detected in the network, the topology explorer responds strategically by forming connections with more trustworthy participants to secure the model. Experimental evaluations show that Voyager effectively mitigates various poisoning attacks without imposing significant resource and computational burdens on participants. These findings highlight the proposed reactive MTD as a potent defense mechanism in the context of DFL.

cs.CR↗

arXiv2024

GuardFS: a File System for Integrated Detection and Mitigation of Linux-based Ransomware

Although ransomware has received broad attention in media and research, this evolving threat vector still poses a systematic threat. Related literature has explored their detection using various approaches leveraging Machine and Deep Learning. While these approaches are effective in detecting malware, they do not answer how to use this intelligence to protect against threats, raising concerns about their applicability in a hostile environment. Solutions that focus on mitigation rarely explore how to prevent and not just alert or halt its execution, especially when considering Linux-based samples. This paper presents GuardFS, a file system-based approach to investigate the integration of detection and mitigation of ransomware. Using a bespoke overlay file system, data is extracted before files are accessed. Models trained on this data are used by three novel defense configurations that obfuscate, delay, or track access to the file system. The experiments on GuardFS test the configurations in a reactive setting. The results demonstrate that although data loss cannot be completely prevented, it can be significantly reduced. Usability and performance analysis demonstrate that the defense effectiveness of the configurations relates to their impact on resource consumption and usability.

cs.CR↗

arXiv2024

Binding Touch to Everything: Learning Unified Multimodal Tactile Representations

The ability to associate touch with other modalities has huge implications for humans and computational systems. However, multimodal learning with touch remains challenging due to the expensive data collection process and non-standardized sensor outputs. We introduce UniTouch, a unified tactile model for vision-based touch sensors connected to multiple modalities, including vision, language, and sound. We achieve this by aligning our UniTouch embeddings to pretrained image embeddings already associated with a variety of other modalities. We further propose learnable sensor-specific tokens, allowing the model to learn from a set of heterogeneous tactile sensors, all at the same time. UniTouch is capable of conducting various touch sensing tasks in the zero-shot setting, from robot grasping prediction to touch image question answering. To the best of our knowledge, UniTouch is the first to demonstrate such capabilities. Project page: https://cfeng16.github.io/UniTouch/

cs.CV↗

arXiv2023

Opportunities for Gas-Phase Science at Short-Wavelength Free-Electron Lasers with Undulator-Based Polarization Control

Free-electron lasers (FELs) are the world's most brilliant light sources with rapidly evolving technological capabilities in terms of ultrabright and ultrashort pulses over a large range of accessible photon energies. Their revolutionary and innovative developments have opened new fields of science regarding nonlinear light-matter interaction, the investigation of ultrafast processes from specific observer sites, and approaches to imaging matter with atomic resolution. A core aspect of FEL science is the study of isolated and prototypical systems in the gas phase with the possibility of addressing well-defined electronic transitions or particular atomic sites in molecules. Notably for polarization-controlled short-wavelength FELs, the gas phase offers new avenues for investigations of nonlinear and ultrafast phenomena in spin orientated systems, for decoding the function of the chiral building blocks of life as well as steering reactions and particle emission dynamics in otherwise inaccessible ways. This roadmap comprises descriptions of technological capabilities of facilities worldwide, innovative diagnostics and instrumentation, as well as recent scientific highlights, novel methodology and mathematical modeling. The experimental and theoretical landscape of using polarization controllable FELs for dichroic light-matter interaction in the gas phase will be discussed and comprehensively outlined to stimulate and strengthen global collaborative efforts of all disciplines.

physics.atom-ph↗

arXiv2023

Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs

Large language models (LLMs), such as ChatGPT and GPT-4, are versatile and can solve different tasks due to their emergent ability and generalizability. However, LLMs sometimes lack domain-specific knowledge to perform tasks, which would also cause hallucination during inference. In some previous works, additional modules like graph neural networks (GNNs) are trained on retrieved knowledge from external knowledge bases, aiming to mitigate the problem of lacking domain-specific knowledge. However, incorporating additional modules: 1) would need retraining additional modules when encountering novel domains; 2) would become a bottleneck since LLMs' strong abilities are not fully utilized for retrieval. In this paper, we propose a paradigm, termed Knowledge Solver (KSL), to teach LLMs to search for essential knowledge from external knowledge bases by harnessing their own strong generalizability. Specifically, we design a simple yet effective prompt to transform retrieval into a multi-hop decision sequence, which empowers LLMs with searching knowledge ability in zero-shot manner. Additionally, KSL is able to provide complete retrieval paths and therefore increase explainability of LLMs' reasoning processes. We conduct experiments on three datasets: CommonsenseQA, OpenbookQA, and MedQA-USMLE, and found that our approach improves LLM baseline performance by a relatively large margin.

cs.CL↗

arXiv2023

RCVaR: an Economic Approach to Estimate Cyberattacks Costs using Data from Industry Reports

Digitization increases business opportunities and the risk of companies being victims of devastating cyberattacks. Therefore, managing risk exposure and cybersecurity strategies is essential for digitized companies that want to survive in competitive markets. However, understanding company-specific risks and quantifying their associated costs is not trivial. Current approaches fail to provide individualized and quantitative monetary estimations of cybersecurity impacts. Due to limited resources and technical expertise, SMEs and even large companies are affected and struggle to quantify their cyberattack exposure. Therefore, novel approaches must be placed to support the understanding of the financial loss due to cyberattacks. This article introduces the Real Cyber Value at Risk (RCVaR), an economical approach for estimating cybersecurity costs using real-world information from public cybersecurity reports. RCVaR identifies the most significant cyber risk factors from various sources and combines their quantitative results to estimate specific cyberattacks costs for companies. Furthermore, RCVaR extends current methods to achieve cost and risk estimations based on historical real-world data instead of only probability-based simulations. The evaluation of the approach on unseen data shows the accuracy and efficiency of the RCVaR in predicting and managing cyber risks. Thus, it shows that the RCVaR is a valuable addition to cybersecurity planning and risk management processes.

cs.CR↗

arXiv2023

Self-referenced Spectral Interferometry for Single-shot Characterization of Ultrashort Free-electron Laser Pulses

Attosecond x-ray pulse with known spectro-temporal information is an essential tool for the investigation of ultrafast electron dynamics in quantum systems. Ultrafast free-electron lasers (FELs) have the unique advantage on unprecedented high-intensity at x-ray wavelengths. However, no suitable method has been established so far for the spectro-temporal characterization of these ultrashort x-ray pulses. In this letter, a simple method has been proposed based on self-referenced spectral interferometry for reconstructing ultrashort FEL pulses both in temporal and spectral domains. We have demonstrated that the proposed method is reliable to reconstruct the temporal profile and phase of attosecond x-ray FEL pulses with an error of only a few-percent-level. Moreover, the first proof-of-principle experiment has been performed to achieve single-shot spectro-temporal characterization of ultrashort pulses from a high-gain FEL. The precision of the proposed method will be enhanced with the decrease of the pulse duration, paving a new way for complete attosecond pulse characterization at x-ray FELs.

physics.acc-ph↗

arXiv2023

Self-Supervised Video Forensics by Audio-Visual Anomaly Detection

Manipulated videos often contain subtle inconsistencies between their visual and audio signals. We propose a video forensics method, based on anomaly detection, that can identify these inconsistencies, and that can be trained solely using real, unlabeled data. We train an autoregressive model to generate sequences of audio-visual features, using feature sets that capture the temporal synchronization between video frames and sound. At test time, we then flag videos that the model assigns low probability. Despite being trained entirely on real videos, our model obtains strong performance on the task of detecting manipulated speech videos. Project site: https://cfeng16.github.io/audio-visual-forensics

cs.CV↗

arXiv2023

Near-Field Modelling and Performance Analysis for Extremely Large-Scale IRS Communications

Intelligent reflecting surface (IRS) is an emerging technology for wireless communications, thanks to its powerful capability to engineer the radio environment. However, in practice, this benefit is attainable only when the passive IRS is of sufficiently large size, for which the conventional uniform plane wave (UPW)-based far-field model may become invalid. In this paper, we pursue a near-field modelling and performance analysis for wireless communications with extremely large-scale IRS (XL-IRS). By taking into account the directional gain pattern of IRS's reflecting elements and the variations in signal amplitude across them, we derive both the lower- and upper-bounds of the resulting signal-to-noise ratio (SNR) for the generic uniform planar array (UPA)-based XL-IRS. Our results reveal that, instead of scaling quadratically and unboundedly with the number of reflecting elements M as in the conventional UPW-based model, the SNR under the new non-uniform spherical wave (NUSW)-based model increases with $M$ with a diminishing return and eventually converges to a certain limit. To gain more insights, we further study the special case of uniform linear array (ULA)-based XL-IRS, for which a closed-form SNR expression in terms of the IRS size and locations of the base station (BS) and the user is derived. Our result shows that the SNR is mainly determined by the two geometric angles formed by the BS/user locations with the IRS, as well as the dimension of the IRS. Numerical results validate our analysis and demonstrate the necessity of proper near-field modelling for wireless communications aided by XL-IRS.

cs.IT↗

arXiv2022

Multi-objective Evolutionary Algorithms are Generally Good: Maximizing Monotone Submodular Functions over Sequences

Evolutionary algorithms (EAs) are general-purpose optimization algorithms, inspired by natural evolution. Recent theoretical studies have shown that EAs can achieve good approximation guarantees for solving the problem classes of submodular optimization, which have a wide range of applications, such as maximum coverage, sparse regression, influence maximization, document summarization and sensor placement, just to name a few. Though they have provided some theoretical explanation for the general-purpose nature of EAs, the considered submodular objective functions are defined only over sets or multisets. To complement this line of research, this paper studies the problem class of maximizing monotone submodular functions over sequences, where the objective function depends on the order of items. We prove that for each kind of previously studied monotone submodular objective functions over sequences, i.e., prefix monotone submodular functions, weakly monotone and strongly submodular functions, and DAG monotone submodular functions, a simple multi-objective EA, i.e., GSEMO, can always reach or improve the best known approximation guarantee after running polynomial time in expectation. Note that these best-known approximation guarantees can be obtained only by different greedy-style algorithms before. Empirical studies on various applications, e.g., accomplishing tasks, maximizing information gain, search-and-tracking and recommender systems, show the excellent performance of the GSEMO.

cs.NE↗

arXiv2022

Dynamics of microswimmers near a liquid-liquid interface with viscosity difference

Transport of material across liquid interfaces is ubiquitous for living cells and is also a crucial step in drug delivery and in many industrial processes. The fluids that are present on either side of the interfaces will usually have different viscosities. We present a physical model for the dynamics of microswimmers near a soft and penetrable interface that we solve using computer simulations of Navier-Stokes flows. The literature contains studies of similar isoviscous fluid systems, where the two fluids have the same viscosity. Here we extend this to the more general case where they have different viscosities. We investigate the effect of the fluid viscosity ratio on the movement patterns of microswimmers. We find that swimmers systematically reorientate towards the region containing the lower viscosity fluid. Ultimately this is expected to drive the swimmers to behave as if they are more inclined to swim in low viscosity fluids. Furthermore, in addition to the types of swimming already reported in the isoviscous system, i.e. bouncing, sliding and penetrating, we observed a hovering motion, in which strong pullers swim parallel to the interface with a certain distance, which is consistent with the dynamics of such swimmers near the solid wall.

cond-mat.soft↗