arXiv subjects

Chen Sun

Publications and source records attributed to Chen Sun.

At least 109 records · Page 6Linked to original sources

arXiv2025

Integrated Sensing and Communication for 6G Holographic Digital Twins

With the advent of 6G networks, offering ultra-high bandwidth and ultra-low latency, coupled with the enhancement of terminal device resolutions, holographic communication is gradually becoming a reality. Holographic digital twin (HDT) is considered one of key applications of holographic communication, capable of creating virtual replicas for real-time mapping and prediction of physical entity states, and performing three-dimensional reproduction of spatial information. In this context, integrated sensing and communication (ISAC) is expected to be a crucial pathway for providing data sources to HDT. This paper proposes a four-layer architecture assisted by ISAC for HDT, integrating emerging paradigms and key technologies to achieve low-cost, high-precision environmental data collection for constructing HDT. Specifically, to enhance sensing resolution, we explore super-resolution techniques from the perspectives of parameter estimation and point cloud construction. Additionally, we focus on multi-point collaborative sensing for constructing HDT, and provide a comprehensive review of four key techniques: node selection, multi-band collaboration, cooperative beamforming, and data fusion. Finally, we highlight several interesting research directions to guide and inspire future work.

cs.MM↗

arXiv2025

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts

Do vision-language models (VLMs) pre-trained to caption an image of a "durian" learn visual concepts such as "brown" (color) and "spiky" (texture) at the same time? We aim to answer this question as visual concepts learned "for free" would enable wide applications such as neuro-symbolic reasoning or human-interpretable object classification. We assume that the visual concepts, if captured by pre-trained VLMs, can be extracted by their vision-language interface with text-based concept prompts. We observe that recent works prompting VLMs with concepts often differ in their strategies to define and evaluate the visual concepts, leading to conflicting conclusions. We propose a new concept definition strategy based on two observations: First, certain concept prompts include shortcuts that recognize correct concepts for wrong reasons; Second, multimodal information (e.g. visual discriminativeness, and textual knowledge) should be leveraged when selecting the concepts. Our proposed concept discovery and learning (CDL) framework is thus designed to identify a diverse list of generic visual concepts (e.g. "spiky" as opposed to "spiky durian"), which are ranked and selected based on visual and language mutual information. We carefully design quantitative and human evaluations of the discovered concepts on six diverse visual recognition datasets, which confirm that pre-trained VLMs do learn visual concepts that provide accurate and thorough descriptions for the recognized objects. All code and models are publicly released.

cs.CV↗

arXiv2024

Leveraging NFTs for Spectrum Securitization in 6G Networks

Dynamic Spectrum Sharing can enhance spectrum resource utilization by promoting the dynamic distribution of spectrum resources. However, to effectively implement dynamic spectrum resource allocation, certain mechanisms are needed to incentivize primary users to proactively share their spectrum resources. This paper, based on the ERC404 standard and integrating Non-Fungible Token and Fungible Token technologies, proposes a spectrum securitization model to incentivize spectrum resource sharing and implements it on the Ethereum test net.

cs.CE↗

arXiv2024

Perturbative approach to time-dependent quantum systems and applications to one-crossing multistate Landau-Zener models

We formulate a perturbative approach for studying a class of multi-level time-dependent quantum systems with constant off-diagonal couplings and diabatic energies being odd functions of time. Applying this approach to a general multistate Landau-Zener (MLZ) model with all diabatic levels crossing at one point (named the one-crossing MLZ model), we derive analytical formulas of all its transition probabilities up to $4$th order in the couplings. For one-crossing MLZ models it is difficult to obtain such analytical results by other kinds of approximation methods; thus, these perturbative results can serve as reliable benchmarks for future studies of any one-crossing MLZ models that have not been exactly solved.

quant-ph↗

arXiv2024

Edge AI-Enabled Chicken Health Detection Based on Enhanced FCOS-Lite and Knowledge Distillation

The utilization of AIoT technology has become a crucial trend in modern poultry management, offering the potential to optimize farming operations and reduce human workloads. This paper presents a real-time and compact edge-AI enabled detector designed to identify chickens and their healthy statuses using frames captured by a lightweight and intelligent camera equipped with an edge-AI enabled CMOS sensor. To ensure efficient deployment of the proposed compact detector within the memory-constrained edge-AI enabled CMOS sensor, we employ a FCOS-Lite detector leveraging MobileNet as the backbone. To mitigate the issue of reduced accuracy in compact edge-AI detectors without incurring additional inference costs, we propose a gradient weighting loss function as classification loss and introduce CIOU loss function as localization loss. Additionally, we propose a knowledge distillation scheme to transfer valuable information from a large teacher detector to the proposed FCOS-Lite detector, thereby enhancing its performance while preserving a compact model size. Experimental results demonstrate the proposed edge-AI enabled detector achieves commendable performance metrics, including a mean average precision (mAP) of 95.1$\%$ and an F1-score of 94.2$\%$, etc. Notably, the proposed detector can be efficiently deployed and operates at a speed exceeding 20 FPS on the edge-AI enabled CMOS sensor, achieved through int8 quantization. That meets practical demands for automated poultry health monitoring using lightweight intelligent cameras with low power consumption and minimal bandwidth costs.

cs.CV↗

arXiv2024

Text-Aware Diffusion for Policy Learning

Training an agent to achieve particular goals or perform desired behaviors is often accomplished through reinforcement learning, especially in the absence of expert demonstrations. However, supporting novel goals or behaviors through reinforcement learning requires the ad-hoc design of appropriate reward functions, which quickly becomes intractable. To address this challenge, we propose Text-Aware Diffusion for Policy Learning (TADPoLe), which uses a pretrained, frozen text-conditioned diffusion model to compute dense zero-shot reward signals for text-aligned policy learning. We hypothesize that large-scale pretrained generative models encode rich priors that can supervise a policy to behave not only in a text-aligned manner, but also in alignment with a notion of naturalness summarized from internet-scale training data. In our experiments, we demonstrate that TADPoLe is able to learn policies for novel goal-achievement and continuous locomotion behaviors specified by natural language, in both Humanoid and Dog environments. The behaviors are learned zero-shot without ground-truth rewards or expert demonstrations, and are qualitatively more natural according to human evaluation. We further show that TADPoLe performs competitively when applied to robotic manipulation tasks in the Meta-World environment, without having access to any in-domain demonstrations.

cs.LG↗

arXiv2024

Learning and Unlearning of Fabricated Knowledge in Language Models

What happens when a new piece of knowledge is introduced into the training data and how long does it last while a large language model (LM) continues to train? We investigate this question by injecting facts into LMs from a new probing dataset, "Outlandish", which is designed to permit the testing of a spectrum of different fact types. When studying how robust these memories are, there appears to be a sweet spot in the spectrum of fact novelty between consistency with world knowledge and total randomness, where the injected memory is the most enduring. Specifically we show that facts that conflict with common knowledge are remembered for tens of thousands of training steps, while prompts not conflicting with common knowledge (mundane), as well as scrambled prompts (randomly jumbled) are both forgotten much more rapidly. Further, knowledge-conflicting facts can "prime'' how the language model hallucinates on logically unrelated prompts, showing their propensity for non-target generalization, while both mundane and randomly jumbled facts prime significantly less. Finally, we show that impacts of knowledge-conflicting facts in LMs, though they can be long lasting, can be largely erased by novel application of multi-step sparse updates, even while the training ability of the model is preserved. As such, this very simple procedure has direct implications for mitigating the effects of data poisoning in training.

cs.CL↗

arXiv2024

An Efficient Approach to Generate Safe Drivable Space by LiDAR-Camera-HDmap Fusion

In this paper, we propose an accurate and robust perception module for Autonomous Vehicles (AVs) for drivable space extraction. Perception is crucial in autonomous driving, where many deep learning-based methods, while accurate on benchmark datasets, fail to generalize effectively, especially in diverse and unpredictable environments. Our work introduces a robust easy-to-generalize perception module that leverages LiDAR, camera, and HD map data fusion to deliver a safe and reliable drivable space in all weather conditions. We present an adaptive ground removal and curb detection method integrated with HD map data for enhanced obstacle detection reliability. Additionally, we propose an adaptive DBSCAN clustering algorithm optimized for precipitation noise, and a cost-effective LiDAR-camera frustum association that is resilient to calibration discrepancies. Our comprehensive drivable space representation incorporates all perception data, ensuring compatibility with vehicle dimensions and road regulations. This approach not only improves generalization and efficiency, but also significantly enhances safety in autonomous vehicle operations. Our approach is tested on a real dataset and its reliability is verified during the daily (including harsh snowy weather) operation of our autonomous shuttle, WATonoBus

cs.RO↗

arXiv2024

Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens

Scaling up autoregressive models in vision has not proven as beneficial as in large language models. In this work, we investigate this scaling problem in the context of text-to-image generation, focusing on two critical factors: whether models use discrete or continuous tokens, and whether tokens are generated in a random or fixed raster order using BERT- or GPT-like transformer architectures. Our empirical results show that, while all models scale effectively in terms of validation loss, their evaluation performance -- measured by FID, GenEval score, and visual quality -- follows different trends. Models based on continuous tokens achieve significantly better visual quality than those using discrete tokens. Furthermore, the generation order and attention mechanisms significantly affect the GenEval score: random-order models achieve notably better GenEval scores compared to raster-order models. Inspired by these findings, we train Fluid, a random-order autoregressive model on continuous tokens. Fluid 10.5B model achieves a new state-of-the-art zero-shot FID of 6.16 on MS-COCO 30K, and 0.69 overall score on the GenEval benchmark. We hope our findings and results will encourage future efforts to further bridge the scaling gap between vision and language models.

cs.CV↗

arXiv2024

Dense Video Object Captioning from Disjoint Supervision

We propose a new task and model for dense video object captioning -- detecting, tracking and captioning trajectories of objects in a video. This task unifies spatial and temporal localization in video, whilst also requiring fine-grained visual understanding that is best described by natural language. We propose a unified model, and demonstrate how our end-to-end approach is more accurate and temporally coherent than a multi-stage pipeline combining state-of-the-art detection, tracking, and captioning models. Moreover, we propose a training strategy based on a mixture of disjoint tasks, which allows us to leverage diverse, large-scale datasets which supervise different parts of our model. Although each pretraining task only provides weak supervision, they are complementary and, when combined, result in noteworthy zero-shot ability and serve as strong initialization for additional finetuning to further improve accuracy. We carefully design new metrics capturing all components of our task, and show how we can repurpose existing video grounding datasets (e.g. VidSTG and VLN) for our new task. We show that our model improves upon a number of strong baselines for this new task. Furthermore, we can apply our model to the task of spatial grounding, outperforming prior state-of-the-art on VidSTG and VLN, without explicitly training for it. Code is available at https://github.com/google-research/scenic/tree/main/scenic/projects/densevoc.

cs.CV↗

arXiv2024

EPO: Hierarchical LLM Agents with Environment Preference Optimization

Long-horizon decision-making tasks present significant challenges for LLM-based agents due to the need for extensive planning over multiple steps. In this paper, we propose a hierarchical framework that decomposes complex tasks into manageable subgoals, utilizing separate LLMs for subgoal prediction and low-level action generation. To address the challenge of creating training signals for unannotated datasets, we develop a reward model that leverages multimodal environment feedback to automatically generate reward signals. We introduce Environment Preference Optimization (EPO), a novel method that generates preference signals from the environment's feedback and uses them to train LLM-based agents. Extensive experiments on ALFRED demonstrate the state-of-the-art performance of our framework, achieving first place on the ALFRED public leaderboard and showcasing its potential to improve long-horizon decision-making in diverse environments.

cs.LG↗

arXiv2024

Do Music Generation Models Encode Music Theory?

Music foundation models possess impressive music generation capabilities. When people compose music, they may infuse their understanding of music into their work, by using notes and intervals to craft melodies, chords to build progressions, and tempo to create a rhythmic feel. To what extent is this true of music generation models? More specifically, are fundamental Western music theory concepts observable within the "inner workings" of these models? Recent work proposed leveraging latent audio representations from music generation models towards music information retrieval tasks (e.g. genre classification, emotion recognition), which suggests that high-level musical characteristics are encoded within these models. However, probing individual music theory concepts (e.g. tempo, pitch class, chord quality) remains under-explored. Thus, we introduce SynTheory, a synthetic MIDI and audio music theory dataset, consisting of tempos, time signatures, notes, intervals, scales, chords, and chord progressions concepts. We then propose a framework to probe for these music theory concepts in music foundation models (Jukebox and MusicGen) and assess how strongly they encode these concepts within their internal representations. Our findings suggest that music theory concepts are discernible within foundation models and that the degree to which they are detectable varies by model size and layer.

cs.SD↗

arXiv2024

Fermi Liquid Theory for Spin Current of a Ferromagnet

A recent work [arXiv:2402.04639] considered the dynamical equations for ferromagnets using Onsager's irreversible thermodynamics with fundamental variables magnetization $\vec{M}$ and spin current $\vec{J}_{i}$. The resulting equations have the same structure as Leggett's Fermi liquid theory for the nuclear paramagnet $^{3}$He. Specifically, $\partial_{t}\vec{J}_{i}$ contains a term varying as $\partial_{i}\vec{M}$ that we interpret as associated with a vector spin pressure, and a term giving a mean-field along $\vec{M}$, about which $\vec{J}_{i}$ precesses. (There is also a slow decay term in $\partial_{t}\vec{M}$ not normally present in the Leggett equations, which are intended for shorter-time spin-echo experiments.) The present work applies Fermi liquid theory to $\vec{J}_{i}$ of ferromagnets. The resulting dynamical equation for $\vec J_i$ confirms the form of $\vec J_i$ found in [arXiv:2402.04639], but now the previously unknown non-dissipative parameters are given in terms of the quasiparticle interaction parameters of Fermi liquid theory. In the paramagnetic limit the present theory agrees with Leggett and related work.

cond-mat.mes-hall↗

arXiv2024

Do Pre-trained Vision-Language Models Encode Object States?

For a vision-language model (VLM) to understand the physical world, such as cause and effect, a first step is to capture the temporal dynamics of the visual world, for example how the physical states of objects evolve over time (e.g. a whole apple into a sliced apple). Our paper aims to investigate if VLMs pre-trained on web-scale data learn to encode object states, which can be extracted with zero-shot text prompts. We curate an object state recognition dataset ChangeIt-Frames, and evaluate nine open-source VLMs, including models trained with contrastive and generative objectives. We observe that while these state-of-the-art vision-language models can reliably perform object recognition, they consistently fail to accurately distinguish the objects' physical states. Through extensive experiments, we identify three areas for improvements for VLMs to better encode object states, namely the quality of object localization, the architecture to bind concepts to objects, and the objective to learn discriminative visual and language encoders on object states. Data and code are released.

cs.CV↗

arXiv2024

DiReDi: Distillation and Reverse Distillation for AIoT Applications

Typically, the significant efficiency can be achieved by deploying different edge AI models in various real world scenarios while a few large models manage those edge AI models remotely from cloud servers. However, customizing edge AI models for each user's specific application or extending current models to new application scenarios remains a challenge. Inappropriate local training or fine tuning of edge AI models by users can lead to model malfunction, potentially resulting in legal issues for the manufacturer. To address aforementioned issues, this paper proposes an innovative framework called "DiReD", which involves knowledge DIstillation & REverse DIstillation. In the initial step, an edge AI model is trained with presumed data and a KD process using the cloud AI model in the upper management cloud server. This edge AI model is then dispatched to edge AI devices solely for inference in the user's application scenario. When the user needs to update the edge AI model to better fit the actual scenario, the reverse distillation (RD) process is employed to extract the knowledge: the difference between user preferences and the manufacturer's presumptions from the edge AI model using the user's exclusive data. Only the extracted knowledge is reported back to the upper management cloud server to update the cloud AI model, thus protecting user privacy by not using any exclusive data. The updated cloud AI can then update the edge AI model with the extended knowledge. Simulation results demonstrate that the proposed "DiReDi" framework allows the manufacturer to update the user model by learning new knowledge from the user's actual scenario with private data. The initial redundant knowledge is reduced since the retraining emphasizes user private data.

cs.LG↗

arXiv2024

Axion Magnetic Resonance: A Novel Enhancement in Axion-Photon Conversion

We identify a new resonance, axion magnetic resonance (AMR), that can greatly enhance the conversion rate between axions and photons. A series of axion search experiments rely on converting them into photons inside a constant magnetic field background. A common bottleneck of such experiments is the conversion amplitude being suppressed by the axion mass when $m_a \gtrsim 10^{-4}~$eV. We point out that a spatial or temporal variation in the magnetic field can cancel the difference between the photon dispersion relation and that of the axion, hence greatly enhancing the conversion probability. We demonstrate that the enhancement can be achieved by both a helical magnetic field profile and a harmonic oscillation of the magnitude. Our approach can extend the projected ALPS II reach in the axion-photon coupling ($g_{aγ}$) by two orders of magnitude at $m_a = 10^{-3}\;\mathrm{eV}$ with moderate assumptions.

hep-ph↗

arXiv2024

Probing New Physics with High-Redshift Quasars: Axions and Non-standard Cosmology

The Hubble diagram of quasars, as candidates to ``standardizable" candles, has been used to measure the expansion history of the Universe at late times, up to very high redshifts ($z \sim 7$). It has been shown that this history, as inferred from the quasar dataset, deviates at $\gtrsim 3 σ$ level from the concordance ($Λ$CDM) cosmology model preferred by the cosmic microwave background (CMB) and other datasets. In this article, we investigate whether new physics beyond $Λ$CDM (B$Λ$CDM) or beyond the Standard Model (BSM) could make the quasar data consistent with the concordance model. We first show that an effective redshift-dependent relation between the quasar UV and X-ray luminosities, complementing previous phenomenological work in the literature, can potentially remedy the discrepancy. Such a redshift dependence can be realized in a BSM model with axion-photon conversion in the intergalactic medium (IGM), although the preferred parameter space is {in tension with various other astrophysical constraints on axions, at a level} depending on the specific assumptions made regarding the IGM magnetic field. We briefly discuss a variation of the axion model that could evade these astrophysical constraints. On the other hand, we show that models beyond $Λ$CDM such as one with a varying dark energy equation of state ($w$CDM) or the phenomenological cosmographic model with a polynomial expansion of the luminosity distance, cannot alleviate the tension. The code for our analysis, based on \texttt{emcee}~\cite{Foreman_Mackey_2013} and \texttt{corner.py}~\cite{corner}, is publicly available at \href{https://github.com/ChenSun-Phys/high\_z\_candles.git}{\tt github.com/ChenSun-Phys/high\_z\_candles}.

astro-ph.CO↗

arXiv2024

Blockchain-Enabled Dynamic Spectrum Sharing for Satellite and Terrestrial Communication Networks

Dynamic spectrum sharing (DSS) between satellite and terrestrial networks has increasingly engaged the academic and industrial sectors. Nevertheless, facilitating secure, efficient and scalable sharing continues to pose a pivotal challenge. Emerging as a promising technology to bridge the trust gap among multiple participants, blockchain has been envisioned to enable DSS in a decentralized manner. However, satellites with limited resources may struggle to support the frequent interactions required by blockchain networks. Additionally,given the extensive coverage of satellites, spectrum sharing needs vary by regions, challenging traditional blockchain approaches to accommodate differences. In this work, a partitioned, self-governed, and customized dynamic spectrum sharing approach (PSC-DSS) is proposed for spectrum sharing between satellite access networks and terrestrial access networks. This approach establishes a sharded and tiered architecture which allows various regions to manage spectrum autonomously while jointly maintaining a single blockchain ledger. Moreover, a spectrum-consensus integrated mechanism, which decouples DSS process and couples it with blockchain consensus protocol, is designed to enable regions to conduct DSS transactions in parallel and dynamically innovate spectrum sharing schemes without affecting others. Furthermore, a theoretical framework is derived to justify the stability performance of PSC-DSS. Finally, simulations and experiments are conducted to validate the advantageous performance of PSC-DSS in terms of low-overhead, high efficiency, and robust stability.

cs.DC↗