arXiv subjects

Yu Zhou

Publications and source records attributed to Yu Zhou.

At least 235 records · Page 13Linked to original sources

arXiv2024

TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control

Centred on content modification and style preservation, Scene Text Editing (STE) remains a challenging task despite considerable progress in text-to-image synthesis and text-driven image manipulation recently. GAN-based STE methods generally encounter a common issue of model generalization, while Diffusion-based STE methods suffer from undesired style deviations. To address these problems, we propose TextCtrl, a diffusion-based method that edits text with prior guidance control. Our method consists of two key components: (i) By constructing fine-grained text style disentanglement and robust text glyph structure representation, TextCtrl explicitly incorporates Style-Structure guidance into model design and network training, significantly improving text style consistency and rendering accuracy. (ii) To further leverage the style prior, a Glyph-adaptive Mutual Self-attention mechanism is proposed which deconstructs the implicit fine-grained features of the source image to enhance style consistency and vision quality during inference. Furthermore, to fill the vacancy of the real-world STE evaluation benchmark, we create the first real-world image-pair dataset termed ScenePair for fair comparisons. Experiments demonstrate the effectiveness of TextCtrl compared with previous methods concerning both style fidelity and text accuracy.

cs.CV↗

arXiv2024

First Creating Backgrounds Then Rendering Texts: A New Paradigm for Visual Text Blending

Diffusion models, known for their impressive image generation abilities, have played a pivotal role in the rise of visual text generation. Nevertheless, existing visual text generation methods often focus on generating entire images with text prompts, leading to imprecise control and limited practicality. A more promising direction is visual text blending, which focuses on seamlessly merging texts onto text-free backgrounds. However, existing visual text blending methods often struggle to generate high-fidelity and diverse images due to a shortage of backgrounds for synthesis and limited generalization capabilities. To overcome these challenges, we propose a new visual text blending paradigm including both creating backgrounds and rendering texts. Specifically, a background generator is developed to produce high-fidelity and text-free natural images. Moreover, a text renderer named GlyphOnly is designed for achieving visually plausible text-background integration. GlyphOnly, built on a Stable Diffusion framework, utilizes glyphs and backgrounds as conditions for accurate rendering and consistency control, as well as equipped with an adaptive text block exploration strategy for small-scale text rendering. We also explore several downstream applications based on our method, including scene text dataset synthesis for boosting scene text detectors, as well as text image customization and editing. Code and model will be available at \url{https://github.com/Zhenhang-Li/GlyphOnly}.

cs.CV↗

arXiv2024

HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models

Model merging is a technique that combines multiple large pretrained models into a single model with enhanced performance and broader task adaptability. It has gained popularity in large pretrained model development due to its ability to bypass the need for original training data and further training processes. However, most existing model merging approaches focus solely on exploring the parameter space, merging models with identical architectures. Merging within the architecture space, despite its potential, remains in its early stages due to the vast search space and the challenges of layer compatibility. This paper marks a significant advance toward more flexible and comprehensive model merging techniques by modeling the architecture-space merging process as a reinforcement learning task. We train policy and value networks using offline sampling of weight vectors, which are then employed for the online optimization of merging strategies. Moreover, a multi-objective optimization paradigm is introduced to accommodate users' diverse task preferences, learning the Pareto front of optimal models to offer customized merging suggestions. Experimental results across multiple tasks, including text translation, mathematical reasoning, and code generation, validate the effectiveness and superiority of the proposed framework in model merging. The code will be made publicly available after the review process.

cs.LG↗

arXiv2024

Quantum optical coherence theory based on Feynman's path integral

Compared to classical optical coherence theory based on Maxwell's electromagnetic theory and Glauber's quantum optical coherence theory based on matrix mechanics formulation of quantum mechanics, quantum optical coherence theory based on Feynman's path integral formulation of quantum mechanics provides a novel tool to study optical coherence. It has the advantage of understanding the connection between mathematical calculations and physical interpretations better. Quantum optical coherence theory based on Feynman's path integral is introduced and reviewed in this paper. Based on the results of transient first-order interference of two independent light beams, it is predicted that the classical model for electric field of thermal light introduced by classical optical textbooks may not be accurate. The physics of two-photon bunching of thermal light and Hong-Ou-Mandel dip of entangled photon pairs is the same, which can be interpreted by constructive and destructive two-photon interference, respectively. Quantum optical coherence theory based on Feynman's path integral is helpful to understand the coherence properties of light, which may eventually lead us to the answer of the question: what is a photon?

quant-ph↗

arXiv2024

Study of the relativistic charged particle beam propagation in Earth's magnetic field

Relativistic charged particle beam can be used as destructive beam weapons in space for debris removal tasks. The trajectories of charged particles are affected by both electric and magnetic forces in the Earth's magnetic field. In this paper, we firstly analyzed the correlation parameters of the charged particle beam as a weapon when it propagated in the geomagnetic field. Then the models were constructed based on COMSOL Multiphysics and the IGRF model was adopted in the simulation. The gyro-radius and the related uncertainty were analyzed by simulation of the charged particle transport in the geomagnetic field at different altitudes. The charged beam spot radius divergency was also simulated. The magnetic field pinch effect can be found and can limit the beam spreading.

physics.space-ph↗

arXiv2024

Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper

Pre-trained multilingual speech foundation models, like Whisper, have shown impressive performance across different languages. However, adapting these models to new or specific languages is computationally extensive and faces catastrophic forgetting problems. Addressing these issues, our study investigates strategies to enhance the model on new languages in the absence of original training data, while also preserving the established performance on the original languages. Specifically, we first compare various LoRA-based methods to find out their vulnerability to forgetting. To mitigate this issue, we propose to leverage the LoRA parameters from the original model for approximate orthogonal gradient descent on the new samples. Additionally, we also introduce a learnable rank coefficient to allocate trainable parameters for more efficient training. Our experiments with a Chinese Whisper model (for Uyghur and Tibetan) yield better results with a more compact parameter set.

cs.CL↗

arXiv2024

ARMADA: Attribute-Based Multimodal Data Augmentation

In Multimodal Language Models (MLMs), the cost of manually annotating high-quality image-text pair data for fine-tuning and alignment is extremely high. While existing multimodal data augmentation frameworks propose ways to augment image-text pairs, they either suffer from semantic inconsistency between texts and images, or generate unrealistic images, causing knowledge gap with real world examples. To address these issues, we propose Attribute-based Multimodal Data Augmentation (ARMADA), a novel multimodal data augmentation method via knowledge-guided manipulation of visual attributes of the mentioned entities. Specifically, we extract entities and their visual attributes from the original text data, then search for alternative values for the visual attributes under the guidance of knowledge bases (KBs) and large language models (LLMs). We then utilize an image-editing model to edit the images with the extracted attributes. ARMADA is a novel multimodal data generation framework that: (i) extracts knowledge-grounded attributes from symbolic KBs for semantically consistent yet distinctive image-text pair generation, (ii) generates visually similar images of disparate categories using neighboring entities in the KB hierarchy, and (iii) uses the commonsense knowledge of LLMs to modulate auxiliary visual attributes such as backgrounds for more robust representation of original entities. Our empirical results over four downstream tasks demonstrate the efficacy of our framework to produce high-quality data and enhance the model performance. This also highlights the need to leverage external knowledge proxies for enhanced interpretability and real-world grounding.

cs.AI↗

arXiv2024

Balanced Binary Tree Schemes for Computing Zernike Radial Polynomials

Zernike radial polynomials (ZRP) play a significant role in application areas such as optics design, imaging systems, and image processing systems. Currently, there are two kinds of numerical schemes for computing the ZRP automatically with computer programs: one is based on the definition in which the factorial operations may lead to the overflow problem and the high order derivatives are troublesome, and the other is based on recursion which is either unstable or with high computational complexity. In this paper, our emphasis is focused on exploring the balanced binary tree (BBT) schemes for computing the ZRP: firstly an elegant formulae for computation is established; secondly the recursive and iterative algorithms based-on BBT are proposed; thirdly the computational complexity of the algorithms are analyzed rigorously; finally the performance of BBT schemes by testing the running time is verified and validated. Theoretical analysis shows that the computational complexity of balanced binary tree recursive algorithm (BBRTA) and iterative algorithm are exponential and quadratic respectively, which coincides with the running time test very well. Experiments show that the time consumption is about $1\sim 10$ microseconds with different computation platforms for the balanced binary tree iterative algorithm (BBTIA), which is stable and efficient for real-time applications. In the sense of STEM education, the connection of the BBT and ZRP exhibits the beauty and applications of discrete mathematical structure behind the engineering problem, which is worthy of introducing to the college students, computer programmers and optics engineers.

math.NA↗

arXiv2024

Even Order Explicit Symplectic Geometric Algorithms for Solving Quaternions in Guidance Navigation and Control via Diagonal Padé Approximation and Cayley Transform

Quaternion kinematical differential equation (QKDE) plays a key role in navigation, control and guidance systems. Although explicit symplectic geometric algorithms (ESGA) for this problem are available, there is a lack of a unified way for constructing high order symplectic difference schemes with configurable order parameter and the fractional interval sampling problem should be treated carefully. We present even order explicit symplectic geometric algorithms to solve the QKDE with diagonal Padé approximation via a four-step strategy. Firstly, the Padé-Cayley lemma is proved and used to simplify the symplectic Padé approximation for the linear Hamiltonian system with infinitesimal symplectic structure. Secondly, both parallel and alternative iterative methods are proposed to construct the symplectic difference schemes with even order accuracy. Thirdly, the symplecity, orthogonality and invertibility of the single-step transition matrices are proved rigorously. Finally, the explicit symplectic geometric algorithms are designed for both the linear time-invariant and linear time-varying QKDE. The maximum absolute error for solving the QKDE is $\mathcal{O}((t_f-t_0)τ^{2\ell})$ where $τ$ is the time step, $\ell$ is the order parameter and $[t_0,t_f]$ is the time span. The linear time complexity and constant space complexity of computation as well as the simple algorithmic structure show that our algorithms are appropriate for real-time applications in aeronautics, astronautics, robotics and so on. The performance of the proposed algorithms are verified and validated by mathematical analysis and numerical simulation.

eess.SY↗

arXiv2024

Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrieval

Scene text retrieval aims to find all images containing the query text from an image gallery. Current efforts tend to adopt an Optical Character Recognition (OCR) pipeline, which requires complicated text detection and/or recognition processes, resulting in inefficient and inflexible retrieval. Different from them, in this work we propose to explore the intrinsic potential of Contrastive Language-Image Pre-training (CLIP) for OCR-free scene text retrieval. Through empirical analysis, we observe that the main challenges of CLIP as a text retriever are: 1) limited text perceptual scale, and 2) entangled visual-semantic concepts. To this end, a novel model termed FDP (Focus, Distinguish, and Prompt) is developed. FDP first focuses on scene text via shifting the attention to the text area and probing the hidden text knowledge, and then divides the query text into content word and function word for processing, in which a semantic-aware prompting scheme and a distracted queries assistance module are utilized. Extensive experiments show that FDP significantly enhances the inference speed while achieving better or competitive retrieval accuracy compared to existing methods. Notably, on the IIIT-STR benchmark, FDP surpasses the state-of-the-art model by 4.37% with a 4 times faster speed. Furthermore, additional experiments under phrase-level and attribute-aware scene text retrieval settings validate FDP's particular advantages in handling diverse forms of query text. The source code will be publicly available at https://github.com/Gyann-z/FDP.

cs.CV↗

arXiv2024

Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models

Large language models (LLMs) have demonstrated exceptional performance not only in natural language processing tasks but also in a great variety of non-linguistic domains. In diverse optimization scenarios, there is also a rising trend of applying LLMs. However, whether the application of LLMs in the black-box optimization problems is genuinely beneficial remains unexplored. This paper endeavors to offer deep insights into the potential of LLMs in optimization through a comprehensive investigation, which covers both discrete and continuous optimization problems to assess the efficacy and distinctive characteristics that LLMs bring to this field. Our findings reveal both the limitations and advantages of LLMs in optimization. Specifically, on the one hand, despite the significant power consumed for running the models, LLMs exhibit subpar performance in pure numerical tasks, primarily due to a mismatch between the problem domain and their processing capabilities; on the other hand, although LLMs may not be ideal for traditional numerical optimization, their potential in broader optimization contexts remains promising, where LLMs exhibit the ability to solve problems in non-numerical domains and can leverage heuristics from the prompt to enhance their performance. To the best of our knowledge, this work presents the first systematic evaluation of LLMs for numerical optimization. Our findings pave the way for a deeper understanding of LLMs' role in optimization and guide future application of LLMs in a wide range of scenarios.

cs.NE↗

arXiv2024

Revolutionizing Wireless Networks with Self-Supervised Learning: A Pathway to Intelligent Communications

With the rapid proliferation of mobile devices and data, next-generation wireless communication systems face stringent requirements for ultra-low latency, ultra-high reliability, and massive connectivity. Traditional AI-driven wireless network designs, while promising, often suffer from limitations such as dependency on labeled data and poor generalization. To address these challenges, we present an integration of self-supervised learning (SSL) into wireless networks. SSL leverages large volumes of unlabeled data to train models, enhancing scalability, adaptability, and generalization. This paper offers a comprehensive overview of SSL, categorizing its application scenarios in wireless network optimization and presenting a case study on its impact on semantic communication. Our findings highlight the potentials of SSL to significantly improve wireless network performance without extensive labeled data, paving the way for more intelligent and efficient communication systems.

eess.SP↗

arXiv2024

CodeScore-R: An Automated Robustness Metric for Assessing the FunctionalCorrectness of Code Synthesis

Evaluation metrics are crucial in the field of code synthesis. Commonly used code evaluation metrics canbe classified into three types: match-based, semantic-based, and execution-based. Among them, the execution-basedPass@k metric accurately assesses the functionality of predicted code by executing test cases. However, calculatingthis metric requires a significant amount of overhead, necessitating the design of an automated evaluation metric thatcan assess the functionality of predicted code without the need for test cases. Additionally, a good evaluation metricshould be robust, that is the metric can maintain its accuracy even when the predicted code undergoes minor changes.To address these challenges, we propose an automated robust metric, called CodeScore-R, based on UniXcoder andcontrastive learning, for evaluating the functionality of code synthesis. CodeScore-R employs techniques such assketch-based processing, syntactic-equivalent transformations, and mutation testing to effectively mitigate theinterference caused by identifiers, syntax structures, and operators on evaluation results. Experimental resultsdemonstrate that in the tasks of code generation and migration in Java and Python, CodeScore-R outperforms otherevaluation metrics and is more closely aligned with the Pass@k metric, while exhibiting stronger robustness.

cs.SE↗

arXiv2024

Self-Modifying State Modeling for Simultaneous Machine Translation

Simultaneous Machine Translation (SiMT) generates target outputs while receiving stream source inputs and requires a read/write policy to decide whether to wait for the next source token or generate a new target token, whose decisions form a \textit{decision path}. Existing SiMT methods, which learn the policy by exploring various decision paths in training, face inherent limitations. These methods not only fail to precisely optimize the policy due to the inability to accurately assess the individual impact of each decision on SiMT performance, but also cannot sufficiently explore all potential paths because of their vast number. Besides, building decision paths requires unidirectional encoders to simulate streaming source inputs, which impairs the translation quality of SiMT models. To solve these issues, we propose \textbf{S}elf-\textbf{M}odifying \textbf{S}tate \textbf{M}odeling (SM$^2$), a novel training paradigm for SiMT task. Without building decision paths, SM$^2$ individually optimizes decisions at each state during training. To precisely optimize the policy, SM$^2$ introduces Self-Modifying process to independently assess and adjust decisions at each state. For sufficient exploration, SM$^2$ proposes Prefix Sampling to efficiently traverse all potential states. Moreover, SM$^2$ ensures compatibility with bidirectional encoders, thus achieving higher translation quality. Experiments show that SM$^2$ outperforms strong baselines. Furthermore, SM$^2$ allows offline machine translation models to acquire SiMT ability with fine-tuning.

cs.CL↗

arXiv2024

A DAFT Based Unified Waveform Design Framework for High-Mobility Communications

With the increasing demand for multi-carrier communication in high-mobility scenarios, it is urgent to design new multi-carrier communication waveforms that can resist large delay-Doppler spreads. Various multi-carrier waveforms in the transform domain were proposed for the fast time-varying channels, including orthogonal time frequency space (OTFS), orthogonal chirp division multiplexing (OCDM), and affine frequency division multiplexing (AFDM). Among these, the AFDM is a strong candidate for its low implementation complexity and ability to achieve optimal diversity. This paper unifies the waveforms based on the discrete affine Fourier transform (DAFT) by using the chirp slope factor "k" in the time-frequency representation to construct a unified design framework for high-mobility communications. The design framework is employed to verify that the bit error rate performance of the DAFT-based waveform can be enhanced when the signal-to-noise ratio (SNR) is sufficiently high by adjusting the chirp slope factor "k".

eess.SP↗

arXiv2024

Implicit Neural Image Field for Biological Microscopy Image Compression

The rapid pace of innovation in biological microscopy imaging has led to large images, putting pressure on data storage and impeding efficient sharing, management, and visualization. This necessitates the development of efficient compression solutions. Traditional CODEC methods struggle to adapt to the diverse bioimaging data and often suffer from sub-optimal compression. In this study, we propose an adaptive compression workflow based on Implicit Neural Representation (INR). This approach permits application-specific compression objectives, capable of compressing images of any shape and arbitrary pixel-wise decompression. We demonstrated on a wide range of microscopy images from real applications that our workflow not only achieved high, controllable compression ratios (e.g., 512x) but also preserved detailed information critical for downstream analysis.

cs.AI↗

arXiv2024

Development of the Low Frequency Telescope focal plane detector arrays for LiteBIRD

LiteBIRD, a forthcoming JAXA mission, aims to accurately study the microwave sky within the 40-400 GHz frequency range divided into 15 distinct nominal bands. The primary objective is to constrain the CMB inflationary signal, specifically the primordial B-modes. LiteBIRD targets the CMB B-mode signal on large angular scales, where the primordial inflationary signal is expected to dominate, with the goal of reaching a tensor-to-scalar ratio sensitivity of $σ_r\sim0.001$. LiteBIRD frequency bands will be split among three telescopes, with some overlap between telescopes for better control of systematic effects. Here we report on the development status of the detector arrays for the Low Frequency Telescope (LFT), which spans the 34-161 GHz range, with 12 bands subdivided between four types of trichroic pixels consisting of lenslet-coupled sinuous antennas. The signal from the antenna is bandpass filtered and sensed by AlMn Transition-Edge Sensors (TES). We provide an update on the status of the design and development of LiteBIRD's LFT LF1 (40-60-78 GHz), LF2 (50-68-89 GHz) pixels. We discuss design choices motivated by LiteBIRD scientific goals. In particular we focus on the details of the optimization of the design parameters of the sinuous antenna, on-chip bandpass filters, cross-under and impedance transformers and all the RF components that define the LF1 and LF2 pixel detection chain. We present this work in the context of the technical challenges and physical constraints imposed by the finite size of the instrument.

astro-ph.IM↗

arXiv2024

A Method of Measuring TES Complex ETF Response in Frequency-domain Multiplexed Readout by Single Sideband Power Modulation

The digital frequency domain multiplexing (DfMux) technique is widely used for astrophysical instruments with large detector arrays. Detailed detector characterization is required for instrument calibration and systematics control. We conduct the TES complex electrothermal-feedback (ETF) response measurement with the DfMux readout system as follows. By injecting a single sideband signal, we induce modulation in TES power dissipation over a frequency range encompassing the detector response. The modulated current signal induced by TES heating effect is measured, allowing for the ETF response characterization of the detector. With the injection of an upper sideband, the TES readout current shows both an upper and a lower sideband. We model the upper and lower sideband complex ETF response and verify the model by fitting to experimental data. The model not only can fit for certain physical parameters of the detector, such as loop gain, temperature sensitivity, current sensitivity, and time constant, but also enables us to estimate the systematic effect introduced by the multiplexed readout. The method is therefore useful for in-situ detector calibration and for estimating systematic effects during astronomical telescope observations, such as those performed by the upcoming LiteBIRD satellite.

astro-ph.IM↗