arXiv subjects

Bin Zhu

Publications and source records attributed to Bin Zhu.

At least 145 records · Page 8Linked to original sources

arXiv2024

Enhancing Recipe Retrieval with Foundation Models: A Data Augmentation Perspective

Learning recipe and food image representation in common embedding space is non-trivial but crucial for cross-modal recipe retrieval. In this paper, we propose a new perspective for this problem by utilizing foundation models for data augmentation. Leveraging on the remarkable capabilities of foundation models (i.e., Llama2 and SAM), we propose to augment recipe and food image by extracting alignable information related to the counterpart. Specifically, Llama2 is employed to generate a textual description from the recipe, aiming to capture the visual cues of a food image, and SAM is used to produce image segments that correspond to key ingredients in the recipe. To make full use of the augmented data, we introduce Data Augmented Retrieval framework (DAR) to enhance recipe and image representation learning for cross-modal retrieval. We first inject adapter layers to pre-trained CLIP model to reduce computation cost rather than fully fine-tuning all the parameters. In addition, multi-level circle loss is proposed to align the original and augmented data pairs, which assigns different penalties for positive and negative pairs. On the Recipe1M dataset, our DAR outperforms all existing methods by a large margin. Extensive ablation studies validate the effectiveness of each component of DAR.

cs.IR↗

arXiv2024

Differential Equations for Carrollian Amplitudes

Differential equations are powerful tools in the study of correlation functions in conformal field theories (CFTs). Carrollian amplitudes behave as correlation functions of Carrollian CFT that holographically describes asymptotically flat spacetime. We derive linear differential equations satisfied by Carrollian MHV gluon and graviton amplitudes. We obtain non-distributional solutions for both the gluon and graviton cases. We perform various consistency checks for these differential equations, including compatibility with conformal Carrollian symmetries.

hep-th↗

arXiv2024

Infrared structures of scattering on self-dual radiative backgrounds

The scattering of gluons and gravitons in trivial backgrounds is endowed with many surprising infrared features which have interesting conformal interpretations on the two-dimensional celestial sphere. However, the fate of these structures in more general asymptotically flat backgrounds is far from clear. In this paper, we consider holomorphic infrared structures in the presence of non-perturbative, self-dual background gauge and gravitational fields which are determined by freely specified radiative data. We make use of explicit formulae for tree-level gluon and graviton scattering in these self-dual radiative backgrounds, as well as chiral twistor sigma model descriptions of the classical dynamics. Remarkably, we find that the leading holomorphic part of tree-level collinear splitting functions -- or celestial OPEs -- and infinite-dimensional chiral soft algebras are undeformed by the background. We also compute all-order holomorphic celestial OPEs in the MHV sectors of gauge theory and gravity.

hep-th↗

arXiv2024

Eikonal amplitudes on the celestial sphere

Celestial scattering amplitudes for massless particles are Mellin transforms of momentum-space scattering amplitudes with respect to the energies of the external particles, and behave as conformal correlators on the celestial sphere. However, there are few explicit cases of well-defined celestial amplitudes, particularly for gravitational theories: the mixing between low- and high-energy scales induced by the Mellin transform generically yields divergent integrals. In this paper, we argue that the most natural object to consider is the gravitational amplitude dressed by an oscillating phase arising from semi-classical effects known as eikonal exponentiation. This leads to gravitational celestial amplitudes which are analytic, apart from a set of poles at integer negative conformal dimensions, whose degree and residues we characterize. We also study the large conformal dimension limits, and provide an asymptotic series representation for these celestial eikonal amplitudes. Our investigation covers two different frameworks, related by eikonal exponentiation: $2\to2$ scattering of scalars in flat spacetime and $1\to1$ scattering of a probe scalar particle in a curved, stationary spacetime. These provide data which any putative celestial dual for Minkowski, shockwave or black hole spacetimes must reproduce. We also derive dispersion and monodromy relations for these celestial amplitudes and discuss Carrollian eikonal-probe amplitudes in curved spacetimes.

hep-th↗

arXiv2024

Silting interval reduction and 0-Auslander extriangulated categories

We give a reduction technique for silting intervals in extriangulated categories, which we call "silting interval reduction". It provides a reduction technique for tilting subcategories when the extriangulated categories are exact categories. In 0-Auslander extriangulated categories (a generalization of the well-known two-term category $K^{[-1,0]}(\mathsf{proj}Λ)$ for an Artin algebra $Λ$), we provide a reduction theory for silting objects as an application of silting interval reduction. It unifies two-term silting reduction and Iyama-Yoshino's 2-Calabi-Yau reduction. The mutation theory developed by Gorsky, Nakaoka and Palu recently can be deduced from it. Since there are bijections between the silting objects and the support $τ$-tilting modules over certain finite dimensional algebras, we show it is compatible with $τ$-tilting reduction. This compatibility theorem also unifies the two compatibility theorems obtained by Jasso in his work on $τ$-tilting reduction. We give a new construction for 0-Auslander extriangulated categories using silting mutation, together with silting interval reduction, we obtain some results on silting quivers. Finally, we prove that $d$-Auslander extriangulated categories are related to a certain sequence of silting mutations.

math.RT↗

arXiv2024

Gravitational Wave as a Probe of Light Feebly Interacting Dark Matter

Light feebly interacting dark matter is widely predicted in a plethora of new physics models. However, due to very feeble couplings with the Standard Model particles, its relic density produced via the relativistic thermal freeze-out process easily exceeds the observed value. The entropy dilution in an early matter-dominated era provides an attractive mechanism for solving such an overabundance problem. In this work, we note that this dark matter dilution mechanism will lead to two distinctive kinks in the primordial GW spectrum, whose frequencies strongly correlate with the DM mass. We show that the GW detectors, such as Cosmic Explorer (CE) and Big Bang Observer (BBO), can measure the kinks in the primordial GW spectrum and will offer a new avenue to probe light feebly interacting dark matter.

hep-ph↗

arXiv2024

EfficientGS: Streamlining Gaussian Splatting for Large-Scale High-Resolution Scene Representation

In the domain of 3D scene representation, 3D Gaussian Splatting (3DGS) has emerged as a pivotal technology. However, its application to large-scale, high-resolution scenes (exceeding 4k$\times$4k pixels) is hindered by the excessive computational requirements for managing a large number of Gaussians. Addressing this, we introduce 'EfficientGS', an advanced approach that optimizes 3DGS for high-resolution, large-scale scenes. We analyze the densification process in 3DGS and identify areas of Gaussian over-proliferation. We propose a selective strategy, limiting Gaussian increase to key primitives, thereby enhancing the representational efficiency. Additionally, we develop a pruning mechanism to remove redundant Gaussians, those that are merely auxiliary to adjacent ones. For further enhancement, we integrate a sparse order increment for Spherical Harmonics (SH), designed to alleviate storage constraints and reduce training overhead. Our empirical evaluations, conducted on a range of datasets including extensive 4K+ aerial images, demonstrate that 'EfficientGS' not only expedites training and rendering times but also achieves this with a model size approximately tenfold smaller than conventional 3DGS while maintaining high rendering fidelity.

cs.CV↗

arXiv2024

FoodLMM: A Versatile Food Assistant using Large Multi-modal Model

Large Multi-modal Models (LMMs) have made impressive progress in many vision-language tasks. Nevertheless, the performance of general LMMs in specific domains is still far from satisfactory. This paper proposes FoodLMM, a versatile food assistant based on LMMs with various capabilities, including food recognition, ingredient recognition, recipe generation, nutrition estimation, food segmentation and multi-round conversation. To facilitate FoodLMM to deal with tasks beyond pure text output, we introduce a series of novel task-specific tokens and heads, enabling the model to predict food nutritional values and multiple segmentation masks. We adopt a two-stage training strategy. In the first stage, we utilize multiple public food benchmarks for multi-task learning by leveraging the instruct-following paradigm. In the second stage, we construct a multi-round conversation dataset and a reasoning segmentation dataset to fine-tune the model, enabling it to conduct professional dialogues and generate segmentation masks based on complex reasoning in the food domain. Our fine-tuned FoodLMM achieves state-of-the-art results across several food benchmarks. We will make our code, models and datasets publicly available.

cs.CV↗

arXiv2024

AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding

Nearly a decade of research in software engineering has focused on automating mobile app testing to help engineers in overcoming the unique challenges associated with the software platform. Much of this work has come in the form of Automated Input Generation tools (AIG tools) that dynamically explore app screens. However, such tools have repeatedly been demonstrated to achieve lower-than-expected code coverage - particularly on sophisticated proprietary apps. Prior work has illustrated that a primary cause of these coverage deficiencies is related to so-called tarpits, or complex screens that are difficult to navigate. In this paper, we take a critical step toward enabling AIG tools to effectively navigate tarpits during app exploration through a new form of automated semantic screen understanding. We introduce AURORA, a technique that learns from the visual and textual patterns that exist in mobile app UIs to automatically detect common screen designs and navigate them accordingly. The key idea of AURORA is that there are a finite number of mobile app screen designs, albeit with subtle variations, such that the general patterns of different categories of UI designs can be learned. As such, AURORA employs a multi-modal, neural screen classifier that is able to recognize the most common types of UI screen designs. After recognizing a given screen, it then applies a set of flexible and generalizable heuristics to properly navigate the screen. We evaluated AURORA both on a set of 12 apps with known tarpits from prior work, and on a new set of five of the most popular apps from the Google Play store. Our results indicate that AURORA is able to effectively navigate tarpit screens, outperforming prior approaches that avoid tarpits by 19.6% in terms of method coverage. The improvements can be attributed to AURORA's UI design classification and heuristic navigation techniques.

cs.SE↗

arXiv2024

Carrollian Amplitudes from Strings

Carrollian holography is supposed to describe gravity in four-dimensional asymptotically flat space-time by the three-dimensional Carrollian CFT living at null infinity. We transform superstring scattering amplitudes into the correlation functions of primary fields of Carrollian CFT depending on the three-dimensional coordinates of the celestial sphere and a retarded time coordinate. The power series in the inverse string tension is converted to a whole tower of both UV and IR finite descendants of the underlying field-theoretical Carrollian amplitude. We focus on four-point amplitudes involving gauge bosons and gravitons in type I open superstring theory and in closed heterotic superstring theory at the tree-level. We also discuss the limit of infinite retarded time coordinates, where the string world-sheet becomes celestial.

hep-th↗

arXiv2024

Probing Inelastic Signatures of Dark Matter Detection via Polarized Nucleus

We investigate the inelastic signatures of dark matter-nucleus interactions, explicitly focusing on the ramifications of polarization, dark matter splitting, and the Migdal effect. Direct detection experiments, crucial for testing the existence of dark matter, encounter formidable obstacles such as indomitable neutrino backgrounds and the elusive determination of dark matter spin. To overcome these challenges, we explore the potential of polarized-target dark matter scattering, examining the impact of nonvanishing mass splitting and the role of the Migdal effect in detecting light dark matter. Our analysis demonstrates the valuable utility of the polarized triple-differential event rate as an effective tool for studying inelastic dark matter. It enables us to investigate both angular and energy dependencies, providing valuable insights into the scattering process.

hep-ph↗

arXiv2024

From Canteen Food to Daily Meals: Generalizing Food Recognition to More Practical Scenarios

The precise recognition of food categories plays a pivotal role for intelligent health management, attracting significant research attention in recent years. Prominent benchmarks, such as Food-101 and VIREO Food-172, provide abundant food image resources that catalyze the prosperity of research in this field. Nevertheless, these datasets are well-curated from canteen scenarios and thus deviate from food appearances in daily life. This discrepancy poses great challenges in effectively transferring classifiers trained on these canteen datasets to broader daily-life scenarios encountered by humans. Toward this end, we present two new benchmarks, namely DailyFood-172 and DailyFood-16, specifically designed to curate food images from everyday meals. These two datasets are used to evaluate the transferability of approaches from the well-curated food image domain to the everyday-life food image domain. In addition, we also propose a simple yet effective baseline method named Multi-Cluster Reference Learning (MCRL) to tackle the aforementioned domain gap. MCRL is motivated by the observation that food images in daily-life scenarios exhibit greater intra-class appearance variance compared with those in well-curated benchmarks. Notably, MCRL can be seamlessly coupled with existing approaches, yielding non-trivial performance enhancements. We hope our new benchmarks can inspire the community to explore the transferability of food recognition models trained on well-curated datasets toward practical real-life applications.

cs.CV↗

arXiv2024

Detecting Quadratically Coupled Ultra-light Dark Matter with Stimulated Annihilation

Ultra-light Dark Matter (ULDM) is one of the most promising DM candidates. Due to the Bose enhancement, we find the annihilation rate of the ULDM in the presence of background photon radiation can be greatly enhanced and produce a distinctive reflected electromagnetic wave with an angular frequency equal to the ULDM mass. We propose to utilize such stimulated annihilation to probe the ULDM with the electromagnetic quadratic coupling by emitting a beam of radio into space. With a power of 50 MW emitter, we forecast the sensitivity of quadratic coupling in different local halo models for low-frequency radio telescopes, such as LOFAR, UTR-2 and ngLOBO.

hep-ph↗

arXiv2024

X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection

Generative adversarial networks (GANs) have remarkably advanced in diverse domains, especially image generation and editing. However, the misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns, which have gained widespread attention. Therefore, it is urgent to develop effective detection methods to distinguish between real and fake images. Current research centers around the application of transfer learning. Nevertheless, it encounters challenges such as knowledge forgetting from the original dataset and inadequate performance when dealing with imbalanced data during training. To alleviate this issue, this paper introduces a novel GAN-generated image detection algorithm called X-Transfer, which enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission. In addition, we combine AUC loss and cross-entropy loss to improve the model's performance. We carry out comprehensive experiments on multiple facial image datasets. The results show that our model outperforms the general transferring approach, and the best metric achieves 99.04%, which is increased by approximately 10%. Furthermore, we demonstrate excellent performance on non-face datasets, validating its generality and broader application prospects.

cs.CV↗

arXiv2024

Support $τ$-tilting subcategories in exact categories

Let $\mathcal{E}=(\mathcal{A},\mathcal{S})$ be an exact category with enough projectives $\mathcal{P}$. We introduce the notion of support $τ$-tilting subcategories of $\mathcal{E}$. It is compatible with existing definitions of support $τ$-tilting modules (subcategories) in various context. It is also a generalization of tilting subcategories of exact categories. We show that there is a bijection between support $τ$-tilting subcategories and certain $τ$-cotorsion pairs. Given a support $τ$-tilting subcategory $\mathcal{T}$, we find a subcategory $\mathcal{E}_{\mathcal{T}}$ of $\mathcal{E}$ which is an exact category and $\mathcal{T}$ is a tilting subcategory of $\mathcal{E}_{\mathcal{T}}$. If $\mathcal{E}$ is Krull-Schmidt, we prove the cardinal $|\mathcal{T}|$ is equal to the number of isomorphism classes of indecomposable projectives $Q$ such that ${\rm Hom}_{\mathcal{E}}(Q,\mathcal{T})\neq 0$. We also show a functorial version of Brenner-Butler's theorem.

math.RT↗

arXiv2024

Ideal mutations in triangulated categories and generalized Auslander-Reiten theory

We introduce the notion of ideal mutations in a triangulated category, which generalizes the version of Iyama and Yoshino \cite{iyama2008mutation} by replacing approximations by objects of a subcategory with approximations by morphisms of an ideal. As applications, for a Hom-finite Krull-Schmidt triangulated category $\mathcal{T}$ over an algebraically closed field $K$. (1) We generalize a theorem of Jorgensen \cite[Theorem 3.3]{jorgensen2010quotients} to a more general setting; (2) We provide a method to detect whether $\mathcal{T}$ has Auslander-Reiten triangles or not by checking the necessary and sufficient conditions on its Jacobson radical $\mathcal{J}$: (i) $\mathcal{J}$ is functorially finite, (ii) Gh$_{\mathcal{J}}= {\rm CoGh}_{\mathcal{J}}$, and (iii) Gh$_{\mathcal{J}}$-source maps coincide with Gh$_{\mathcal{J}}$-sink maps; (3) We generalize the classical Auslander-Reiten theory by using ideal mutations.

math.RT↗

arXiv2024

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

The video-language (VL) pretraining has achieved remarkable improvement in multiple downstream tasks. However, the current VL pretraining framework is hard to extend to multiple modalities (N modalities, N>=3) beyond vision and language. We thus propose LanguageBind, taking the language as the bind across different modalities because the language modality is well-explored and contains rich semantics. Specifically, we freeze the language encoder acquired by VL pretraining, then train encoders for other modalities with contrastive learning. As a result, all modalities are mapped to a shared feature space, implementing multi-modal semantic alignment. While LanguageBind ensures that we can extend VL modalities to N modalities, we also need a high-quality dataset with alignment data pairs centered on language. We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M. In our VIDAL-10M, all videos are from short video platforms with complete semantics rather than truncated segments from long videos, and all the video, depth, infrared, and audio modalities are aligned to their textual descriptions. LanguageBind has achieved superior performance on a wide range of 15 benchmarks covering video, audio, depth, and infrared. Moreover, multiple experiments have provided evidence for the effectiveness of LanguageBind in achieving indirect alignment and complementarity among diverse modalities. Code address: https://github.com/PKU-YuanGroup/LanguageBind

cs.CV↗

arXiv2024

An Improved Bound on Accelerated Light Dark Matter

Light (sub-GeV) dark matter has gained increasing interest in terms of direct detection. Accelerated dark matter is a promising candidate that can generate detectable nuclear recoil energy within the sub-GeV range. Because of the large kinetic energy, its interactions with the nucleus are predominantly governed by inelastic scattering, including quasi-elastic and deep inelastic scattering. In this work, we calculated the inelastic effects in dark matter--Earth scattering mediated by a vector particle. Our analysis revealed that the impact of inelastic scattering relies on the mediator mass and the kinetic energy spectrum of dark matter. The results exhibited considerable disparity: the upper bounds of the exclusion limit for the spin-independent cross-section between accelerated dark matter and nuclei via a heavy mediator differ by several tens of times when inelastic scattering is considered.

hep-ph↗