arXiv subjects

Yue Yang

Publications and source records attributed to Yue Yang.

At least 145 records · Page 8Linked to original sources

arXiv2024

PyOptInterface: Design and implementation of an efficient modeling language for mathematical optimization

This paper introduces the design and implementation of PyOptInterface, a modeling language for mathematical optimization embedded in Python programming language. PyOptInterface uses lightweight and compact data structure to bridge high-level entities in optimization models like variables and constraints to internal indices of optimizers efficiently. It supports a variety of optimization solvers and a range of common problem classes. We provide benchmarks to exhibit the competitive performance of PyOptInterface compared with other state-of-the-art modeling languages.

cs.MS↗

arXiv2024

ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models

This paper presents ConvBench, a novel multi-turn conversation evaluation benchmark tailored for Large Vision-Language Models (LVLMs). Unlike existing benchmarks that assess individual capabilities in single-turn dialogues, ConvBench adopts a three-level multimodal capability hierarchy, mimicking human cognitive processes by stacking up perception, reasoning, and creativity. Each level focuses on a distinct capability, mirroring the cognitive progression from basic perception to logical reasoning and ultimately to advanced creativity. ConvBench comprises 577 meticulously curated multi-turn conversations encompassing 215 tasks reflective of real-world demands. Automatic evaluations quantify response performance at each turn and overall conversation level. Leveraging the capability hierarchy, ConvBench enables precise attribution of conversation mistakes to specific levels. Experimental results reveal a performance gap between multi-modal models, including GPT4-V, and human performance in multi-turn conversations. Additionally, weak fine-grained perception in multi-modal models contributes to reasoning and creation failures. ConvBench serves as a catalyst for further research aimed at enhancing visual dialogues.

cs.MM↗

arXiv2024

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

Large Vision-Language Models (LVLMs) show significant strides in general-purpose multimodal applications such as visual dialogue and embodied navigation. However, existing multimodal evaluation benchmarks cover a limited number of multimodal tasks testing rudimentary capabilities, falling short in tracking LVLM development. In this study, we present MMT-Bench, a comprehensive benchmark designed to assess LVLMs across massive multimodal tasks requiring expert knowledge and deliberate visual recognition, localization, reasoning, and planning. MMT-Bench comprises $31,325$ meticulously curated multi-choice visual questions from various multimodal scenarios such as vehicle driving and embodied navigation, covering $32$ core meta-tasks and $162$ subtasks in multimodal understanding. Due to its extensive task coverage, MMT-Bench enables the evaluation of LVLMs using a task map, facilitating the discovery of in- and out-of-domain tasks. Evaluation results involving $30$ LVLMs such as the proprietary GPT-4V, GeminiProVision, and open-sourced InternVL-Chat, underscore the significant challenges posed by MMT-Bench. We anticipate that MMT-Bench will inspire the community to develop next-generation multimodal foundation models aimed at achieving general-purpose multimodal intelligence.

cs.CV↗

arXiv2024

Simulating unsteady fluid flows on a superconducting quantum processor

Recent advancements of intermediate-scale quantum processors have triggered tremendous interest in the exploration of practical quantum advantage. The simulation of fluid dynamics, a highly challenging problem in classical physics but vital for practical applications, emerges as a good candidate for showing quantum utility. Here, we report an experiment on the digital simulation of unsteady flows, which consists of quantum encoding, evolution, and detection of flow states, with a superconducting quantum processor. The quantum algorithm is based on the Hamiltonian simulation using the hydrodynamic formulation of the Schrödinger equation. With the median fidelities of 99.97% and 99.67% for parallel single- and two-qubit gates respectively, we simulate the dynamics of a two-dimensional (2D) compressible diverging flow and a 2D decaying vortex with ten qubits. The experimental results well capture the temporal evolution of averaged density and momentum profiles, and qualitatively reproduce spatial flow fields with moderate noises. This work demonstrates the potential of quantum computing in simulating more complex flows, such as turbulence, for practical applications.

quant-ph↗

arXiv2024

Holodeck: Language Guided Generation of 3D Embodied AI Environments

3D simulated environments play a critical role in Embodied AI, but their creation requires expertise and extensive manual effort, restricting their diversity and scope. To mitigate this limitation, we present Holodeck, a system that generates 3D environments to match a user-supplied prompt fully automatedly. Holodeck can generate diverse scenes, e.g., arcades, spas, and museums, adjust the designs for styles, and can capture the semantics of complex queries such as "apartment for a researcher with a cat" and "office of a professor who is a fan of Star Wars". Holodeck leverages a large language model (i.e., GPT-4) for common sense knowledge about what the scene might look like and uses a large collection of 3D assets from Objaverse to populate the scene with diverse objects. To address the challenge of positioning objects correctly, we prompt GPT-4 to generate spatial relational constraints between objects and then optimize the layout to satisfy those constraints. Our large-scale human evaluation shows that annotators prefer Holodeck over manually designed procedural baselines in residential scenes and that Holodeck can produce high-quality outputs for diverse scene types. We also demonstrate an exciting application of Holodeck in Embodied AI, training agents to navigate in novel scenes like music rooms and daycares without human-constructed data, which is a significant step forward in developing general-purpose embodied agents.

cs.CV↗

arXiv2024

Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneck

Black-box deep neural networks excel in text classification, yet their application in high-stakes domains is hindered by their lack of interpretability. To address this, we propose Text Bottleneck Models (TBM), an intrinsically interpretable text classification framework that offers both global and local explanations. Rather than directly predicting the output label, TBM predicts categorical values for a sparse set of salient concepts and uses a linear layer over those concept values to produce the final prediction. These concepts can be automatically discovered and measured by a Large Language Model (LLM) without the need for human curation. Experiments on 12 diverse text understanding datasets demonstrate that TBM can rival the performance of black-box baselines such as few-shot GPT-4 and finetuned DeBERTa while falling short against finetuned GPT-3.5. Comprehensive human evaluation validates that TBM can generate high-quality concepts relevant to the task, and the concept measurement aligns well with human judgments, suggesting that the predictions made by TBMs are interpretable. Overall, our findings suggest that TBM is a promising new framework that enhances interpretability with minimal performance tradeoffs.

cs.CL↗

arXiv2024

DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model

Text-to-image (T2I) generative models have attracted significant attention and found extensive applications within and beyond academic research. For example, the Civitai community, a platform for T2I innovation, currently hosts an impressive array of 74,492 distinct models. However, this diversity presents a formidable challenge in selecting the most appropriate model and parameters, a process that typically requires numerous trials. Drawing inspiration from the tool usage research of large language models (LLMs), we introduce DiffAgent, an LLM agent designed to screen the accurate selection in seconds via API calls. DiffAgent leverages a novel two-stage training framework, SFTA, enabling it to accurately align T2I API responses with user input in accordance with human preferences. To train and evaluate DiffAgent's capabilities, we present DABench, a comprehensive dataset encompassing an extensive range of T2I APIs from the community. Our evaluations reveal that DiffAgent not only excels in identifying the appropriate T2I API but also underscores the effectiveness of the SFTA training framework. Codes are available at https://github.com/OpenGVLab/DiffAgent.

cs.CL↗

arXiv2024

Augmented Reality Demonstrations for Scalable Robot Imitation Learning

Robot Imitation Learning (IL) is a widely used method for training robots to perform manipulation tasks that involve mimicking human demonstrations to acquire skills. However, its practicality has been limited due to its requirement that users be trained in operating real robot arms to provide demonstrations. This paper presents an innovative solution: an Augmented Reality (AR)-assisted framework for demonstration collection, empowering non-roboticist users to produce demonstrations for robot IL using devices like the HoloLens 2. Our framework facilitates scalable and diverse demonstration collection for real-world tasks. We validate our approach with experiments on three classical robotics tasks: reach, push, and pick-and-place. The real robot performs each task successfully while replaying demonstrations collected via AR.

cs.RO↗

arXiv2024

Cross or Wait? Predicting Pedestrian Interaction Outcomes at Unsignalized Crossings

Predicting pedestrian behavior when interacting with vehicles is one of the most critical challenges in the field of automated driving. Pedestrian crossing behavior is influenced by various interaction factors, including time to arrival, pedestrian waiting time, the presence of zebra crossing, and the properties and personality traits of both pedestrians and drivers. However, these factors have not been fully explored for use in predicting interaction outcomes. In this paper, we use machine learning to predict pedestrian crossing behavior including pedestrian crossing decision, crossing initiation time (CIT), and crossing duration (CD) when interacting with vehicles at unsignalized crossings. Distributed simulator data are utilized for predicting and analyzing the interaction factors. Compared with the logistic regression baseline model, our proposed neural network model improves the prediction accuracy and F1 score by 4.46% and 3.23%, respectively. Our model also reduces the root mean squared error (RMSE) for CIT and CD by 21.56% and 30.14% compared with the linear regression model. Additionally, we have analyzed the importance of interaction factors, and present the results of models using fewer factors. This provides information for model selection in different scenarios with limited input features.

cs.LG↗

arXiv2024

A Concept-based Interpretable Model for the Diagnosis of Choroid Neoplasias using Multimodal Data

Diagnosing rare diseases presents a common challenge in clinical practice, necessitating the expertise of specialists for accurate identification. The advent of machine learning offers a promising solution, while the development of such technologies is hindered by the scarcity of data on rare conditions and the demand for models that are both interpretable and trustworthy in a clinical context. Interpretable AI, with its capacity for human-readable outputs, can facilitate validation by clinicians and contribute to medical education. In the current work, we focus on choroid neoplasias, the most prevalent form of eye cancer in adults, albeit rare with 5.1 per million. We built the so-far largest dataset consisting of 750 patients, incorporating three distinct imaging modalities collected from 2004 to 2022. Our work introduces a concept-based interpretable model that distinguishes between three types of choroidal tumors, integrating insights from domain experts via radiological reports. Remarkably, this model not only achieves an F1 score of 0.91, rivaling that of black-box models, but also boosts the diagnostic accuracy of junior doctors by 42%. This study highlights the significant potential of interpretable machine learning in improving the diagnosis of rare diseases, laying a groundwork for future breakthroughs in medical AI that could tackle a wider array of complex health scenarios.

cs.LG↗

arXiv2024

Annotated Hands for Generative Models

Generative models such as GANs and diffusion models have demonstrated impressive image generation capabilities. Despite these successes, these systems are surprisingly poor at creating images with hands. We propose a novel training framework for generative models that substantially improves the ability of such systems to create hand images. Our approach is to augment the training images with three additional channels that provide annotations to hands in the image. These annotations provide additional structure that coax the generative model to produce higher quality hand images. We demonstrate this approach on two different generative models: a generative adversarial network and a diffusion model. We demonstrate our method both on a new synthetic dataset of hand images and also on real photographs that contain hands. We measure the improved quality of the generated hands through higher confidence in finger joint identification using an off-the-shelf hand detector.

cs.CV↗

arXiv2024

Weaving classical turbulence with quantum skeleton

Matter entanglement is a common chaotic structure in both quantum and classical systems. Turbulence can be pictured as a tangle of vortex filaments in superfluids and viscous vortices in classical fluids. However, it is hard to explain how the statistical properties of turbulence arise from elemental structures. Here we use the quantum vortex tangle as a skeleton to generate an instantaneous classical turbulent field with intertwined vortex tubes. Combining the quantum skeleton and tunable vortex thickness makes the synthetic turbulence satisfy key statistical laws and provides valuable insights for elucidating energy cascade and extreme events. By manipulating the elemental structures, we customize turbulence with desired statistical features. This bottom-up approach of "weaving" turbulence provides a testbed for analyzing and modeling turbulence.

physics.flu-dyn↗

arXiv2024

Electromagnetic Information Theory: Fundamentals and Applications for 6G Wireless Communication Systems

In wireless communications, electromagnetic theory and information theory constitute a pair of fundamental theories, bridged by antenna theory and wireless propagation channel modeling theory. Up to the fifth generation (5G) wireless communication networks, these four theories have been developing relatively independently. However, in sixth generation (6G) space-air-ground-sea wireless communication networks, seamless coverage is expected in the three-dimensional (3D) space, potentially necessitating the acquisition of channel state information (CSI) and channel capacity calculation at anywhere and any time. Additionally, the key 6G technologies such as ultra-massive multiple-input multiple-output (MIMO) and holographic MIMO achieves intricate interaction of the antennas and wireless propagation environments, which necessitates the joint modeling of antennas and wireless propagation channels. To address the challenges in 6G, the integration of the above four theories becomes inevitable, leading to the concept of the so-called electromagnetic information theory (EIT). In this article, a suite of 6G key technologies is highlighted. Then, the concepts and relationships of the four theories are unveiled. Finally, the necessity and benefits of integrating them into the EIT are revealed.

cs.IT↗

arXiv2023

CornerFormer: Boosting Corner Representation for Fine-Grained Structured Reconstruction

Structured reconstruction is a non-trivial dense prediction problem, which extracts structural information (\eg, building corners and edges) from a raster image, then reconstructs it to a 2D planar graph accordingly. Compared with common segmentation or detection problems, it significantly relays on the capability that leveraging holistic geometric information for structural reasoning. Current transformer-based approaches tackle this challenging problem in a two-stage manner, which detect corners in the first model and classify the proposed edges (corner-pairs) in the second model. However, they separate two-stage into different models and only share the backbone encoder. Unlike the existing modeling strategies, we present an enhanced corner representation method: 1) It fuses knowledge between the corner detection and edge prediction by sharing feature in different granularity; 2) Corner candidates are proposed in four heatmap channels w.r.t its direction. Both qualitative and quantitative evaluations demonstrate that our proposed method can better reconstruct fine-grained structures, such as adjacent corners and tiny edges. Consequently, it outperforms the state-of-the-art model by +1.9\%@F-1 on Corner and +3.0\%@F-1 on Edge.

cs.CV↗

arXiv2023

Generating Chord Progression from Melody with Flexible Harmonic Rhythm and Controllable Harmonic Density

Melody harmonization, which involves generating a chord progression that complements a user-provided melody, continues to pose a significant challenge. A chord progression must not only be in harmony with the melody, but also interdependent on its rhythmic pattern. While previous neural network-based systems have been successful in producing chord progressions for given melodies, they have not adequately addressed controllable melody harmonization, nor have they focused on generating harmonic rhythms with flexibility in the rates or patterns of chord changes. This paper presents AutoHarmonizer, a novel system for harmonic density-controllable melody harmonization with such a flexible harmonic rhythm. AutoHarmonizer is equipped with an extensive vocabulary of 1,462 chord types and can generate chord progressions that vary in harmonic density for a given melody. Experimental results indicate that the AutoHarmonizer-generated chord progressions exhibit a diverse range of harmonic rhythms and that the system's controllable harmonic density is effective.

cs.SD↗

arXiv2023

Colossal Magnetoresistance in Twisted Intertwined Graphene Spirals

Colossal magnetoresistance (CMR) is highly applicable in spintronic devices such as magnetic sensors, magnetic memory, and hard drives. Typically, CMR is found in Weyl semimetals characterized by perfect electron-hole symmetry or exceptionally high electric conductivity and mobility. Our study explores this phenomenon in a recently developed graphene moir$\acute{e}$ system, which demonstrates CMR owing to its topological structure and high-quality crystal formation. We specifically investigate the electronic properties of three-dimensional (3D) intertwined twisted graphene spirals (TGS), manipulating the screw dislocation axis to achieve a rotation angle of 7.3$^{\circ}$. Notably, at 14 T and 2 K, the magnetoresistance of these structures reached 1.7$\times$10$^7$%, accompanied by an unexpected metal-to-insulator transition as the temperature increased. This transition becomes noticeable when the magnetic field exceeds a minimal threshold of approximately 0.1 T. These observations suggest the existence of complex, correlated states within the partially filled three-dimensional Landau levels of the 3D TGS system. Our findings open up new possibilities for achieving CMR by engineering the topological structure of 2D layered moir$\acute{e}$ systems.

cond-mat.mes-hall↗

arXiv2023

Lagrangian dynamics and regularity of the spin Euler equation

We derive the spin Euler equation for ideal flows by applying the spherical Clebsch mapping. This equation is based on the spin vector rather than the velocity. It enables a feasible Lagrangian study of fluid dynamics, as the isosurface of a spin-vector component is a vortex surface and material surface in ideal flows. The spin Euler equation is also equivalent to a special case of the Landau-Lifshitz equation with a specific effective magnetic field, revealing a possible connection between ideal flow and magnetic crystal. We conduct direct numerical simulations of three ideal flows of the vortex knot, vortex link and modified Taylor-Green flow by solving the spin Euler equation. The evolution of the Lagrangian vortex surface illustrates that the regions with large vorticity are rapidly stretched into spiral sheets. We establish a non-blowup criterion for the spin Euler equation, suggesting that the Laplacian of the spin vector must diverge if the solution forms a singularity at some finite time. The DNS result exhibits a pronounced double-exponential growth of the maximum norm of Laplacian of the spin vector, showing no evidence of the finite-time singularity formation if the double-exponential growth holds at later times. Moreover, the present criterion with Lagrangian nature appears to be more sensitive than the Beale-Kato-Majda criterion in detecting the flows that are incapable of producing finite-time singularities.

physics.flu-dyn↗

arXiv2023

Isola and mushroom dynamics of limit cycles and bifurcations in a predator-prey system with additive Allee effect

This paper investigates a predator-prey system with an additive Allee effect and a generalized Holling IV functional response using a dynamical system approach. By means of a mixture of analytical and numerical procedures, we find the existence of codimension two and three Bogdanov-Takens bifurcation, codimension-three generalized Hopf bifurcation, and codimension-two cusp of limit cycles. We also found mushroom and isola bifurcations of limit cycles as the first examples of such phenomena in a predatory interaction. The model predicts that extinction of both populations may only occur if the Allee effect is strong. However, long term coexistence is possible in both weak and strong Allee regimes indicating that predation has a balancing role in the interaction dynamics. Nonetheless, a weak Allee effect can result in complex dynamics as well, including the presence of isolas, mushrooms and cusps of limit cycles.

math.DS↗