arXiv Science⌕ Search

arXiv subjects

Shuo Yan

Publications and source records attributed to Shuo Yan.

23 records · Page 2Linked to original sources

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria

Multimodal large language models (MLLMs) have broadened the scope of AI applications. Existing automatic evaluation methodologies for MLLMs are mainly limited in evaluating queries without considering user experiences, inadequately addressing the nuances of creative and associative multimodal tasks. However, the open-ended and subjective nature of such tasks poses a significant challenge to the evaluation methodology, where it is difficult to define the ground-truth answers for them. To this end, in our paper, we propose a new evaluation paradigm for MLLMs, which is evaluating MLLMs with per-sample criteria using potent MLLM as the judge. To validate the feasibility and effectiveness of this paradigm, we design a benchmark, dubbed MLLM-Bench, by curating the evaluation samples across six comprehensive cognitive levels. We benchmark 21 popular MLLMs in a pairwise-comparison fashion, showing diverse performance across models. Moreover, the validity of our benchmark manifests itself in reaching 88.02% agreement with human evaluation. We contend that the proposed paradigm explores the potential of MLLMs as effective evaluation tools with the help of per-sample criteria. See online leaderboard at \url{https://mllm-bench.llmzoo.com}.

cs.CL↗

Fast-oscillating random perturbations of Hamiltonian systems

We consider coupled slow-fast stochastic processes, where the averaged slow motion is given by a two-dimensional Hamiltonian system with multiple critical points. On a proper time scale, the evolution of the first integral converges to a diffusion process on the corresponding Reeb graph, with certain gluing conditions specified at the interior vertices, as in the case of additive white noise perturbations of Hamiltonian systems considered by M. Freidlin and A. Wentzell. The current paper provides the first result where the motion on a graph and the corresponding gluing conditions appear due to the averaging of a slow-fast system, with a Hamiltonian structure, on a large time scale. The result allows one to consider, for instance, long-time diffusion approximation for an oscillator with a potential with more than one well.

math.PR↗

Optical neural network architecture for deep learning with the temporal synthetic dimension

The physical concept of synthetic dimensions has recently been introduced into optics. The fundamental physics and applications are not yet fully understood, and this report explores an approach to optical neural networks using synthetic dimension in time domain, by theoretically proposing to utilize a single resonator network, where the arrival times of optical pulses are interconnected to construct a temporal synthetic dimension. The set of pulses in each roundtrip therefore provides the sites in each layer in the optical neural network, and can be linearly transformed with splitters and delay lines, including the phase modulators, when pulses circulate inside the network. Such linear transformation can be arbitrarily controlled by applied modulation phases, which serve as the building block of the neural network together with a nonlinear component for pulses. We validate the functionality of the proposed optical neural network for the deep learning purpose with examples handwritten digit recognition and optical pulse train distribution classification problems. This proof of principle computational work explores the new concept of developing a photonics-based machine learning in a single ring network using synthetic dimensions, which allows flexibility and easiness of reconfiguration with complex functionality in achieving desired optical tasks.

physics.optics↗

Large Deviations for Hamiltonian Systems on Intermediate Time Scales

We consider a two-dimensional Hamiltonian system perturbed by a small diffusion term, whose coefficient is state-dependent and non-degenerate. As a result, the process consists of the fast motion along the level curves and slow motion across them. On finite time intervals, the large deviation principle applies, while on time scales that are inversely proportional to the size of the perturbation, the averaging principle holds, i.e., the projection of the process onto the Reeb graph converges to a Markov process. In our paper, we consider the intermediate time scales and prove the large deviation principle, with the action functional determined in terms of the averaged process on the graph.

math.PR↗

Developing spatiotemporal solitons in step-index multimode fibers

Spatiotemporal solitons (STSs) are localized solitary waves in both space and time that involve complex linear and nonlinear processes. Optical STSs have been observed in various media, but they are difficult to be realized in multimode fibers due to their large modal dispersion. Here, we report STS mode-locking and spatiotemporal nonlinear dynamics in a step-index multimode fiber mediated by gain. Gain competition and energy redistribution among different spatial modes help nonlinearity effectively cancel both temporal splitting and spatial diffracting of the entire pulse (multicomponent soliton) and thereby maintain its shape during propagation. Optical STSs in multimode fibers therefore open a new way to clarify the fundamental science of spatiotemporal solitary waves and also imply important applications in multi-channel communications, optical switches and mode-area power scaling of fiber laser pulses.

nlin.PS↗