arXiv subjects

Yin Li

Publications and source records attributed to Yin Li.

At least 145 records · Page 8Linked to original sources

arXiv2021

Virtuoso: Video-based Intelligence for real-time tuning on SOCs

Efficient and adaptive computer vision systems have been proposed to make computer vision tasks, such as image classification and object detection, optimized for embedded or mobile devices. These solutions, quite recent in their origin, focus on optimizing the model (a deep neural network, DNN) or the system by designing an adaptive system with approximation knobs. In spite of several recent efforts, we show that existing solutions suffer from two major drawbacks. First, the system does not consider energy consumption of the models while making a decision on which model to run. Second, the evaluation does not consider the practical scenario of contention on the device, due to other co-resident workloads. In this work, we propose an efficient and adaptive video object detection system, Virtuoso, which is jointly optimized for accuracy, energy efficiency, and latency. Underlying Virtuoso is a multi-branch execution kernel that is capable of running at different operating points in the accuracy-energy-latency axes, and a lightweight runtime scheduler to select the best fit execution branch to satisfy the user requirement. To fairly compare with Virtuoso, we benchmark 15 state-of-the-art or widely used protocols, including Faster R-CNN (FRCNN), YOLO v3, SSD, EfficientDet, SELSA, MEGA, REPP, FastAdapt, and our in-house adaptive variants of FRCNN+, YOLO+, SSD+, and EfficientDet+ (our variants have enhanced efficiency for mobiles). With this comprehensive benchmark, Virtuoso has shown superiority to all the above protocols, leading the accuracy frontier at every efficiency level on NVIDIA Jetson mobile GPUs. Specifically, Virtuoso has achieved an accuracy of 63.9%, which is more than 10% higher than some of the popular object detection models, FRCNN at 51.1%, and YOLO at 49.5%.

cs.CV↗

arXiv2021

RegionCLIP: Region-based Language-Image Pretraining

Contrastive language-image pretraining (CLIP) using image-text pairs has achieved impressive results on image classification in both zero-shot and transfer learning settings. However, we show that directly applying such models to recognize image regions for object detection leads to poor performance due to a domain shift: CLIP was trained to match an image as a whole to a text description, without capturing the fine-grained alignment between image regions and text spans. To mitigate this issue, we propose a new method called RegionCLIP that significantly extends CLIP to learn region-level visual representations, thus enabling fine-grained alignment between image regions and textual concepts. Our method leverages a CLIP model to match image regions with template captions and then pretrains our model to align these region-text pairs in the feature space. When transferring our pretrained model to the open-vocabulary object detection tasks, our method significantly outperforms the state of the art by 3.8 AP50 and 2.2 AP for novel categories on COCO and LVIS datasets, respectively. Moreoever, the learned region representations support zero-shot inference for object detection, showing promising results on both COCO and LVIS datasets. Our code is available at https://github.com/microsoft/RegionCLIP.

cs.CV↗

arXiv2021

Nonexistence of exact Lagrangian tori in affine conic bundles over $\mathbb{C}^n$

Let $M\subset\mathbb{C}^{n+1}$ be a smooth affine hypersurface defined by the equation $xy+p(z_1,\cdots,z_{n-1})=1$, where $p$ is a Brieskorn-Pham polynomial and $n\geq2$. We prove that if $L\subset M$ is an orientable exact Lagrangian submanifold, then $L$ does not admit a Riemannian metric with non-positive sectional curvature. The key point of the proof is to establish a version of homological mirror symmetry for the wrapped Fukaya category of $M$, from which the finite-dimensionality of the symplectic cohomology group $\mathit{SH}^0(M)$ follows by a Hochschild cohomology computation.

math.SG↗

arXiv2021

3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image

Visual content creation has spurred a soaring interest given its applications in mobile photography and AR / VR. Style transfer and single-image 3D photography as two representative tasks have so far evolved independently. In this paper, we make a connection between the two, and address the challenging task of 3D photo stylization - generating stylized novel views from a single image given an arbitrary style. Our key intuition is that style transfer and view synthesis have to be jointly modeled for this task. To this end, we propose a deep model that learns geometry-aware content features for stylization from a point cloud representation of the scene, resulting in high-quality stylized images that are consistent across views. Further, we introduce a novel training protocol to enable the learning using only 2D images. We demonstrate the superiority of our method via extensive qualitative and quantitative studies, and showcase key applications of our method in light of the growing demand for 3D content creation from 2D image assets.

cs.CV↗

arXiv2021

NECOLA: Towards a Universal Field-level Cosmological Emulator

We train convolutional neural networks to correct the output of fast and approximate N-body simulations at the field level. Our model, Neural Enhanced COLA --NECOLA--, takes as input a snapshot generated by the computationally efficient COLA code and corrects the positions of the cold dark matter particles to match the results of full N-body Quijote simulations. We quantify the accuracy of the network using several summary statistics, and find that NECOLA can reproduce the results of the full N-body simulations with sub-percent accuracy down to $k\simeq1~h{\rm Mpc}^{-1}$. Furthermore, the model, that was trained on simulations with a fixed value of the cosmological parameters, is also able to correct the output of COLA simulations with different values of $Ω_{\rm m}$, $Ω_{\rm b}$, $h$, $n_s$, $σ_8$, $w$, and $M_ν$ with very high accuracy: the power spectrum and the cross-correlation coefficients are within $\simeq1\%$ down to $k=1~h{\rm Mpc}^{-1}$. Our results indicate that the correction to the power spectrum from fast/approximate simulations or field-level perturbation theory is rather universal. Our model represents a first step towards the development of a fast field-level emulator to sample not only primordial mode amplitudes and phases, but also the parameter space defined by the values of the cosmological parameters.

astro-ph.CO↗

arXiv2021

The CAMELS Multifield Dataset: Learning the Universe's Fundamental Parameters with Artificial Intelligence

We present the Cosmology and Astrophysics with MachinE Learning Simulations (CAMELS) Multifield Dataset, CMD, a collection of hundreds of thousands of 2D maps and 3D grids containing many different properties of cosmic gas, dark matter, and stars from 2,000 distinct simulated universes at several cosmic times. The 2D maps and 3D grids represent cosmic regions that span $\sim$100 million light years and have been generated from thousands of state-of-the-art hydrodynamic and gravity-only N-body simulations from the CAMELS project. Designed to train machine learning models, CMD is the largest dataset of its kind containing more than 70 Terabytes of data. In this paper we describe CMD in detail and outline a few of its applications. We focus our attention on one such task, parameter inference, formulating the problems we face as a challenge to the community. We release all data and provide further technical details at https://camels-multifield-dataset.readthedocs.io.

cs.LG↗

arXiv2021

Robust marginalization of baryonic effects for cosmological inference at the field level

We train neural networks to perform likelihood-free inference from $(25\,h^{-1}{\rm Mpc})^2$ 2D maps containing the total mass surface density from thousands of hydrodynamic simulations of the CAMELS project. We show that the networks can extract information beyond one-point functions and power spectra from all resolved scales ($\gtrsim 100\,h^{-1}{\rm kpc}$) while performing a robust marginalization over baryonic physics at the field level: the model can infer the value of $Ω_{\rm m} (\pm 4\%)$ and $σ_8 (\pm 2.5\%)$ from simulations completely different to the ones used to train it.

astro-ph.CO↗

arXiv2021

Multifield Cosmology with Artificial Intelligence

Astrophysical processes such as feedback from supernovae and active galactic nuclei modify the properties and spatial distribution of dark matter, gas, and galaxies in a poorly understood way. This uncertainty is one of the main theoretical obstacles to extract information from cosmological surveys. We use 2,000 state-of-the-art hydrodynamic simulations from the CAMELS project spanning a wide variety of cosmological and astrophysical models and generate hundreds of thousands of 2-dimensional maps for 13 different fields: from dark matter to gas and stellar properties. We use these maps to train convolutional neural networks to extract the maximum amount of cosmological information while marginalizing over astrophysical effects at the field level. Although our maps only cover a small area of $(25~h^{-1}{\rm Mpc})^2$, and the different fields are contaminated by astrophysical effects in very different ways, our networks can infer the values of $Ω_{\rm m}$ and $σ_8$ with a few percent level precision for most of the fields. We find that the marginalization performed by the network retains a wealth of cosmological information compared to a model trained on maps from gravity-only N-body simulations that are not contaminated by astrophysical effects. Finally, we train our networks on multifields -- 2D maps that contain several fields as different colors or channels -- and find that not only they can infer the value of all parameters with higher accuracy than networks trained on individual fields, but they can constrain the value of $Ω_{\rm m}$ with higher accuracy than the maps from the N-body simulations.

astro-ph.CO↗

arXiv2021

AI-assisted super-resolution cosmological simulations II: Halo substructures, velocities and higher order statistics

In this work, we expand and test the capabilities of our recently developed super-resolution (SR) model to generate high-resolution (HR) realizations of the full phase-space matter distribution, including both displacement and velocity, from computationally cheap low-resolution (LR) cosmological N-body simulations. The SR model enhances the simulation resolution by generating 512 times more tracer particles, extending into the deeply non-linear regime where complex structure formation processes take place. We validate the SR model by deploying the model in 10 test simulations of box size 100 Mpc/h, and examine the matter power spectra, bispectra and 2D power spectra in redshift space. We find the generated SR field matches the true HR result at percent level down to scales of k ~ 10 h/Mpc. We also identify and inspect dark matter halos and their substructures. Our SR model generate visually authentic small-scale structures, that cannot be resolved by the LR input, and are in good statistical agreement with the real HR results. The SR model performs satisfactorily on the halo occupation distribution, halo correlations in both real and redshift space, and the pairwise velocity distribution, matching the HR results with comparable scatter, thus demonstrating its potential in making mock halo catalogs. The SR technique can be a powerful and promising tool for modelling small-scale galaxy formation physics in large cosmological volumes.

astro-ph.CO↗

arXiv2021

Learning to Generate Scene Graph from Natural Language Supervision

Learning from image-text data has demonstrated recent success for many recognition tasks, yet is currently limited to visual features or individual visual concepts such as objects. In this paper, we propose one of the first methods that learn from image-sentence pairs to extract a graphical representation of localized objects and their relationships within an image, known as scene graph. To bridge the gap between images and texts, we leverage an off-the-shelf object detector to identify and localize object instances, match labels of detected regions to concepts parsed from captions, and thus create "pseudo" labels for learning scene graph. Further, we design a Transformer-based model to predict these "pseudo" labels via a masked token prediction task. Learning from only image-sentence pairs, our model achieves 30% relative gain over a latest method trained with human-annotated unlocalized scene graphs. Our model also shows strong results for weakly and fully supervised scene graph generation. In addition, we explore an open-vocabulary setting for detecting scene graphs, and present the first result for open-set scene graph generation. Our code is available at https://github.com/YiwuZhong/SGG_from_NLS.

cs.CV↗

arXiv2021

Global existence and decay rates for a generic compressible two-fluid model

We investigate global existence and optimal decay rates of a generic non-conservative compressible two--fluid model with general constant viscosities and capillary coefficients.The main novelty of this work is three--fold: First, for any integer $\ell\geq3$, we show that the densities and velocities converge to their corresponding equilibrium states at the $L^2$ rate $(1+t)^{-\frac{3}{4}}$, and the $k$($\in [1, \ell]$)--order spatial derivatives of them converge to zero at the $L^2$ rate $(1+t)^{-\frac{3}{4}-\frac{k}{2}}$, which are the same as ones of the compressible Navier--Stokes system, Navier--Stokes--Korteweg system and heat equation. Second, the linear combination of the fraction densities ($β^+α^+ρ^++β^-α^-ρ^-$) converges to its corresponding equilibrium state at the $L^2$ rate $(1+t)^{-\frac{3}{4}}$, and its $k$($\in [1, \ell]$)--order spatial derivative converges to zero at the $L^2$ rate $(1+t)^{-\frac{3}{4}-\frac{k}{2}}$, but the fraction densities ($α^\pmρ^\pm$) themselves converge to their corresponding equilibrium states at the $L^2$ rate $(1+t)^{-\frac{1}{4}}$, and the $k$($\in [1, \ell]$)--order spatial derivatives of them converge to zero at the $L^2$ rate $(1+t)^{-\frac{1}{4}-\frac{k}{2}}$, which are slower than ones of their linear combination ($β^+α^+ρ^++β^-α^-ρ^-$) and the densities. We think that this phenomenon should owe to the special structure of the system. Finally, for well--chosen initial data, we also prove the lower bounds on the decay rates, which are the same as those of the upper decay rates. Therefore, these decay rates are optimal for the compressible two--fluid model.

math.AP↗

arXiv2021

The CAMELS project: Cosmology and Astrophysics with MachinE Learning Simulations

We present the Cosmology and Astrophysics with MachinE Learning Simulations --CAMELS-- project. CAMELS is a suite of 4,233 cosmological simulations of $(25~h^{-1}{\rm Mpc})^3$ volume each: 2,184 state-of-the-art (magneto-)hydrodynamic simulations run with the AREPO and GIZMO codes, employing the same baryonic subgrid physics as the IllustrisTNG and SIMBA simulations, and 2,049 N-body simulations. The goal of the CAMELS project is to provide theory predictions for different observables as a function of cosmology and astrophysics, and it is the largest suite of cosmological (magneto-)hydrodynamic simulations designed to train machine learning algorithms. CAMELS contains thousands of different cosmological and astrophysical models by way of varying $Ω_m$, $σ_8$, and four parameters controlling stellar and AGN feedback, following the evolution of more than 100 billion particles and fluid elements over a combined volume of $(400~h^{-1}{\rm Mpc})^3$. We describe the simulations in detail and characterize the large range of conditions represented in terms of the matter power spectrum, cosmic star formation rate density, galaxy stellar mass function, halo baryon fractions, and several galaxy scaling relations. We show that the IllustrisTNG and SIMBA suites produce roughly similar distributions of galaxy properties over the full parameter space but significantly different halo baryon fractions and baryonic effects on the matter power spectrum. This emphasizes the need for marginalizing over baryonic effects to extract the maximum amount of information from cosmological surveys. We illustrate the unique potential of CAMELS using several machine learning applications, including non-linear interpolation, parameter estimation, symbolic regression, data generation with Generative Adversarial Networks (GANs), dimensionality reduction, and anomaly detection.

astro-ph.CO↗

arXiv2021

The Quijote simulations

The Quijote simulations are a set of 44,100 full N-body simulations spanning more than 7,000 cosmological models in the $\{Ω_{\rm m}, Ω_{\rm b}, h, n_s, σ_8, M_ν, w \}$ hyperplane. At a single redshift the simulations contain more than 8.5 trillions of particles over a combined volume of 44,100 $(h^{-1}{\rm Gpc})^3$; each simulation follow the evolution of $256^3$, $512^3$ or $1024^3$ particles in a box of $1~h^{-1}{\rm Gpc}$ length. Billions of dark matter halos and cosmic voids have been identified in the simulations, whose runs required more than 35 million core hours. The Quijote simulations have been designed for two main purposes: 1) to quantify the information content on cosmological observables, and 2) to provide enough data to train machine learning algorithms. In this paper we describe the simulations and show a few of their applications. We also release the Petabyte of data generated, comprising hundreds of thousands of simulation snapshots at multiple redshifts, halo and void catalogs, together with millions of summary statistics such as power spectra, bispectra, correlation functions, marked power spectra, and estimated probability density functions.

astro-ph.CO↗

arXiv2021

Weakly Supervised Foreground Learning for Weakly Supervised Localization and Detection

Modern deep learning models require large amounts of accurately annotated data, which is often difficult to satisfy. Hence, weakly supervised tasks, including weakly supervised object localization~(WSOL) and detection~(WSOD), have recently received attention in the computer vision community. In this paper, we motivate and propose the weakly supervised foreground learning (WSFL) task by showing that both WSOL and WSOD can be greatly improved if groundtruth foreground masks are available. More importantly, we propose a complete WSFL pipeline with low computational cost, which generates pseudo boxes, learns foreground masks, and does not need any localization annotations. With the help of foreground masks predicted by our WSFL model, we achieve 72.97% correct localization accuracy on CUB for WSOL, and 55.7% mean average precision on VOC07 for WSOD, thereby establish new state-of-the-art for both tasks. Our WSFL model also shows excellent transfer ability.

cs.CV↗

arXiv2021

Hyperspectral Remote Sensing Image Classification Based on Multi-scale Cross Graphic Convolution

The mining and utilization of features directly affect the classification performance of models used in the classification and recognition of hyperspectral remote sensing images. Traditional models usually conduct feature mining from a single perspective, with the features mined being limited and the internal relationships between them being ignored. Consequently, useful features are lost and classification results are unsatisfactory. To fully mine and utilize image features, a new multi-scale feature-mining learning algorithm (MGRNet) is proposed. The model uses principal component analysis to reduce the dimensionality of the original hyperspectral image (HSI) to retain 99.99% of its semantic information and extract dimensionality reduction features. Using a multi-scale convolution algorithm, the input dimensionality reduction features were mined to obtain shallow features, which then served as inputs into a multi-scale graph convolution algorithm to construct the internal relationships between eigenvalues at different scales. We then carried out cross fusion of multi-scale information obtained by graph convolution, before inputting the new information obtained into the residual network algorithm for deep feature mining. Finally, a flexible maximum transfer function classifier was used to predict the final features and complete the classification. Experiments on three common hyperspectral datasets showed the MGRNet algorithm proposed in this paper to be superior to traditional methods in recognition accuracy.

cs.CV↗

arXiv2021

Inferring Black Hole Properties from Astronomical Multivariate Time Series with Bayesian Attentive Neural Processes

Among the most extreme objects in the Universe, active galactic nuclei (AGN) are luminous centers of galaxies where a black hole feeds on surrounding matter. The variability patterns of the light emitted by an AGN contain information about the physical properties of the underlying black hole. Upcoming telescopes will observe over 100 million AGN in multiple broadband wavelengths, yielding a large sample of multivariate time series with long gaps and irregular sampling. We present a method that reconstructs the AGN time series and simultaneously infers the posterior probability density distribution (PDF) over the physical quantities of the black hole, including its mass and luminosity. We apply this method to a simulated dataset of 11,000 AGN and report precision and accuracy of 0.4 dex and 0.3 dex in the inferred black hole mass. This work is the first to address probabilistic time series reconstruction and parameter inference for AGN in an end-to-end fashion.

astro-ph.IM↗

arXiv2021

Cosmological simulation in tides: power spectra and halo shape responses, and shape assembly bias

The well-developed separate universe technique enables accurate calibration of the response of any observable to an isotropic long-wavelength density fluctuation. The large-scale environment also hosts tidal modes that perturb all observables anisotropically. As in the separate universe, both the long tidal and density modes can be absorbed by an effective anisotropic background, on which the interaction and evolution of the short modes change accordingly. We further develop the tidal simulation method, including proper corrections to the second order Lagrangian perturbation theory (2LPT) to generate initial conditions of the simulations. We measure the linear tidal responses of the matter power spectrum, at high redshift from our modified 2LPT, and at low redshift from the tidal simulations. Our results agree qualitatively with previous works, but exhibit quantitative differences in both cases. We also measure the linear tidal response of the halo shapes, or the shape bias, and find its universal relation with the linear halo bias, for which we provide a fitting formula. Furthermore, analogous to the assembly bias, we study the secondary dependence of the shape bias, and discover for the first time dependence on halo concentration and axis ratio. Our results provide useful insights for studies of the intrinsic alignment as source of either contamination or information. These effects need to be correctly taken into account when one uses intrinsic alignments of galaxy shapes as a precision cosmological tool.

astro-ph.CO↗

arXiv2021

AI-assisted super-resolution cosmological simulations

Cosmological simulations of galaxy formation are limited by finite computational resources. We draw from the ongoing rapid advances in Artificial Intelligence (specifically Deep Learning) to address this problem. Neural networks have been developed to learn from high-resolution (HR) image data, and then make accurate super-resolution (SR) versions of different low-resolution (LR) images. We apply such techniques to LR cosmological N-body simulations, generating SR versions. Specifically, we are able to enhance the simulation resolution by generating 512 times more particles and predicting their displacements from the initial positions. Therefore our results can be viewed as new simulation realizations themselves rather than projections, e.g., to their density fields. Furthermore, the generation process is stochastic, enabling us to sample the small-scale modes conditioning on the large-scale environment. Our model learns from only 16 pairs of small-volume LR-HR simulations, and is then able to generate SR simulations that successfully reproduce the HR matter power spectrum to percent level up to $16\,h^{-1}\mathrm{Mpc}$, and the HR halo mass function to within $10 \%$ down to $10^{11} \, M_\odot$. We successfully deploy the model in a box 1000 times larger than the training simulation box, showing that high-resolution mock surveys can be generated rapidly. We conclude that AI assistance has the potential to revolutionize modeling of small-scale galaxy formation physics in large cosmological volumes.

astro-ph.CO↗