arXiv subjects

Yi Wang

Publications and source records attributed to Yi Wang.

At least 685 records · Page 38Linked to original sources

arXiv2022

InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges

In this report, we present our champion solutions to five tracks at Ego4D challenge. We leverage our developed InternVideo, a video foundation model, for five Ego4D tasks, including Moment Queries, Natural Language Queries, Future Hand Prediction, State Change Object Detection, and Short-term Object Interaction Anticipation. InternVideo-Ego4D is an effective paradigm to adapt the strong foundation model to the downstream ego-centric video understanding tasks with simple head designs. In these five tasks, the performance of InternVideo-Ego4D comprehensively surpasses the baseline methods and the champions of CVPR2022, demonstrating the powerful representation ability of InternVideo as a video foundation model. Our code will be released at https://github.com/OpenGVLab/ego4d-eccv2022-solutions

cs.CV↗

arXiv2022

UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer

Learning discriminative spatiotemporal representation is the key problem of video understanding. Recently, Vision Transformers (ViTs) have shown their power in learning long-term video dependency with self-attention. Unfortunately, they exhibit limitations in tackling local video redundancy, due to the blind global comparison among tokens. UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format. However, this model has to require a tiresome and complicated image-pretraining phrase, before being finetuned on videos. This blocks its wide usage in practice. On the contrary, open-sourced ViTs are readily available and well-pretrained with rich image supervision. Based on these observations, we propose a generic paradigm to build a powerful family of video networks, by arming the pretrained ViTs with efficient UniFormer designs. We call this family UniFormerV2, since it inherits the concise style of the UniFormer block. But it contains brand-new local and global relation aggregators, which allow for preferable accuracy-computation balance by seamlessly integrating advantages from both ViTs and UniFormer. Without any bells and whistles, our UniFormerV2 gets the state-of-the-art recognition performance on 8 popular video benchmarks, including scene-related Kinetics-400/600/700 and Moments in Time, temporal-related Something-Something V1/V2, untrimmed ActivityNet and HACS. In particular, it is the first model to achieve 90% top-1 accuracy on Kinetics-400, to our best knowledge. Code will be available at https://github.com/OpenGVLab/UniFormerV2.

cs.CV↗

arXiv2022

LARO: Learned Acquisition and Reconstruction Optimization to accelerate Quantitative Susceptibility Mapping

Quantitative susceptibility mapping (QSM) involves acquisition and reconstruction of a series of images at multi-echo time points to estimate tissue field, which prolongs scan time and requires specific reconstruction technique. In this paper, we present our new framework, called Learned Acquisition and Reconstruction Optimization (LARO), which aims to accelerate the multi-echo gradient echo (mGRE) pulse sequence for QSM. Our approach involves optimizing a Cartesian multi-echo k-space sampling pattern with a deep reconstruction network. Next, this optimized sampling pattern was implemented in an mGRE sequence using Cartesian fan-beam k-space segmenting and ordering for prospective scans. Furthermore, we propose to insert a recurrent temporal feature fusion module into the reconstruction network to capture signal redundancies along echo time. Our ablation studies show that both the optimized sampling pattern and proposed reconstruction strategy help improve the quality of the multi-echo image reconstructions. Generalization experiments show that LARO is robust on the test data with new pathologies and different sequence parameters. Our code is available at https://github.com/Jinwei1209/LARO.git.

eess.IV↗

arXiv2022

Numerical Simulation of Solar Magnetic Flux Emergence Using the AMR--CESE--MHD Code

Magnetic flux emergence from the solar interior to the atmosphere is believed to be a key process of formation of solar active regions and driving solar eruptions. Due to the limited capability of observation, the flux emergence process is commonly studied using numerical simulations. In this paper, we developed a numerical model to simulate the emergence of a twisted magnetic flux tube from the convection zone to the corona using the AMR--CESE--MHD code, which is based on the conservation-element solution-element method with adaptive mesh refinement. The result of our simulation agrees with that of many previous ones with similar initial conditions but using different numerical codes. In the early stage, the flux tube rises from the convection zone as driven by the magnetic buoyancy until it reaches close to the photosphere. The emergence is decelerated there and with piling-up of the magnetic flux, the magnetic buoyancy instability is triggered, which allows the magnetic field to partially enter into the atmosphere. Meanwhile, two gradually separated polarity concentration zones appear in the photospheric layer, transporting the magnetic field and energy into the atmosphere through their vortical and shearing motions. Correspondingly, the coronal magnetic field has also been reshaped to a sigmoid configuration containing a thin current layer, which resembles the typical pre-eruptive magnetic configuration of an active region. Such a numerical framework of magnetic flux emergence as established will be applied in future investigations of how solar eruptions are initiated in flux emergence active regions.

astro-ph.SR↗

arXiv2022

PalGAN: Image Colorization with Palette Generative Adversarial Networks

Multimodal ambiguity and color bleeding remain challenging in colorization. To tackle these problems, we propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention. To circumvent the multimodality issue, we present a new colorization formulation that estimates a probabilistic palette from the input gray image first, then conducts color assignment conditioned on the palette through a generative model. Further, we handle color bleeding with chromatic attention. It studies color affinities by considering both semantic and intensity correlation. In extensive experiments, PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances. With the palette design, our method enables color transfer between images even with irrelevant contexts.

cs.CV↗

arXiv2022

Nonexistence of observable chaos and its robustness in strongly monotone dynamical systems

For strongly monotone dynamical systems on a Banach space, we show that the largest Lyapunov exponent $λ_{\max}>0$ holds on a shy set in the measure-theoretic sense. This exhibits that strongly monotone dynamical systems admit no observable chaos, the notion of which was formulated by L.S. Young. We further show that such phenomenon of no observable chaos is robust under the $C^1$-perturbation of the systems.

math.DS↗

arXiv2022

Thermodynamic and electron-transport properties of Ca3Ru2O7 from first-principles phonon calculations and Boltzmann transport theory

This work demonstrates a first-principles-based approach to obtaining finite temperature thermal and electronic transport properties which can be employed to model and understand mesoscale structural evolution during electronic, magnetic, and structural phase transitions. A computationally tractable model was introduced to estimate the temperature dependence of the electron relaxation time. The model is applied to Ca3Ru2O7 with a focus on understanding its electrical resistivity across the electronic phase transition at 48 K. A quasiharmonic phonon approach to the lattice vibrations was employed to account for thermal expansion while the Boltzmann transport theory including spin-orbit coupling was used to calculate the electron-transport properties, including the temperature dependence of electrical conductivity.

cond-mat.mtrl-sci↗

arXiv2022

Helton-Howe Trace, Connes-Chern character and Quantization

We study the Helton-Howe trace and the Connes-Chern character for Toeplitz operators on weighted Bergman spaces via the idea of quantization. We prove a local formula for the large $t$-limit of the Connes-Chern character as the weight goes to infinity. And we show that the Helton-Howe trace of Toeplitz operators is independent of the weight $t$ and obtain a local formula for the Helton-Howe trace for all weighted Bergman spaces using harmonic analysis and quantization.

math.FA↗

arXiv2022

Trace Formula of Semicommutators

For weighted Bergman spaces on the unit disk, we give trace formulas of semicommutators of Toeplitz operators with $\mathscr{C}^2(\overline{\mathbb{D}})$ symbols. We generalize this formula to weighted Bergman spaces on the unit ball in higher dimensions. Applications and examples on the Hankel operators are also discussed.

math.CV↗

arXiv2022

Obstacles from interstellar matters and distortion in warp drive superluminal travel scenario

We investigate obstacles of superluminal "warp drive" travels from interactions with interstellar matter and from curvature effects. The effect of collision of interstellar dust particles and photons with the spacecraft will all lead to a pressure proportional to the apparent velocity of the spaceship $v_s$. The force exerted on the spacecraft from the curvature effect has two non-trivial components. The radial and longitudinal components scales as $v_s^2$ and $v_s^4$ respectively. The above obstacles become increasingly important when the spaceship travels at high superluminal speeds.

gr-qc↗

arXiv2022

Compositionally Complex Perovskite Oxides for Solar Thermochemical Water Splitting

Solar thermochemical hydrogen generation (STCH) is a promising approach for eco-friendly H2 production, but conventional STCH redox compounds often suffer from thermodynamic and kinetic limitations with limited tunability. Expanding from the nascent high-entropy ceramics field, this study explores a new class of compositionally complex perovskite oxides (La0.8Sr0.2)(Mn(1-x)/3Fe(1-x)/3CoxAl(1-x)/3)O3 for STCH. In situ X-ray diffraction demonstrates the phase stability during redox cycling and in situ X-ray photoelectron spectroscopy shows preferential redox of Co. The extent of reduction increases, but the intrinsic kinetics decreases, with increased Co content. Consequently, (La0.8Sr0.2)(Mn0.2Fe0.2Co0.4Al0.2)O3-δ achieves an optimal balance between the thermodynamics and kinetics properties. The combination of a moderate enthalpy of reduction, high entropy of reduction, and preferable surface oxygen exchange kinetics enables a maximum H2 yield of 395 +- 11 μmol g-1 in a short 1-hour redox duration. Entropy stabilization expectedly contributes to the structure stability during redox without phase transformation, which enables an exceptional STCH stability for >50 cycles under harsh interrupted conditions. The underlying redox mechanism is further elucidated by the density functional theory based parallel Monte Carlo computation, which represents a new computation paradigm first established here. This study suggests a new class of non-equimolar compositionally complex ceramics for STCH and chemical looping.

cond-mat.mtrl-sci↗

arXiv2022

Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?

Vision Transformers (ViTs) have proven to be effective, in solving 2D image understanding tasks by training over large-scale image datasets; and meanwhile as a somehow separate track, in modeling the 3D visual world too such as voxels or point clouds. However, with the growing hope that transformers can become the "universal" modeling tool for heterogeneous data, ViTs for 2D and 3D tasks have so far adopted vastly different architecture designs that are hardly transferable. That invites an (over-)ambitious question: can we close the gap between the 2D and 3D ViT architectures? As a piloting study, this paper demonstrates the appealing promise to understand the 3D visual world, using a standard 2D ViT architecture, with only minimal customization at the input and output levels without redesigning the pipeline. To build a 3D ViT from its 2D sibling, we "inflate" the patch embedding and token sequence, accompanied with new positional encoding mechanisms designed to match the 3D data geometry. The resultant "minimalist" 3D ViT, named Simple3D-Former, performs surprisingly robustly on popular 3D tasks such as object classification, point cloud segmentation and indoor scene detection, compared to highly customized 3D-specific designs. It can hence act as a strong baseline for new 3D ViTs. Moreover, we note that pursing a unified 2D-3D ViT design has practical relevance besides just scientific curiosity. Specifically, we demonstrate that Simple3D-Former naturally enables to exploit the wealth of pre-trained weights from large-scale realistic 2D images (e.g., ImageNet), which can be plugged in to enhancing the 3D task performance "for free".

cs.CV↗

arXiv2022

A multi view multi stage and multi window framework for pulmonary artery segmentation from CT scans

This is the technical report of the 9th place in the final result of PARSE2022 Challenge. We solve the segmentation problem of the pulmonary artery by using a two-stage method based on a 3D CNN network. The coarse model is used to locate the ROI, and the fine model is used to refine the segmentation result. In addition, in order to improve the segmentation performance, we adopt multi-view and multi-window level method, at the same time we employ a fine-tune strategy to mitigate the impact of inconsistent labeling.

eess.IV↗

arXiv2022

Reinforcement Learning of Self Enhancing Camera Image and Signal Processing

Current camera image and signal processing pipelines (ISPs), including deep-trained versions, tend to apply a single filter that is uniformly applied to the entire image. This is despite the fact that most acquired camera images have spatially heterogeneous artifacts. This spatial heterogeneity manifests itself across the image space as varied Moire ringing, motion-blur, color-bleaching, or lens-based projection distortions. Moreover, combinations of these image artifacts can be present in small or large pixel neighborhoods, within an acquired image. Here, we present a deep reinforcement learning model that works in learned latent subspaces, and recursively improves camera image quality through a patch-based spatially adaptive artifact filtering and image enhancement. Our \textit{Recursive Self Enhancement Reinforcement Learning}(RSE-RL) model views the identification and correction of artifacts as a recursive self-learning and self-improvement exercise and consists of two major sub-modules: (i) The latent feature sub-space clustering/grouping obtained through variational auto-encoders enabling rapid identification of the correspondence and discrepancy between noisy and clean image patches. (ii) The adaptive learned transformation is controlled by a soft actor-critic agent that progressively filters and enhances the noisy patches using its closest feature distance neighbors of clean patches. Artificial artifacts that may be introduced in a patch-based ISP, are also removed through a reward-based de-blocking recovery and image enhancement. We demonstrate the self-improvement feature of our model by recursively training and testing on images, wherein the enhanced images resulting from each epoch provide a natural data augmentation and robustness to the RSE-RL training-filtering pipeline. Our method shows advantage for heterogeneous noise and artifact removal.

cs.CV↗

arXiv2022

A Comprehensive Review of Deep Learning-based Single Image Super-resolution

Image super-resolution (SR) is one of the vital image processing methods that improve the resolution of an image in the field of computer vision. In the last two decades, significant progress has been made in the field of super-resolution, especially by utilizing deep learning methods. This survey is an effort to provide a detailed survey of recent progress in single-image super-resolution in the perspective of deep learning while also informing about the initial classical methods used for image super-resolution. The survey classifies the image SR methods into four categories, i.e., classical methods, supervised learning-based methods, unsupervised learning-based methods, and domain-specific SR methods. We also introduce the problem of SR to provide intuition about image quality metrics, available reference datasets, and SR challenges. Deep learning-based approaches of SR are evaluated using a reference dataset. Some of the reviewed state-of-the-art image SR methods include the enhanced deep SR network (EDSR), cycle-in-cycle GAN (CinCGAN), multiscale residual network (MSRN), meta residual dense network (Meta-RDN), recurrent back-projection network (RBPN), second-order attention network (SAN), SR feedback network (SRFBN) and the wavelet-based residual attention network (WRAN). Finally, this survey is concluded with future directions and trends in SR and open problems in SR to be addressed by the researchers.

cs.CV↗

arXiv2022

Self-supervised Learning in Remote Sensing: A Review

In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities. While there has been a big success in computer vision, most of the potential of SSL in the domain of earth observation remains locked. In this paper, we provide an introduction to, and a review of the concepts and latest developments in SSL for computer vision in the context of remote sensing. Further, we provide a preliminary benchmark of modern SSL algorithms on popular remote sensing datasets, verifying the potential of SSL in remote sensing and providing an extended study on data augmentations. Finally, we identify a list of promising directions of future research in SSL for earth observation (SSL4EO) to pave the way for fruitful interaction of both domains.

cs.CV↗

arXiv2022

Stability of quermassintegral inequalities along inverse curvature flows

In this paper, we consider the stability of quermassintegral inequalities along a inverse curvature flow. We choose a special rescaling of the flow such that the $k$-th quermassintegral is decreasing and the $k-1$-th quermassintegral is preserved. Along this rescaled flow, we prove that the decreasing rate of the $k$-th quermassintegral is faster than the Fraenkel asymmetry of the domain when approaching to the sphere. This leads to the stability inequality of quermassintegral inequalities for nearly spherical sets using the flow method.

math.DG↗

arXiv2022

Quality-Constant Per-Shot Encoding by Two-Pass Learning-based Rate Factor Prediction

Providing quality-constant streams can simultaneously guarantee user experience and prevent wasting bit-rate. In this paper, we propose a novel deep learning based two-pass encoder parameter prediction framework to decide rate factor (RF), with which encoder can output streams with constant quality. For each one-shot segment in a video, the proposed method firstly extracts spatial, temporal and pre-coding features by an ultra fast pre-process. Based on these features, a RF parameter is predicted by a deep neural network. Video encoder uses the RF to compress segment as the first encoding pass. Then VMAF quality of the first pass encoding is measured. If the quality doesn't meet target, a second pass RF prediction and encoding will be performed. With the help of first pass predicted RF and corresponding actual quality as feedback, the second pass prediction will be highly accurate. Experiments show the proposed method requires only 1.55 times encoding complexity on average, meanwhile the accuracy, that the compressed video's actual VMAF is within $\pm1$ around the target VMAF, reaches 98.88%.

cs.CV↗