arXiv subjects

Lin Wang

Publications and source records attributed to Lin Wang.

At least 379 records · Page 21Linked to original sources

arXiv2021

SphereSR: 360° Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation

The 360°imaging has recently gained great attention; however, its angular resolution is relatively lower than that of a narrow field-of-view (FOV) perspective image as it is captured by using fisheye lenses with the same sensor size. Therefore, it is beneficial to super-resolve a 360°image. Some attempts have been made but mostly considered the equirectangular projection (ERP) as one of the way for 360°image representation despite of latitude-dependent distortions. In that case, as the output high-resolution(HR) image is always in the same ERP format as the low-resolution (LR) input, another information loss may occur when transforming the HR image to other projection types. In this paper, we propose SphereSR, a novel framework to generate a continuous spherical image representation from an LR 360°image, aiming at predicting the RGB values at given spherical coordinates for super-resolution with an arbitrary 360°image projection. Specifically, we first propose a feature extraction module that represents the spherical data based on icosahedron and efficiently extracts features on the spherical surface. We then propose a spherical local implicit image function (SLIIF) to predict RGB values at the spherical coordinates. As such, SphereSR flexibly reconstructs an HR image under an arbitrary projection type. Experiments on various benchmark datasets show that our method significantly surpasses existing methods.

cs.CV↗

arXiv2021

BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learning

Providing omnidirectional depth along with RGB information is important for numerous applications, eg, VR/AR. However, as omnidirectional RGB-D data is not always available, synthesizing RGB-D panorama data from limited information of a scene can be useful. Therefore, some prior works tried to synthesize RGB panorama images from perspective RGB images; however, they suffer from limited image quality and can not be directly extended for RGB-D panorama synthesis. In this paper, we study a new problem: RGB-D panorama synthesis under the arbitrary configurations of cameras and depth sensors. Accordingly, we propose a novel bi-modal (RGB-D) panorama synthesis (BIPS) framework. Especially, we focus on indoor environments where the RGB-D panorama can provide a complete 3D model for many applications. We design a generator that fuses the bi-modal information and train it with residual-aided adversarial learning (RDAL). RDAL allows to synthesize realistic indoor layout structures and interiors by jointly inferring RGB panorama, layout depth, and residual depth. In addition, as there is no tailored evaluation metric for RGB-D panorama synthesis, we propose a novel metric to effectively evaluate its perceptual quality. Extensive experiments show that our method synthesizes high-quality indoor RGB-D panoramas and provides realistic 3D indoor models than prior methods. Code will be released upon acceptance.

cs.CV↗

arXiv2021

A Case for a Programmable Edge Storage Middleware

Edge computing is a fast-growing computing paradigm where data is processed at the local site where it is generated, close to the end-devices. This can benefit a set of disruptive applications like autonomous driving, augmented reality, and collaborative machine learning, which produce incredible amounts of data that need to be shared, processed and stored at the edge to meet low latency requirements. However, edge storage poses new challenges due to the scarcity and heterogeneity of edge infrastructures and the diversity of edge applications. In particular, edge applications may impose conflicting constraints and optimizations that are hard to be reconciled on the limited, hard-to-scale edge resources. In this vision paper we argue that a new middleware for constrained edge resources is needed, providing a unified storage service for diverse edge applications. We identify programmability as a critical feature that should be leveraged to optimize the resource sharing while delivering the specialization needed for edge applications. Following this line, we make a case for eBPF and present the design for Griffin - a flexible, lightweight programmable edge storage middleware powered by eBPF.

cs.DC↗

arXiv2021

Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation

Event cameras are novel sensors that perceive the per-pixel intensity changes and output asynchronous event streams with high dynamic range and less motion blur. It has been shown that events alone can be used for end-task learning, e.g., semantic segmentation, based on encoder-decoder-like networks. However, as events are sparse and mostly reflect edge information, it is difficult to recover original details merely relying on the decoder. Moreover, most methods resort to pixel-wise loss alone for supervision, which might be insufficient to fully exploit the visual details from sparse events, thus leading to less optimal performance. In this paper, we propose a simple yet flexible two-stream framework named Dual Transfer Learning (DTL) to effectively enhance the performance on the end-tasks without adding extra inference cost. The proposed approach consists of three parts: event to end-task learning (EEL) branch, event to image translation (EIT) branch, and transfer learning (TL) module that simultaneously explores the feature-level affinity information and pixel-level knowledge from the EIT branch to improve the EEL branch. This simple yet novel method leads to strong representation learning from events and is evidenced by the significant performance boost on the end-tasks such as semantic segmentation and depth estimation.

cs.CV↗

arXiv2021

EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation

Event cameras sense per-pixel intensity changes and produce asynchronous event streams with high dynamic range and less motion blur, showing advantages over conventional cameras. A hurdle of training event-based models is the lack of large qualitative labeled data. Prior works learning end-tasks mostly rely on labeled or pseudo-labeled datasets obtained from the active pixel sensor (APS) frames; however, such datasets' quality is far from rivaling those based on the canonical images. In this paper, we propose a novel approach, called \textbf{EvDistill}, to learn a student network on the unlabeled and unpaired event data (target modality) via knowledge distillation (KD) from a teacher network trained with large-scale, labeled image data (source modality). To enable KD across the unpaired modalities, we first propose a bidirectional modality reconstruction (BMR) module to bridge both modalities and simultaneously exploit them to distill knowledge via the crafted pairs, causing no extra computation in the inference. The BMR is improved by the end-tasks and KD losses in an end-to-end manner. Second, we leverage the structural similarities of both modalities and adapt the knowledge by matching their distributions. Moreover, as most prior feature KD methods are uni-modality and less applicable to our problem, we propose to leverage an affinity graph KD loss to boost the distillation. Our extensive experiments on semantic segmentation and object recognition demonstrate that EvDistill achieves significantly better results than the prior works and KD with only events and APS frames.

cs.CV↗

arXiv2021

An Alternative Approach for Computing Discrete Logarithms in Compressed SIDH

Currently, public-key compression of supersingular isogeny Diffie-Hellman (SIDH) and its variant, supersingular isogeny key encapsulation (SIKE) involve pairing computation and discrete logarithm computation. In this paper, we propose novel methods to compute only 3 discrete logarithms instead of 4, in exchange for computing a lookup table efficiently. The algorithms also allow us to make a trade-off between memory and efficiency. Our implementation shows that the efficiency of our algorithms is close to that of the previous work, and our algorithms perform better in some special cases.

cs.CR↗

arXiv2021

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.

eess.IV↗

arXiv2021

All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda

Since the popularisation of the Internet in the 1990s, the cyberspace has kept evolving. We have created various computer-mediated virtual environments including social networks, video conferencing, virtual 3D worlds (e.g., VR Chat), augmented reality applications (e.g., Pokemon Go), and Non-Fungible Token Games (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have bought us various degrees of digital transformation. The term `metaverse' has been coined to further facilitate the digital transformation in every aspect of our physical lives. At the core of the metaverse stands the vision of an immersive Internet as a gigantic, unified, persistent, and shared realm. While the metaverse may seem futuristic, catalysed by emerging technologies such as Extended Reality, 5G, and Artificial Intelligence, the digital `big bang' of our cyberspace is not far away. This survey paper presents the first effort to offer a comprehensive framework that examines the latest metaverse development under the dimensions of state-of-the-art technologies and metaverse ecosystems, and illustrates the possibility of the digital `big bang'. First, technologies are the enablers that drive the transition from the current Internet to the metaverse. We thus examine eight enabling technologies rigorously - Extended Reality, User Interactivity (Human-Computer Interaction), Artificial Intelligence, Blockchain, Computer Vision, IoT and Robotics, Edge and Cloud computing, and Future Mobile Networks. In terms of applications, the metaverse ecosystem allows human users to live and play within a self-sustaining, persistent, and shared realm. Therefore, we discuss six user-centric factors -- Avatar, Content Creation, Virtual Economy, Social Acceptability, Security and Privacy, and Trust and Accountability. Finally, we propose a concrete research agenda for the development of the metaverse.

cs.CY↗

arXiv2021

Subsystems with shadowing property for $\mathbb{Z}^{k}$-actions

In this paper, subsystems with shadowing property for $\mathbb{Z}^{k}$-actions are investigated. Let $α$ be a continuous $\mathbb{Z}^{k}$-action on a compact metric space $X$. We introduce the notions of pseudo orbit and shadowing property for $α$ along subsets, particularly subspaces, of $\mathbb{R}^{k}$. Combining with another important property "expansiveness" for subsystems of $α$ which was introduced and systematically investigated by Boyle and Lind, we show that if $α$ has the shadowing property and is expansive along a subspace $V$ of $\mathbb{R}^{k}$, then so does for $α$ along any subspace $W$ of $\mathbb{R}^{k}$ containing $V$. Let $α$ be a smooth $\mathbb{Z}^{k}$-action on a closed Riemannian manifold $M$, $μ$ an ergodic probability measure and $Γ$ the Oseledec set. We show that, under a basic assumption on the Lyapunov spectrum, $α$ has the shadowing property and is expansive on $Γ$ along any subspace $V$ of $\mathbb{R}^{k}$ containing a regular vector; furthermore, $α$ has the quasi-shadowing property on $Γ$ along any 1-dimensional subspace $V$ of $\mathbb{R}^{k}$ containing a first-type singular vector. As an application, we also consider the 1-dimensional subsystems (i.e., flows) with shadowing property for the $\mathbb{R}^{k}$-action on the suspension manifold induced by $α$.

math.DS↗

arXiv2021

SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks

Event cameras are novel sensors that perceive the per-pixel intensity changes and output asynchronous event streams, showing lots of advantages over traditional cameras, such as high dynamic range (HDR) and no motion blur. It has been shown that events alone can be used for object tracking by motion compensation or prediction. However, existing methods assume that the target always moves and is the stand-alone object. Moreover, they fail to track the stopped non-independent moving objects on fixed scenes. In this paper, we propose a novel event-based object tracking framework, called SiamEvent, using Siamese networks via edge-aware similarity learning. Importantly, to find the part having the most similar edge structure of target, we propose to correlate the embedded events at two timestamps to compute the target edge similarity. The Siamese network enables tracking arbitrary target edge by finding the part with the highest similarity score. This extends the possibility of event-based object tracking applied not only for the independent stand-alone moving objects, but also for various settings of the camera and scenes. In addition, target edge initialization and edge detector are also proposed to prevent SiamEvent from the drifting problem. Lastly, we built an open dataset including various synthetic and real scenes to train and evaluate SiamEvent. Extensive experiments demonstrate that SiamEvent achieves up to 15% tracking performance enhancement than the baselines on the real-world scenes and more robust tracking performance in the challenging HDR and motion blur conditions.

cs.CV↗

arXiv2021

Quantitative destruction of invariant circles

For area-preserving twist maps on the annulus, we consider the problem on quantitative destruction of invariant circles with a given frequency $ω$ of an integrable system by a trigonometric polynomial of degree $N$ perturbation $R_N$ with $\|R_N\|_{C^r}<ε$. We obtain a relation among $N$, $r$, $ε$ and the arithmetic property of $ω$, for which the area-preserving map admit no invariant circles with $ω$.

math.DS↗

arXiv2021

Hamilton-Jacobi equations with their Hamiltonians depending Lipschitz continuously on the unknown

We study the Hamilton-Jacobi equations $H(x,Du,u)=0$ in $M$ and $\partial u/\partial t +H(x,D_xu,u)=0$ in $M\times(0,\infty)$, where the Hamiltonian $H=H(x,p,u)$ depends Lipschitz continuously on the variable $u$. In the framework of the semicontinuous viscosity solutions due to Barron-Jensen, we establish the comparison principle, existence theorem, and representation formula as value functions for extended real-valued, lower semicontinuous solutions for the Cauchy problem. We also establish some results on the long-time behavior of solutions for the Cauchy problem and classification of solutions for the stationary problem.

math.AP↗

arXiv2021

DQN Control Solution for KDD Cup 2021 City Brain Challenge

We took part in the city brain challenge competition and achieved the 8th place. In this competition, the players are provided with a real-world city-scale road network and its traffic demand derived from real traffic data. The players are asked to coordinate the traffic signals with a self-designed agent to maximize the number of vehicles served while maintaining an acceptable delay. In this abstract paper, we present an overall analysis and our detailed solution to this competition. Our approach is mainly based on the adaptation of the deep Q-network (DQN) for real-time traffic signal control. From our perspective, the major challenge of this competition is how to extend the classical DQN framework to traffic signals control in real-world complex road network and traffic flow situation. After trying and implementing several classical reward functions, we finally chose to apply our newly-designed reward in our agent. By applying our newly-proposed reward function and carefully tuning the control scheme, an agent based on a single DQN model can rank among the top 15 teams. We hope this paper could serve, to some extent, as a baseline solution to traffic signal control of real-world road network and inspire further attempts and researches.

cs.LG↗

arXiv2021

Unsupervised Domain Adaptation for Retinal Vessel Segmentation with Adversarial Learning and Transfer Normalization

Retinal vessel segmentation plays a key role in computer-aided screening, diagnosis, and treatment of various cardiovascular and ophthalmic diseases. Recently, deep learning-based retinal vessel segmentation algorithms have achieved remarkable performance. However, due to the domain shift problem, the performance of these algorithms often degrades when they are applied to new data that is different from the training data. Manually labeling new data for each test domain is often a time-consuming and laborious task. In this work, we explore unsupervised domain adaptation in retinal vessel segmentation by using entropy-based adversarial learning and transfer normalization layer to train a segmentation network, which generalizes well across domains and requires no annotation of the target domain. Specifically, first, an entropy-based adversarial learning strategy is developed to reduce the distribution discrepancy between the source and target domains while also achieving the objective of entropy minimization on the target domain. In addition, a new transfer normalization layer is proposed to further boost the transferability of the deep network. It normalizes the features of each domain separately to compensate for the domain distribution gap. Besides, it also adaptively selects those feature channels that are more transferable between domains, thus further enhancing the generalization performance of the network. We conducted extensive experiments on three regular fundus image datasets and an ultra-widefield fundus image dataset, and the results show that our approach yields significant performance gains compared to other state-of-the-art methods.

eess.IV↗

arXiv2021

Aubry-Mather theory for contact Hamiltonian systems II

In this paper, we continue to develop Aubry-Mather and weak KAM theories for contact Hamiltonian systems $H(x,u,p)$ with certain dependence on the contact variable $u$. For the Lipschitz dependence case, we obtain some properties of the Mañé set. For the non-decreasing case, we provide some information on the Aubry set, such as the comparison property, graph property and a partially ordered relation for the collection of all projected Aubry sets with respect to backward weak KAM solutions. Moreover, we find a new flow-invariant set $\tilde{\mathcal{S}}_s$ consists of strongly static orbits, which coincides with the Aubry set $\tilde{\mathcal{A}}$ in classical Hamiltonian systems. Nevertheless, a class of examples are constructed to show $\tilde{\mathcal{S}}_s\subsetneqq\tilde{\mathcal{A}}$ in the contact case. As their applications, we find some new phenomena appear even if the strictly increasing dependence of $H$ on $u$ fails at only one point, and we show that there is a difference for the vanishing discount problem from the negative direction between the minimal viscosity solution and non-minimal ones.

math.DS↗

arXiv2021

Prediction of high-Tc superconductivity in ternary lanthanum borohydrides

The study of superconductivity in compressed hydrides is of great interest due to measurements of high critical temperatures (Tc) in the vicinity of room temperature, beginning with the observations of LaH10 at 170-190 GPa. However, the pressures required for synthesis of these high Tc superconducting hydrides currently remain extremely high. Here we show the investigation of crystal structures and superconductivity in the La-B-H system under pressure with particle-swarm intelligence structure searches methods in combination with first-principles calculations. Structures with six stoichiometries, LaBH, LaBH3, LaBH4, LaBH6, LaBH7 and LaBH8, were predicted to become stable under pressure. Remarkably, the hydrogen atoms in LaBH8 were found to bond with B atoms in a manner that is similar to that in H3S. Lattice dynamics calculations indicate that LaBH7 and LaBH8 become dynamically stable at pressures as low as 109.2 and 48.3 GPa, respectively. Moreover, the two phases were predicted to be superconducting with a critical temperature (Tc) of 93 K and 156 K at 110 GPa and 55 GPa, respectively. Our results provide guidance for future experiments targeting new hydride superconductors with both low synthesis pressures and high Tc.

cond-mat.mtrl-sci↗

arXiv2021

Operator as a Service: Stateful Serverless Complex Event Processing

Complex Event Processing (CEP) is a powerful paradigm for scalable data management that is employed in many real-world scenarios such as detecting credit card fraud in banks. The so-called complex events are expressed using a specification language that is typically implemented and executed on a specific runtime system. While the tight coupling of these two components has been regarded as the key for supporting CEP at high performance, such dependencies pose several inherent challenges as follows. (1) Application development atop a CEP system requires extensive knowledge of how the runtime system operates, which is typically highly complex in nature. (2) The specification language dependence requires the need of domain experts and further restricts and steepens the learning curve for application developers. In this paper, we propose CEPLESS, a scalable data management system that decouples the specification from the runtime system by building on the principles of serverless computing. CEPLESS provides operator as a service and offers flexibility by enabling the development of CEP application in any specification language while abstracting away the complexity of the CEP runtime system. As part of CEPLESS, we designed and evaluated novel mechanisms for in-memory processing and batching that enables the stateful processing of CEP operators even under high rates of ingested events. Our evaluation demonstrates that CEPLESS can be easily integrated into existing CEP systems like Apache Flink while attaining similar throughput under a high scale of events (up to 100K events per second) and dynamic operator update in up to 238 ms.

cs.NI↗

arXiv2021

Multi-Granularity Network with Modal Attention for Dense Affective Understanding

Video affective understanding, which aims to predict the evoked expressions by the video content, is desired for video creation and recommendation. In the recent EEV challenge, a dense affective understanding task is proposed and requires frame-level affective prediction. In this paper, we propose a multi-granularity network with modal attention (MGN-MA), which employs multi-granularity features for better description of the target frame. Specifically, the multi-granularity features could be divided into frame-level, clips-level and video-level features, which corresponds to visual-salient content, semantic-context and video theme information. Then the modal attention fusion module is designed to fuse the multi-granularity features and emphasize more affection-relevant modals. Finally, the fused feature is fed into a Mixtures Of Experts (MOE) classifier to predict the expressions. Further employing model-ensemble post-processing, the proposed method achieves the correlation score of 0.02292 in the EEV challenge.

cs.CV↗