arXiv subjects

He Wang

Publications and source records attributed to He Wang.

At least 379 records · Page 21Linked to original sources

arXiv2021

Enhanced 3D Human Pose Estimation from Videos by using Attention-Based Neural Network with Dilated Convolutions

The attention mechanism provides a sequential prediction framework for learning spatial models with enhanced implicit temporal consistency. In this work, we show a systematic design (from 2D to 3D) for how conventional networks and other forms of constraints can be incorporated into the attention framework for learning long-range dependencies for the task of pose estimation. The contribution of this paper is to provide a systematic approach for designing and training of attention-based models for the end-to-end pose estimation, with the flexibility and scalability of arbitrary video sequences as input. We achieve this by adapting temporal receptive field via a multi-scale structure of dilated convolutions. Besides, the proposed architecture can be easily adapted to a causal model enabling real-time performance. Any off-the-shelf 2D pose estimation systems, e.g. Mocap libraries, can be easily integrated in an ad-hoc fashion. Our method achieves the state-of-the-art performance and outperforms existing methods by reducing the mean per joint position error to 33.4 mm on Human3.6M dataset.

cs.CV↗

arXiv2021

Distributed Optimization with Coupling Constraints

In this paper, we develop a novel distributed algorithm for addressing convex optimization with both nonlinear inequality and linear equality constraints, where the objective function can be a general nonsmooth convex function and all the constraints can be fully coupled. Specifically, we first separate the constraints into three groups, and design two primal-dual methods and utilize a virtual-queue-based method to handle each group of the constraints independently. Then, we integrate these three methods in a strategic way, leading to an integrated primal-dual proximal (IPLUX) algorithm, and enable the distributed implementation of IPLUX. We show that IPLUX achieves an $O(1/k)$ rate of convergence in terms of optimality and feasibility, which is stronger than the convergence results of the state-of-the-art distributed algorithms for convex optimization with coupling nonlinear constraints. Finally, IPLUX exhibits competitive practical performance in the simulations.

math.OC↗

arXiv2021

In-game Residential Home Planning via Visual Context-aware Global Relation Learning

In this paper, we propose an effective global relation learning algorithm to recommend an appropriate location of a building unit for in-game customization of residential home complex. Given a construction layout, we propose a visual context-aware graph generation network that learns the implicit global relations among the scene components and infers the location of a new building unit. The proposed network takes as input the scene graph and the corresponding top-view depth image. It provides the location recommendations for a newly-added building units by learning an auto-regressive edge distribution conditioned on existing scenes. We also introduce a global graph-image matching loss to enhance the awareness of essential geometry semantics of the site. Qualitative and quantitative experiments demonstrate that the recommended location well reflects the implicit spatial rules of components in the residential estates, and it is instructive and practical to locate the building units in the 3D scene of the complex construction.

cs.CV↗

arXiv2021

High-order Differentiable Autoencoder for Nonlinear Model Reduction

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton's method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., $10+$ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable \emph{weight-varying} Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.

cs.LG↗

arXiv2021

Systematic electrochemical etching of various metal tips for tunneling spectroscopy and scanning probe microscopy

Hard point-contact spectroscopy and scanning probe microscopy/spectroscopy are powerful techniques for investigating materials with strong expandability. To support these studies, tips with various physical and chemical properties are required. To ensure the reproducibility of experimental results, the fabrication of tips should be standardized, and a controllable and convenient system should be set up. Here a systematic methodology to fabricate various tips is proposed, involving electrochemical etching reactions. The reaction parameters fall into four categories: solution, power supply, immersion depth, and interruption. An etching system was designed and built so that these parameters could be accurately controlled. With this system, etching parameters for copper, silver, gold, platinum/iridium alloy, tungsten, lead, niobium, iron, nickel, cobalt, and permalloy were explored and standardized. Among these tips, silver and niobium's new recipes were explored and standardized. Optical and scanning electron microscopies were performed to characterize the sharp needles. Relevant point-contact experiments were carried out with an etched silver tip to confirm the suitability of the fabricated tips.

cond-mat.mtrl-sci↗

arXiv2020

Detection of magnetic gap in the topological surface states of MnBi2Te4

Recently, intrinsic antiferromagnetic topological insulator MnBi2Te4 has drawn intense research interest and leads to plenty of significant progress in physics and materials science by hosting quantum anomalous Hall effect, axion insulator state, and other quantum phases. An essential ingredient to realize these quantum states is the magnetic gap in the topological surface states induced by the out-of-plane ferromagnetism on the surface of MnBi2Te4. However, the experimental observations of the surface gap remain controversial. Here, we report the observation of the surface gap via the point contact tunneling spectroscopy. In agreement with theoretical calculations, the gap size is around 50 meV, which vanishes as the sample becomes paramagnetic with increasing temperature. The magnetoresistance hysteresis is detected through the point contact junction on the sample surface with an out-of-plane magnetic field, substantiating the surface ferromagnetism. Furthermore, the non-zero transport spin polarization coming from the ferromagnetism is determined by the point contact Andreev reflection spectroscopy. Combining these results, the magnetism-induced gap in topological surface states of MnBi2Te4 is revealed.

cond-mat.mtrl-sci↗

arXiv2020

Constant Regret Re-solving Heuristics for Price-based Revenue Management

Price-based revenue management is an important problem in operations management with many practical applications. The problem considers a retailer who sells a product (or multiple products) over $T$ consecutive time periods and is subject to constraints on the initial inventory levels. While the optimal pricing policy could be obtained via dynamic programming, such an approach is sometimes undesirable because of high computational costs. Approximate policies, such as the re-solving heuristics, are often applied as computationally tractable alternatives. In this paper, we show the following two results. First, we prove that a natural re-solving heuristic attains $O(1)$ regret compared to the value of the optimal policy. This improves the $O(\ln T)$ regret upper bound established in the prior work of \cite{jasin2014reoptimization}. Second, we prove that there is an $Ω(\ln T)$ gap between the value of the optimal policy and that of the fluid model. This complements our upper bound result by showing that the fluid is not an adequate information-relaxed benchmark when analyzing price-based revenue management algorithms.

math.OC↗

arXiv2020

Dynamic Future Net: Diversified Human Motion Generation

Human motion modelling is crucial in many areas such as computer graphics, vision and virtual reality. Acquiring high-quality skeletal motions is difficult due to the need for specialized equipment and laborious manual post-posting, which necessitates maximizing the use of existing data to synthesize new data. However, it is a challenge due to the intrinsic motion stochasticity of human motion dynamics, manifested in the short and long terms. In the short term, there is strong randomness within a couple frames, e.g. one frame followed by multiple possible frames leading to different motion styles; while in the long term, there are non-deterministic action transitions. In this paper, we present Dynamic Future Net, a new deep learning model where we explicitly focuses on the aforementioned motion stochasticity by constructing a generative model with non-trivial modelling capacity in temporal stochasticity. Given limited amounts of data, our model can generate a large number of high-quality motions with arbitrary duration, and visually-convincing variations in both space and time. We evaluate our model on a wide range of motions and compare it with the state-of-the-art methods. Both qualitative and quantitative results show the superiority of our method, for its robustness, versatility and high-quality.

cs.CV↗

arXiv2020

The nonequilibrium back-reaction of Hawking radiation to a Schwarzschild black hole

We investigate the nonequilibrium back reaction on the Schwarzschild black hole from the radiation field. The back reactions are characterized by the membrane close to the black hole. When the membrane is thin, we found that larger temperature difference can lead to more significant negative surface tension, larger thermodynamic dissipation cost and back reaction in energy and entropy as well as larger black hole area. This may be relevant to the primordial black holes in early universe. Moreover, our nonequilibrium model can resolve the inconsistency issue of the black hole back reaction under zero mass limit in the equilibrium case. In the thick membrane case, the nonequilibrium back reaction is found to be more significant than that in the thin membrane case. The nonequilibrium temperature difference can increase the energy and entropy loss as well as the thermodynamic dissipation of the black hole and the membrane back reactions. The nonequilibrium dissipation cost characterized by the entropy production rate appears to be significant compared to the entropy rate radiated by the black hole under finite temperature difference. This may shed light on the black hole information paradox due to the information loss from the entropy production rate in the nonequilibirum cases. The nonequilibrium thermodynamic fluctuations can also reflect the effects of the back-reactions of the Hawking radiation on the evolution of a black hole.

hep-th↗

arXiv2020

FASTSWARM: A Data-driven FrAmework for Real-time Flying InSecT SWARM Simulation

Insect swarms are common phenomena in nature and therefore have been actively pursued in computer animation. Realistic insect swarm simulation is difficult due to two challenges: high-fidelity behaviors and large scales, which make the simulation practice subject to laborious manual work and excessive trial-and-error processes. To address both challenges, we present a novel data-driven framework, FASTSWARM, to model complex behaviors of flying insects based on real-world data and simulate plausible animations of flying insect swarms. FASTSWARM has a linear time complexity and achieves real-time performance for large swarms. The high-fidelity behavior model of FASTSWARM explicitly takes into consideration the most common behaviors of flying insects, including the interactions among insects such as repulsion and attraction, the self-propelled behaviors such as target following and obstacle avoidance, and other characteristics such as the random movements. To achieve scalability, an energy minimization problem is formed with different behaviors modelled as energy terms, where the minimizer is the desired behavior. The minimizer is computed from the real-world data, which ensures the plausibility of the simulation results. Extensive simulation results and evaluations show that FASTSWARM is versatile in simulating various swarm behaviors, high fidelity measured by various metrics, easily controllable in inducing user controls and highly scalable.

cs.GR↗

arXiv2020

Object-Centric Multi-View Aggregation

We present an approach for aggregating a sparse set of views of an object in order to compute a semi-implicit 3D representation in the form of a volumetric feature grid. Key to our approach is an object-centric canonical 3D coordinate system into which views can be lifted, without explicit camera pose estimation, and then combined -- in a manner that can accommodate a variable number of views and is view order independent. We show that computing a symmetry-aware mapping from pixels to the canonical coordinate system allows us to better propagate information to unseen regions, as well as to robustly overcome pose ambiguities during inference. Our aggregate representation enables us to perform 3D inference tasks like volumetric reconstruction and novel view synthesis, and we use these tasks to demonstrate the benefits of our aggregation approach as compared to implicit or camera-centric alternatives.

cs.CV↗

arXiv2020

Curriculum DeepSDF

When learning to sketch, beginners start with simple and flexible shapes, and then gradually strive for more complex and accurate ones in the subsequent training sessions. In this paper, we design a "shape curriculum" for learning continuous Signed Distance Function (SDF) on shapes, namely Curriculum DeepSDF. Inspired by how humans learn, Curriculum DeepSDF organizes the learning task in ascending order of difficulty according to the following two criteria: surface accuracy and sample difficulty. The former considers stringency in supervising with ground truth, while the latter regards the weights of hard training samples near complex geometry and fine structure. More specifically, Curriculum DeepSDF learns to reconstruct coarse shapes at first, and then gradually increases the accuracy and focuses more on complex local details. Experimental results show that a carefully-designed curriculum leads to significantly better shape reconstructions with the same training data, training epochs and network architecture as DeepSDF. We believe that the application of shape curricula can benefit the training process of a wide variety of 3D shape representation learning methods.

cs.CV↗

arXiv2020

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

3D generative shape modeling is a fundamental research area in computer vision and interactive computer graphics, with many real-world applications. This paper investigates the novel problem of generating 3D shape point cloud geometry from a symbolic part tree representation. In order to learn such a conditional shape generation procedure in an end-to-end fashion, we propose a conditional GAN "part tree"-to-"point cloud" model (PT2PC) that disentangles the structural and geometric factors. The proposed model incorporates the part tree condition into the architecture design by passing messages top-down and bottom-up along the part tree hierarchy. Experimental results and user study demonstrate the strengths of our method in generating perceptually plausible and diverse 3D point clouds, given the part tree condition. We also propose a novel structural measure for evaluating if the generated shape point clouds satisfy the part tree conditions.

cs.CV↗

arXiv2020

Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks

In this paper, we examine the long-neglected yet important effects of point sampling patterns in point cloud GANs. Through extensive experiments, we show that sampling-insensitive discriminators (e.g.PointNet-Max) produce shape point clouds with point clustering artifacts while sampling-oversensitive discriminators (e.g.PointNet++, DGCNN) fail to guide valid shape generation. We propose the concept of sampling spectrum to depict the different sampling sensitivities of discriminators. We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics. Guided by the proposed sampling spectrum, we discover a middle-point sampling-aware baseline discriminator, PointNet-Mix, which improves all existing point cloud generators by a large margin on sampling-related metrics. We point out that, though recent research has been focused on the generator design, the main bottleneck of point cloud GAN actually lies in the discriminator design. Our work provides both suggestions and tools for building future discriminators. We will release the code to facilitate future research.

cs.CV↗

arXiv2020

Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance

Crowd simulation is a central topic in several fields including graphics. To achieve high-fidelity simulations, data has been increasingly relied upon for analysis and simulation guidance. However, the information in real-world data is often noisy, mixed and unstructured, making it difficult for effective analysis, therefore has not been fully utilized. With the fast-growing volume of crowd data, such a bottleneck needs to be addressed. In this paper, we propose a new framework which comprehensively tackles this problem. It centers at an unsupervised method for analysis. The method takes as input raw and noisy data with highly mixed multi-dimensional (space, time and dynamics) information, and automatically structure it by learning the correlations among these dimensions. The dimensions together with their correlations fully describe the scene semantics which consists of recurring activity patterns in a scene, manifested as space flows with temporal and dynamics profiles. The effectiveness and robustness of the analysis have been tested on datasets with great variations in volume, duration, environment and crowd dynamics. Based on the analysis, new methods for data visualization, simulation evaluation and simulation guidance are also proposed. Together, our framework establishes a highly automated pipeline from raw data to crowd analysis, comparison and simulation guidance. Extensive experiments and evaluations have been conducted to show the flexibility, versatility and intuitiveness of our framework.

cs.GR↗

arXiv2020

MeshingNet: A New Mesh Generation Method based on Deep Learning

We introduce a novel approach to automatic unstructured mesh generation using machine learning to predict an optimal finite element mesh for a previously unseen problem. The framework that we have developed is based around training an artificial neural network (ANN) to guide standard mesh generation software, based upon a prediction of the required local mesh density throughout the domain. We describe the training regime that is proposed, based upon the use of \emph{a posteriori} error estimation, and discuss the topologies of the ANNs that we have considered. We then illustrate performance using two standard test problems, a single elliptic partial differential equation (PDE) and a system of PDEs associated with linear elasticity. We demonstrate the effective generation of high quality meshes for arbitrary polygonal geometries and a range of material parameters, using a variety of user-selected error norms.

math.NA↗

arXiv2020

Category-Level Articulated Object Pose Estimation

This project addresses the task of category-level pose estimation for articulated objects from a single depth image. We present a novel category-level approach that correctly accommodates object instances previously unseen during training. We introduce Articulation-aware Normalized Coordinate Space Hierarchy (ANCSH) - a canonical representation for different articulated objects in a given category. As the key to achieve intra-category generalization, the representation constructs a canonical object space as well as a set of canonical part spaces. The canonical object space normalizes the object orientation,scales and articulations (e.g. joint parameters and states) while each canonical part space further normalizes its part pose and scale. We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space. By leveraging the canonicalized joints, we demonstrate: 1) improved performance in part pose and scale estimations using the induced kinematic constraints from joints; 2) high accuracy for joint parameter estimation in camera space.

cs.CV↗

arXiv2020

The Online Saddle Point Problem and Online Convex Optimization with Knapsacks

We study the online saddle point problem, an online learning problem where at each iteration a pair of actions need to be chosen without knowledge of the current and future (convex-concave) payoff functions. The objective is to minimize the gap between the cumulative payoffs and the saddle point value of the aggregate payoff function, which we measure using a metric called "SP-Regret". The problem generalizes the online convex optimization framework but here we must ensure both players incur cumulative payoffs close to that of the Nash equilibrium of the sum of the games. We propose an algorithm that achieves SP-Regret proportional to $\sqrt{\ln(T)T}$ in the general case, and $\log(T)$ SP-Regret for the strongly convex-concave case. We also consider the special case where the payoff functions are bilinear and the decision sets are the probability simplex. In this setting we are able to design algorithms that reduce the bounds on SP-Regret from a linear dependence in the dimension of the problem to a \textit{logarithmic} one. We also study the problem under bandit feedback and provide an algorithm that achieves sublinear SP-Regret. We then consider an online convex optimization with knapsacks problem motivated by a wide variety of applications such as: dynamic pricing, auctions, and crowdsourcing. We relate this problem to the online saddle point problem and establish $O(\sqrt{T})$ regret using a primal-dual algorithm.

stat.ML↗