arXiv subjects

Chen Sun

Publications and source records attributed to Chen Sun.

At least 217 records · Page 12Linked to original sources

arXiv2019

Relational Action Forecasting

This paper focuses on multi-person action forecasting in videos. More precisely, given a history of H previous frames, the goal is to detect actors and to predict their future actions for the next T frames. Our approach jointly models temporal and spatial interactions among different actors by constructing a recurrent graph, using actor proposals obtained with Faster R-CNN as nodes. Our method learns to select a subset of discriminative relations without requiring explicit supervision, thus enabling us to tackle challenging visual data. We refer to our model as Discriminative Relational Recurrent Network (DRRN). Evaluation of action prediction on AVA demonstrates the effectiveness of our proposed method compared to simpler baselines. Furthermore, we significantly improve performance on the task of early action classification on J-HMDB, from the previous SOTA of 48% to 60%.

cs.CV↗

arXiv2019

Affordance Learning In Direct Perception for Autonomous Driving

Recent development in autonomous driving involves high-level computer vision and detailed road scene understanding. Today, most autonomous vehicles are using mediated perception approach for path planning and control, which highly rely on high-definition 3D maps and real time sensors. Recent research efforts aim to substitute the massive HD maps with coarse road attributes. In this paper, we follow the direct perception based method to train a deep neural network for affordance learning in autonomous driving. Our goal in this work is to develop the affordance learning model based on freely available Google Street View panoramas and Open Street Map road vector attributes. Driving scene understanding can be achieved by learning affordances from the images captured by car-mounted cameras. Such scene understanding by learning affordances may be useful for corroborating base maps such as HD maps so that the required data storage space is minimized and available for processing in real time. We compare capability in road attribute identification between human volunteers and our model by experimental evaluation. Our results indicate that this method could act as a cheaper way for training data collection in autonomous driving. The cross validation results also indicate the effectiveness of our model.

cs.CV↗

arXiv2019

Boson Star from Repulsive Light Scalars and Gravitational Waves

We study properties of boson stars consisting of ultra-light scalar dark matter with repulsive self-interactions. We investigate the origin of the maximum mass of spherically symmetric stable stars which emerges only when solving the full equations of motion in curved space-time, but not when solving the approximated Schrödinger-Newton equations. When the repulsion is weak, the backreaction of the curvature on the scalars acts as an additional source of attraction and can overcome the repulsion, resulting in a maximum star mass and compactness. We also point out that the potential in a UV completed particle physics model of light scalar dark matter is generally more complicated than the widely used $ϕ^4$ interaction. Additional interactions beyond $ϕ^4$ in the potential can dramatically change the properties of boson stars as well as modify the prospect of LIGO gravitational wave detection for binary mergers of boson stars.

hep-ph↗

arXiv2019

Unsupervised Discovery of Parts, Structure, and Dynamics

Humans easily recognize object parts and their hierarchical structure by watching how they move; they can then predict how each part moves in the future. In this paper, we propose a novel formulation that simultaneously learns a hierarchical, disentangled object representation and a dynamics model for object parts from unlabeled videos. Our Parts, Structure, and Dynamics (PSD) model learns to, first, recognize the object parts via a layered image representation; second, predict hierarchy via a structural descriptor that composes low-level concepts into a hierarchical structure; and third, model the system dynamics by predicting the future. Experiments on multiple real and synthetic datasets demonstrate that our PSD model works well on all three tasks: segmenting object parts, building their hierarchical structure, and capturing their motion distributions.

cs.CV↗

arXiv2019

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents. Our method is based on a graph-structured variational recurrent neural network (Graph-VRNN), which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states. We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.

cs.LG↗

arXiv2019

Machine Learning Allows Calibration Models to Predict Trace Element Concentration in Soil with Generalized LIBS Spectra

Calibration models have been developed for determination of trace elements, silver for instance, in soil using laser-induced breakdown spectroscopy (LIBS). The major concern is the matrix effect. Although it affects the accuracy of LIBS measurements in a general way, the effect appears accentuated for soil because of large variation of chemical and physical properties among different soils. The purpose is to reduce its influence in such way an accurate and soil-independent calibration model can be constructed. At the same time, the developed model should efficiently reduce experimental fluctuations affecting measurement precision. A univariate model first reveals obvious influence of matrix effect and important experimental fluctuation. A multivariate model has been then developed. A key point is the introduction of generalized spectrum where variables representing the soil type are explicitly included. Machine learning has been used to develop the model. After a necessary pretreatment where a feature selection process reduces the dimension of raw spectrum accordingly to the number of available spectra, the data have been fed in to a back-propagation neuronal networks (BPNN) to train and validate the model. The resulted soilindependent calibration model allows average relative error of calibration (REC) and average relative error of prediction (REP) within the range of 5-6%.

physics.chem-ph↗

arXiv2019

D3D: Distilled 3D Networks for Video Action Recognition

State-of-the-art methods for video action recognition commonly use an ensemble of two networks: the spatial stream, which takes RGB frames as input, and the temporal stream, which takes optical flow as input. In recent work, both of these streams consist of 3D Convolutional Neural Networks, which apply spatiotemporal filters to the video clip before performing classification. Conceptually, the temporal filters should allow the spatial stream to learn motion representations, making the temporal stream redundant. However, we still see significant benefits in action recognition performance by including an entirely separate temporal stream, indicating that the spatial stream is "missing" some of the signal captured by the temporal stream. In this work, we first investigate whether motion representations are indeed missing in the spatial stream of 3D CNNs. Second, we demonstrate that these motion representations can be improved by distillation, by tuning the spatial stream to predict the outputs of the temporal stream, effectively combining both models into a single stream. Finally, we show that our Distilled 3D Network (D3D) achieves performance on par with two-stream approaches, using only a single model and with no need to compute optical flow.

cs.CV↗

arXiv2018

Composing Text and Image for Image Retrieval - An Empirical Odyssey

In this paper, we study the task of image retrieval, where the input query is specified in the form of an image plus some text that describes desired modifications to the input image. For example, we may present an image of the Eiffel tower, and ask the system to find images which are visually similar but are modified in small ways, such as being taken at nighttime instead of during the day. To tackle this task, we learn a similarity metric between a target image and a source image plus source text, an embedding and composing function such that target image feature is close to the source image plus text composition feature. We propose a new way to combine image and text using such function that is designed for the retrieval task. We show this outperforms existing approaches on 3 different datasets, namely Fashion-200k, MIT-States and a new synthetic dataset we create based on CLEVR. We also show that our approach can be used to classify input queries, in addition to image retrieval.

cs.CV↗

arXiv2018

Beam Domain Massive MIMO for Optical Wireless Communications with Transmit Lens

This paper presents a novel massive multiple-input multiple-output (MIMO) transmission in beam domain for optical wireless communications. The optical base station equipped with massive optical transmitters communicates with a number of user terminals (UTs) through a transmit lens. Focusing on LED transmitters, we analyze light refraction of the lens and establish a channel model for optical massive MIMO transmissions. For a large number of LEDs, channel vectors of different UTs become asymptotically orthogonal. We investigate the maximum ratio transmission and regularized zero-forcing precoding in the optical massive MIMO system, and propose a linear precoding design to maximize the sum rate. We further design the precoding when the number of transmitters grows asymptotically large, and show that beam division multiple access (BDMA) transmission achieves the asymptotically optimal performance for sum rate maximization. Unlike optical MIMO without a transmit lens, BDMA can increase the sum rate proportionally to $2K$ and $K$ under the total and per transmitter power constraints, respectively, where $K$ is the number of UTs. In the non-asymptotic case, we prove the orthogonality conditions of the optimal power allocation in beam domain and propose efficient beam allocation algorithms. Numerical results confirm the significantly improved performance of our proposed beam domain optical massive MIMO communication approaches.

eess.SP↗

arXiv2018

$B$-decay anomalies and scalar leptoquarks in unified Pati-Salam models from noncommutative geometry

Motivated by possible scalar-leptoquark explanations of the recently reported $B$-decay anomalies, we investigate whether the required leptoquarks can be accommodated within models based on noncommutative geometry (NCG). The models considered have the gauge structure of Pati-Salam models, $SU(4)\times SU(2)_L\times SU(2)_R$, with gauge coupling unification at a single scale. In one of the models, we find a unique scalar leptoquark with quantum numbers $(3,1,-\frac{1}{3})_{321}$, originating from a complex multiplet $(6,1,1)_{422}$, which can potentially explain the $B$-decay anomalies if its mass is on the order of a few TeV. The unification of couplings can be realized with the inclusion of a single step of intermediate symmetry breaking. The scalar leptoquark under consideration does not contribute to proton decay due to the absence of diquark couplings, as dictated by the underlying noncommutative geometry.

hep-ph↗

arXiv2018

CoCo: Compact and Optimized Consolidation of Modularized Service Function Chains in NFV

The modularization of Service Function Chains (SFCs) in Network Function Virtualization (NFV) could introduce significant performance overhead and resource efficiency degradation due to introducing frequent packet transfer and consuming much more hardware resources. In response, we exploit the lightweight and individually scalable features of elements in Modularized SFCs (MSFCs) and propose CoCo, a compact and optimized consolidation framework for MSFC in NFV. CoCo addresses the above problems in two ways. First, CoCo Optimized Placer pays attention to the problem of which elements to consolidate and provides a performance-aware placement algorithm to place MSFCs compactly and optimize the global packet transfer cost. Second, CoCo Individual Scaler innovatively introduces a push-aside scaling up strategy to avoid degrading performance and taking up new CPU cores. To support MSFC consolidation, CoCo also provides an automatic runtime scheduler to ensure fairness when elements are consolidated on CPU core. Our evaluation results show that CoCo achieves significant performance improvement and efficient resource utilization.

cs.NI↗

arXiv2018

Actor-Centric Relation Network

Current state-of-the-art approaches for spatio-temporal action localization rely on detections at the frame level and model temporal context with 3D ConvNets. Here, we go one step further and model spatio-temporal relations to capture the interactions between human actors, relevant objects and scene elements essential to differentiate similar human actions. Our approach is weakly supervised and mines the relevant elements automatically with an actor-centric relational network (ACRN). ACRN computes and accumulates pair-wise relation information from actor and global scene features, and generates relation features for action classification. It is implemented as neural networks and can be trained jointly with an existing action detection system. We show that ACRN outperforms alternative approaches which capture relation information, and that the proposed framework improves upon the state-of-the-art performance on JHMDB and AVA. A visualization of the learned relation features confirms that our approach is able to attend to the relevant relations for each action.

cs.CV↗

arXiv2018

Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification

Despite the steady progress in video analysis led by the adoption of convolutional neural networks (CNNs), the relative improvement has been less drastic as that in 2D static image classification. Three main challenges exist including spatial (image) feature representation, temporal information representation, and model/computation complexity. It was recently shown by Carreira and Zisserman that 3D CNNs, inflated from 2D networks and pretrained on ImageNet, could be a promising way for spatial and temporal representation learning. However, as for model/computation complexity, 3D CNNs are much more expensive than 2D CNNs and prone to overfit. We seek a balance between speed and accuracy by building an effective and efficient video classification system through systematic exploration of critical network design choices. In particular, we show that it is possible to replace many of the 3D convolutions by low-cost 2D convolutions. Rather surprisingly, best result (in both speed and accuracy) is achieved when replacing the 3D convolutions at the bottom of the network, suggesting that temporal representation learning on high-level semantic features is more useful. Our conclusion generalizes to datasets with very different properties. When combined with several other cost-effective designs including separable spatial/temporal convolution and feature gating, our system results in an effective video classification system that that produces very competitive results on several action classification benchmarks (Kinetics, Something-something, UCF101 and HMDB), as well as two action detection (localization) benchmarks (JHMDB and UCF101-24).

cs.CV↗

arXiv2018

PAM: When Overloaded, Push Your Neighbor Aside!

Recently SmartNICs are widely used to accelerate service chains in NFV. However, when the SmartNIC is overloaded, casually migrating vNFs away from SmartNIC to CPU may lead to additional packet transmissions between SmartNIC and CPU. To address this problem, we present PAM, push aside migration to effectively alleviate the hot spot on SmartNIC with no performance overhead. Our key novelty is to push vNFs on the border of SmartNIC and CPU aside to release resources for the bottleneck vNF. Evaluation shows that PAM could efficiently alleviate the hot spot on SmartNIC and generate a service chain with much lower latency compared with the naive solution.

cs.NI↗

arXiv2018

Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning

Transferring the knowledge learned from large scale datasets (e.g., ImageNet) via fine-tuning offers an effective solution for domain-specific fine-grained visual categorization (FGVC) tasks (e.g., recognizing bird species or car make and model). In such scenarios, data annotation often calls for specialized domain knowledge and thus is difficult to scale. In this work, we first tackle a problem in large scale FGVC. Our method won first place in iNaturalist 2017 large scale species classification challenge. Central to the success of our approach is a training scheme that uses higher image resolution and deals with the long-tailed distribution of training data. Next, we study transfer learning via fine-tuning from large scale datasets to small scale, domain-specific FGVC datasets. We propose a measure to estimate domain similarity via Earth Mover's Distance and demonstrate that transfer learning benefits from pre-training on a source domain that is similar to the target domain by this measure. Our proposed transfer learning outperforms ImageNet pre-training and obtains state-of-the-art results on multiple commonly used FGVC datasets.

cs.CV↗

arXiv2018

A large class of solvable multistate Landau-Zener models and quantum integrability

The concept of quantum integrability has been introduced recently for quantum systems with explicitly time-dependent Hamiltonians. Within the multistate Landau-Zener (MLZ) theory, however, there has been a successful alternative approach to identify and solve complex time-dependent models. Here we compare both methods by applying them to a new class of exactly solvable MLZ models. This class contains systems with an arbitrary number $N\ge 4$ of interacting states and shows a quickly growing with $N$ number of exact adiabatic energy crossing points, which appear at different moments of time. At each $N$, transition probabilities in these systems can be found analytically and exactly but complexity and variety of solutions in this class also grow with $N$ quickly. We illustrate how common features of solvable MLZ systems appear from quantum integrability and develop an approach to further classification of solvable MLZ problems.

quant-ph↗

arXiv2018

Hidden-Sector Spectroscopy with Gravitational Waves from Binary Neutron Stars

We show that neutron star binaries can be ideal laboratories to probe hidden sectors with a long range force. In particular, it is possible for gravitational wave detectors such as LIGO and Virgo to resolve the correction of waveforms from ultralight dark gauge bosons coupled to neutron stars. We observe that the interaction of the hidden sector affects both the gravitational wave frequency and amplitude in a way that cannot be fitted by pure gravity.

hep-ph↗

arXiv2018

AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions

This paper introduces a video dataset of spatio-temporally localized Atomic Visual Actions (AVA). The AVA dataset densely annotates 80 atomic visual actions in 430 15-minute video clips, where actions are localized in space and time, resulting in 1.58M action labels with multiple labels per person occurring frequently. The key characteristics of our dataset are: (1) the definition of atomic visual actions, rather than composite actions; (2) precise spatio-temporal annotations with possibly multiple annotations for each person; (3) exhaustive annotation of these atomic actions over 15-minute video clips; (4) people temporally linked across consecutive segments; and (5) using movies to gather a varied set of action representations. This departs from existing datasets for spatio-temporal action recognition, which typically provide sparse annotations for composite actions in short video clips. We will release the dataset publicly. AVA, with its realistic scene and action complexity, exposes the intrinsic difficulty of action recognition. To benchmark this, we present a novel approach for action localization that builds upon the current state-of-the-art methods, and demonstrates better performance on JHMDB and UCF101-24 categories. While setting a new state of the art on existing datasets, the overall results on AVA are low at 15.6% mAP, underscoring the need for developing new approaches for video understanding.

cs.CV↗