arXiv subjects

Hao Sun

Publications and source records attributed to Hao Sun.

At least 397 records · Page 22Linked to original sources

arXiv2023

Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search

Nonlinear dynamics is ubiquitous in nature and commonly seen in various science and engineering disciplines. Distilling analytical expressions that govern nonlinear dynamics from limited data remains vital but challenging. To tackle this fundamental issue, we propose a novel Symbolic Physics Learner (SPL) machine to discover the mathematical structure of nonlinear dynamics. The key concept is to interpret mathematical operations and system state variables by computational rules and symbols, establish symbolic reasoning of mathematical formulas via expression trees, and employ a Monte Carlo tree search (MCTS) agent to explore optimal expression trees based on measurement data. The MCTS agent obtains an optimistic selection policy through the traversal of expression trees, featuring the one that maps to the arithmetic expression of underlying physics. Salient features of the proposed framework include search flexibility and enforcement of parsimony for discovered equations. The efficacy and superiority of the SPL machine are demonstrated by numerical examples, compared with state-of-the-art baselines.

cs.AI↗

arXiv2023

Accurate Gaze Estimation using an Active-gaze Morphable Model

Rather than regressing gaze direction directly from images, we show that adding a 3D shape model can: i) improve gaze estimation accuracy, ii) perform well with lower resolution inputs and iii) provide a richer understanding of the eye-region and its constituent gaze system. Specifically, we use an `eyes and nose' 3D morphable model (3DMM) to capture the eye-region 3D facial geometry and appearance and we equip this with a geometric vergence model of gaze to give an `active-gaze 3DMM'. We show that our approach achieves state-of-the-art results on the Eyediap dataset and we present an ablation study. Our method can learn with only the ground truth gaze target point and the camera parameters, without access to the ground truth gaze origin points, thus widening the applicability of our approach compared to other methods.

cs.CV↗

arXiv2022

Propagation Map Reconstruction via Interpolation Assisted Matrix Completion

Constructing a propagation map from a set of scattered measurements finds important applications in many areas, such as localization, spectrum monitoring and management. Classical interpolation-type methods have poor performance in regions with very sparse measurements. Recent advance in matrix completion has the potential to reconstruct a propagation map from sparse measurements, but the spatial resolution is limited. This paper proposes to integrate interpolation with matrix completion to exploit both the spatial correlation and the potential low rank structure of the propagation map. The proposed method first enriches matrix observations using interpolation, and develops the statistics of the interpolation error based on a local polynomial regression model. Then, two uncertainty-aware matrix completion algorithms are developed to exploit the interpolation error statistics. It is numerically demonstrated that the proposed method outperforms Kriging and other state-of-the-art schemes, and reduces the mean squared error (MSE) of propagation map reconstruction by 10%-50% for a medium to large number of measurements.

eess.SP↗

arXiv2022

SemanticBEVFusion: Rethink LiDAR-Camera Fusion in Unified Bird's-Eye View Representation for 3D Object Detection

LiDAR and camera are two essential sensors for 3D object detection in autonomous driving. LiDAR provides accurate and reliable 3D geometry information while the camera provides rich texture with color. Despite the increasing popularity of fusing these two complementary sensors, the challenge remains in how to effectively fuse 3D LiDAR point cloud with 2D camera images. Recent methods focus on point-level fusion which paints the LiDAR point cloud with camera features in the perspective view or bird's-eye view (BEV)-level fusion which unifies multi-modality features in the BEV representation. In this paper, we rethink these previous fusion strategies and analyze their information loss and influences on geometric and semantic features. We present SemanticBEVFusion to deeply fuse camera features with LiDAR features in a unified BEV representation while maintaining per-modality strengths for 3D object detection. Our method achieves state-of-the-art performance on the large-scale nuScenes dataset, especially for challenging distant objects. The code will be made publicly available.

cs.CV↗

arXiv2022

Complete NLO Operators in the Higgs Effective Field Theory

We enumerate the complete and independent sets of operators at the next-to-leading order (NLO) in the Higgs effective field theory (HEFT), based on the Young tensor technique on the Lorentz, gauge and flavor structures. The operator-amplitude correspondence tells a type of operators forms the on-shell amplitude basis, and for operators involving in Nambu-Goldstone bosons, the amplitude basis is further reduced to the subspace satisfying the Adler's zero condition in the soft momentum limit. Different from dynamical field, the spurion should not enter into the Lorentz sector, instead it only plays a role of forming the $SU(2)$ invariant together with other dynamical fields. With these new treatments, for the first time we could obtain the 224 (7704) operators for one (three) generation fermions, 295 (11307) with right-handed neutrinos, and find there were 6 (9) terms of operators missing and many redundant operators can be removed in the effective theory without (with) right-handed neutrinos.

hep-ph↗

arXiv2022

Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation

Large pretrained language models can easily produce toxic or biased content, which is prohibitive for practical use. In order to detect such toxic generations, existing methods rely on templates, real-world data extraction, crowdsourcing workers, or automatic generation to construct adversarial contexts that are likely to induce toxic generations. However, what type of context is more likely to induce unsafe responses is still under-explored. In this paper, we identify that context toxicity and context category (e.g., \textit{profanity}, \textit{insult}, \textit{drugs}, etc.) are two important factors to cause safety issues in response generation. Hence, we propose a method called \emph{reverse generation} to construct adversarial contexts conditioned on a given response, with the flexibility to control category, toxicity level, and inductivity of the generated contexts. Via reverse generation, we augment the existing BAD dataset and construct a new dataset BAD+ which contains more than 120K diverse and highly inductive contexts in 12 categories. We test three popular pretrained dialogue models (Blender, DialoGPT, and Plato2) and find that BAD+ can largely expose their safety problems. Furthermore, we show that BAD+ can greatly enhance the safety of generation and reveal the key factors of safety improvement. Our code and dataset is available at \url{https://github.com/thu-coai/Reverse_Generation}.

cs.CL↗

arXiv2022

Hilbert Series and Operator Counting on the Higgs Effective Field Theory

We present a systematic procedure for determining the Hilbert series that counts the number of independent operators in the Higgs effective field theory. After removing the redundancies from equation-of-motion and integration-by-part, we provide an algorithm of treating the redundancy from the operators involving in spurion fields parametrizing the custodial symmetry breaking. Furthermore, we utilize the outer automorphism of the Lorentz and internal symmetries to separate operators with different CP properties. With these new implements, the Hilbert series up to chiral dimension 10 are obtained, and CP-even and CP-odd operators can be further divided. Extensions to all orders are straightforward.

hep-ph↗

arXiv2022

Scalar dark matter with $Z_3$ symmetry in Type-II Seesaw

We study a simple complex scalar singlet dark matter (DM) model with $Z_3$ symmetry in the framework of type-II seesaw mechanism. We use the model to explain the excess of electron-positron flux measured by AMS-02, DAMPE and Fermi-LAT collaborations, which is encouraged by the decay of the triplets produced from dark matter annihilations in the Galactic halo. We focus on the non-degenerate case in which the mass of DM is larger than that of the triplets' and deliberately alleviate the leptophilic properties of the DM, so that the semi-annihilation effects are enhanced. With the guarantee of $Z_3$ symmetry, by fitting the antiproton spectrum observed in PAMELA and AMS experiments, we find that the DM cubic terms and the couplings between DM and Higgs are strongly constrained, leading to the semi-annihilation cross section fraction less than $3\%$ when DM mass is given at 3 TeV.

hep-ph↗

arXiv2022

SeismicNet: Physics-informed neural networks for seismic wave modeling in semi-infinite domain

There has been an increasing interest in integrating physics knowledge and machine learning for modeling dynamical systems. However, very limited studies have been conducted on seismic wave modeling tasks. A critical challenge is that these geophysical problems are typically defined in large domains (i.e., semi-infinite), which leads to high computational cost. In this paper, we present a novel physics-informed neural network (PINN) model for seismic wave modeling in semi-infinite domain without the nedd of labeled data. In specific, the absorbing boundary condition is introduced into the network as a soft regularizer for handling truncated boundaries. In terms of computational efficiency, we consider a sequential training strategy via temporal domain decomposition to improve the scalability of the network and solution accuracy. Moreover, we design a novel surrogate modeling strategy for parametric loading, which estimates the wave propagation in semin-infinite domain given the seismic loading at different locations. Various numerical experiments have been implemented to evaluate the performance of the proposed PINN model in the context of forward modeling of seismic wave propagation. In particular, we define diverse material distributions to test the versatility of this approach. The results demonstrate excellent solution accuracy under distinctive scenarios.

physics.geo-ph↗

arXiv2022

Novel Policy Seeking with Constrained Optimization

In problem-solving, we humans can come up with multiple novel solutions to the same problem. However, reinforcement learning algorithms can only produce a set of monotonous policies that maximize the cumulative reward but lack diversity and novelty. In this work, we address the problem of generating novel policies in reinforcement learning tasks. Instead of following the multi-objective framework used in existing methods, we propose to rethink the problem under a novel perspective of constrained optimization. We first introduce a new metric to evaluate the difference between policies and then design two practical novel policy generation methods following the new perspective. The two proposed methods, namely the Constrained Task Novel Bisector (CTNB) and the Interior Policy Differentiation (IPD), are derived from the feasible direction method and the interior point method commonly known in the constrained optimization literature. Experimental comparisons on the MuJoCo control suite show our methods can achieve substantial improvement over previous novelty-seeking methods in terms of both the novelty of policies and their performances in the primal task.

cs.LG↗

arXiv2022

Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference

Although it is well known that exploration plays a key role in Reinforcement Learning (RL), prevailing exploration strategies for continuous control tasks in RL are mainly based on naive isotropic Gaussian noise regardless of the causality relationship between action space and the task and consider all dimensions of actions equally important. In this work, we propose to conduct interventions on the primal action space to discover the causal relationship between the action space and the task reward. We propose the method of State-Wise Action Refined (SWAR), which addresses the issue of action space redundancy and promote causality discovery in RL. We formulate causality discovery in RL tasks as a state-dependent action space selection problem and propose two practical algorithms as solutions. The first approach, TD-SWAR, detects task-related actions during temporal difference learning, while the second approach, Dyn-SWAR, reveals important actions through dynamic model prediction. Empirically, both methods provide approaches to understand the decisions made by RL agents and improve learning efficiency in action-redundant tasks.

cs.LG↗

arXiv2022

Complete NNLO Operator Bases in Higgs Effective Field Theory

For the first time, we list the complete and independent set of operators at the next-to-next-to-leading order (NNLO) in the Higgs effective field theory (HEFT), also named as the electroweak chiral Lagrangian with the light Higgs boson. The Young tensor technique utilized in this work guarantees the completeness and independence of the on-shell amplitude basis while the Adler zero condition imposes non-linear symmetry on the Nambu-Goldstone bosons that play the central role in the chiral Lagrangian. The spurion fields are incorporated into the gauge structure of operators following the Littlewood-Richardson rule to accommodate custodial symmetry breaking. We enumerate 11506 (1927574) NNLO operators for one (three) flavor of fermions for the electroweak chiral Lagrangian with the light Higgs. Below the electroweak symmetry breaking scale, the dimension-8 standard model effective field theory (SMEFT) operators could be matched to these HEFT operators at both the NLO and NNLO orders.

hep-ph↗

arXiv2022

COLD: A Benchmark for Chinese Offensive Language Detection

Offensive language detection is increasingly crucial for maintaining a civilized social media platform and deploying pre-trained language models. However, this task in Chinese is still under exploration due to the scarcity of reliable datasets. To this end, we propose a benchmark --COLD for Chinese offensive language analysis, including a Chinese Offensive Language Dataset --COLDATASET and a baseline detector --COLDETECTOR which is trained on the dataset. We show that the COLD benchmark contributes to Chinese offensive language detection which is challenging for existing resources. We then deploy the COLDETECTOR and conduct detailed analyses on popular Chinese pre-trained language models. We first analyze the offensiveness of existing generative models and show that these models inevitably expose varying degrees of offensive issues. Furthermore, we investigate the factors that influence the offensive generations, and we find that anti-bias contents and keywords referring to certain groups or revealing negative attitudes trigger offensive outputs easier.

cs.CL↗

arXiv2022

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping

In this work, we study the simple yet universally applicable case of reward shaping in value-based Deep Reinforcement Learning (DRL). We show that reward shifting in the form of the linear transformation is equivalent to changing the initialization of the $Q$-function in function approximation. Based on such an equivalence, we bring the key insight that a positive reward shifting leads to conservative exploitation, while a negative reward shifting leads to curiosity-driven exploration. Accordingly, conservative exploitation improves offline RL value estimation, and optimistic value estimation improves exploration for online RL. We validate our insight on a range of RL tasks and show its improvement over baselines: (1) In offline RL, the conservative exploitation leads to improved performance based on off-the-shelf algorithms; (2) In online continuous control, multiple value functions with different shifting constants can be used to tackle the exploration-exploitation dilemma for better sample efficiency; (3) In discrete control tasks, a negative reward shifting yields an improvement over the curiosity-based exploration method.

cs.LG↗

arXiv2022

Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty

Nonlinear dynamics are ubiquitous in science and engineering applications, but the physics of most complex systems is far from being fully understood. Discovering interpretable governing equations from measurement data can help us understand and predict the behavior of complex dynamic systems. Although extensive work has recently been done in this field, robustly distilling explicit model forms from very sparse data with considerable noise remains intractable. Moreover, quantifying and propagating the uncertainty of the identified system from noisy data is challenging, and relevant literature is still limited. To bridge this gap, we develop a novel Bayesian spline learning framework to identify parsimonious governing equations of nonlinear (spatio)temporal dynamics from sparse, noisy data with quantified uncertainty. The proposed method utilizes spline basis to handle the data scarcity and measurement noise, upon which a group of derivatives can be accurately computed to form a library of candidate model terms. The equation residuals are used to inform the spline learning in a Bayesian manner, where approximate Bayesian uncertainty calibration techniques are employed to approximate posterior distributions of the trainable parameters. To promote the sparsity, an iterative sequential-threshold Bayesian learning approach is developed, using the alternative direction optimization strategy to systematically approximate L0 sparsity constraints. The proposed algorithm is evaluated on multiple nonlinear dynamical systems governed by canonical ordinary and partial differential equations, and the merit/superiority of the proposed method is demonstrated by comparison with state-of-the-art methods.

cs.LG↗

arXiv2022

The strong coupling $g_{X J/ψϕ}$ of $X(4700) \to J/ψϕ$ in the light-cone sum rules

We assign the scalar tetraquark and the D-wave tetraquark state for $X(4700)$ and calculate the width of the decay $X(4700)$ $\to J/ψϕ$ within the framework of light-cone sum rules. The strong coupling $g_{X J/ψϕ}$ is obtained by considering the technique of soft-meson approximation. We also investigate the mass and the decay constant of $X(4700)$ in the framework of SVZ sum rules. Our prediction for the mass is in agreement with the experimental measurement, and that for the decay width of $X(4700)$ $\to J/ψϕ$ support the possibility that $X(4700)$ could be a scalar tetraquark state if $X(4700)$ $\to J/ψϕ$ is the predominant decay channel, or a D-wave tetraquark state if $X(4700)$ $\to J/ψϕ$ is not the predominant one and there exist other decays.

hep-ph↗

arXiv2022

CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

Multimodal sentiment analysis and depression estimation are two important research topics that aim to predict human mental states using multimodal data. Previous research has focused on developing effective fusion strategies for exchanging and integrating mind-related information from different modalities. Some MLP-based techniques have recently achieved considerable success in a variety of computer vision tasks. Inspired by this, we explore multimodal approaches with a feature-mixing perspective in this study. To this end, we introduce CubeMLP, a multimodal feature processing framework based entirely on MLP. CubeMLP consists of three independent MLP units, each of which has two affine transformations. CubeMLP accepts all relevant modality features as input and mixes them across three axes. After extracting the characteristics using CubeMLP, the mixed multimodal features are flattened for task predictions. Our experiments are conducted on sentiment analysis datasets: CMU-MOSI and CMU-MOSEI, and depression estimation dataset: AVEC2019. The results show that CubeMLP can achieve state-of-the-art performance with a much lower computing cost.

cs.MM↗

arXiv2022

A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language

Although artificial intelligence (AI) has made significant progress in understanding molecules in a wide range of fields, existing models generally acquire the single cognitive ability from the single molecular modality. Since the hierarchy of molecular knowledge is profound, even humans learn from different modalities including both intuitive diagrams and professional texts to assist their understanding. Inspired by this, we propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data (crawled from published Scientific Citation Index papers) via contrastive learning. This AI model represents a critical attempt that directly bridges molecular graphs and natural language. Importantly, through capturing the specific and complementary information of the two modalities, our proposed model can better grasp molecular expertise. Experimental results show that our model not only exhibits promising performance in cross-modal tasks such as cross-modal retrieval and molecule caption, but also enhances molecular property prediction and possesses capability to generate meaningful molecular graphs from natural language descriptions. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine, among others.

cs.LG↗