arXiv subjects

Zhao Zhang

Publications and source records attributed to Zhao Zhang.

At least 235 records · Page 13Linked to original sources

arXiv2021

Towards Robust Knowledge Graph Embedding via Multi-task Reinforcement Learning

Nowadays, Knowledge graphs (KGs) have been playing a pivotal role in AI-related applications. Despite the large sizes, existing KGs are far from complete and comprehensive. In order to continuously enrich KGs, automatic knowledge construction and update mechanisms are usually utilized, which inevitably bring in plenty of noise. However, most existing knowledge graph embedding (KGE) methods assume that all the triple facts in KGs are correct, and project both entities and relations into a low-dimensional space without considering noise and knowledge conflicts. This will lead to low-quality and unreliable representations of KGs. To this end, in this paper, we propose a general multi-task reinforcement learning framework, which can greatly alleviate the noisy data problem. In our framework, we exploit reinforcement learning for choosing high-quality knowledge triples while filtering out the noisy ones. Also, in order to take full advantage of the correlations among semantically similar relations, the triple selection processes of similar relations are trained in a collective way with multi-task learning. Moreover, we extend popular KGE models TransE, DistMult, ConvE and RotatE with the proposed framework. Finally, the experimental validation shows that our approach is able to enhance existing KGE models and can provide more robust representations of KGs in noisy scenarios.

cs.CL↗

arXiv2021

Evidence for gravitational lensing of GRB 200716C

Observationally, there is a small fraction of Gamma-ray bursts (GRBs) with prompt emission observed by Fermi/GBM that are composed of two pulses. Occasionally, the distance to a GRB may be lensed when a high mass astrophysical object resides in the path between the GRB source and observer. In this paper, we describe GRB 200716C, which has a two-pulse emission and duration of a few seconds. We present a Bayesian analysis identifying gravitational lensing in both temporal and spectral properties, and calculate the time delay ($Δt\sim 1.92$ s) and magnification ($γ\sim 1.5$) between those two pulses based on the temporal fits. One can roughly estimate the lens mass to be about $2.4\times 10^{5}~M_{\odot}$ in the rest frame. We also calculate the false alarm probability for this detection to be about 0.07\% with trial factors, and a present-day number density of about $808 \rm~Mpc^{-3}$ with an energy density $Ω\sim 1.4\times 10^{-3}$. If the first pulse of this GRB near the trigger time is indeed gravitationally echoed by a second pulse, GRB 200716C may be a short GRB candidate with extended emission.

astro-ph.HE↗

arXiv2021

A Multi-Stencil Fast Marching Method with Path Correction for Efficient Reservoir Simulation and Automated History Matching

The efficiency of reservoir simulation is important for automated history matching (AHM) and production optimization, etc. The fast marching marching method (FMM) has been used for efficient reservoir simulation. FMM can be regarded as a generalised streamline method but without the need to construct streamlines. In FMM reservoir simulation, the Eikonal equation for the diffusive time-of-flight (DTOF) is solved by FMM and then the governing equations are computed on the 1D DTOF coordinate. Standard FMM solves the Eikonal equation using a 4-stencil algorithm on 2D Cartesian grids, ignoring the diagonal neighbouring cells. In the current study, we build a 8-stencil algorithm considering all neighbouring cells, and use local analytical propagation speeds. In addition, a local path-correction coefficient is introduced to further increase the accuracy of DTOF solution. Next, a discretisation scheme is built on the 1D DTOF coordinate for efficient reservoir simulation. The algorithm is validated on homogeneous and heterogeneous test cases, and its potential for efficient forward simulation in AHM is demonstrated by two examples of dimensions 2 and 6.

physics.flu-dyn↗

arXiv2021

GRB 210121A: A Typical Fireball Burst Detected by Two Small Missions

The Chinese CubeSat Mission, Gamma Ray Integrated Detectors (GRID), recently detected its first gamma-ray burst, GRB 210121A, which was jointly observed by the Gravitational wave high-energy Electromagnetic Counterpart All-sky Monitor (GECAM). This burst is confirmed by several other missions, including \fermi and \textit{Insight}-HXMT. We combined multi-mission observational data and performed a comprehensive analysis of the burst's temporal and spectral properties. Our results show that the burst is relatively special in its high peak energy, thermal-like low energy indices, and large fluence. By putting it to the $E_{\rm p}$-$E_{\rmγ, iso}$ relation diagram with assumed distance, we found this burst can be constrained at the redshift range of [0.3,3.0]. The thermal spectral component is also confirmed by the direct fit of the physical models to the observed spectra. Interestingly, the physical photosphere model also constrained a redshift of $z\sim$ 0.3 for this burst, which help us to identify a host galaxy candidate at such a distance within the location error box. Assuming the host galaxy is real, we found the burst can be best explained by the photosphere emission of a typical fireball with an initial radius of $r_0\sim$ 3.2 $\times 10^7$ cm.

astro-ph.HE↗

arXiv2021

KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks

Kronecker-factored Approximate Curvature (K-FAC) has recently been shown to converge faster in deep neural network (DNN) training than stochastic gradient descent (SGD); however, K-FAC's larger memory footprint hinders its applicability to large models. We present KAISA, a K-FAC-enabled, Adaptable, Improved, and ScAlable second-order optimizer framework that adapts the memory footprint, communication, and computation given specific models and hardware to improve performance and increase scalability. We quantify the tradeoffs between memory and communication cost and evaluate KAISA on large models, including ResNet-50, Mask R-CNN, U-Net, and BERT, on up to 128 NVIDIA A100 GPUs. Compared to the original optimizers, KAISA converges 18.1-36.3% faster across applications with the same global batch size. Under a fixed memory budget, KAISA converges 32.5% and 41.6% faster in ResNet-50 and BERT-Large, respectively. KAISA can balance memory and communication to achieve scaling efficiency equal to or better than the baseline optimizers. KAISA is open source and available at https://github.com/gpauloski/kfac_pytorch.

cs.LG↗

arXiv2021

Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior

Nonnegative matrix factorization is usually powerful for learning the "shallow" parts-based representation, but it clearly fails to discover deep hierarchical information within both the basis and representation spaces. In this paper, we technically propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net, for learning the hierarchical coupled representations. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Specifically, DS2CF-Net designs a deep coupled factorization architecture using multi-layers of linear transformations, which coupled updates the bases and new representations in each layer. To improve the discriminating ability of learned deep representations and deep coefficients, our network clearly considers enriching the supervised prior by the joint deep coefficients-regularized label prediction, and incorporates enriched prior information as additional label and structure constraints. The label constraint can enable the samples of the same label to have the same coordinate in the new feature space, while the structure constraint forces the coefficient matrices in each layer to be block-diagonal so that the enhanced prior using the self-expressive label propagation are more accurate. Our network also integrates the adaptive dual-graph learning to retain the local manifold structures of both the data manifold and feature manifold by minimizing the reconstruction errors in each layer. Extensive experiments on several real databases demonstrate that our DS2CF-Net can obtain state-of-the-art performance for representation learning and clustering.

cs.LG↗

arXiv2021

Characterizing Impacts of Storage Faults on HPC Applications: A Methodology and Insights

In recent years, the increasing complexity in scientific simulations and emerging demands for training heavy artificial intelligence models require massive and fast data accesses, which urges high-performance computing (HPC) platforms to equip with more advanced storage infrastructures such as solid-state disks (SSDs). While SSDs offer high-performance I/O, the reliability challenges faced by the HPC applications under the SSD-related failures remains unclear, in particular for failures resulting in data corruptions. The goal of this paper is to understand the impact of SSD-related faults on the behaviors of complex HPC applications. To this end, we propose FFIS, a FUSE-based fault injection framework that systematically introduces storage faults into the application layer to model the errors originated from SSDs. FFIS is able to plant different I/O related faults into the data returned from underlying file systems, which enables the investigation on the error resilience characteristics of the scientific file format. We demonstrate the use of FFIS with three representative real HPC applications, showing how each application reacts to the data corruptions, and provide insights on the error resilience of the widely adopted HDF5 file format for the HPC applications.

cs.DC↗

arXiv2021

Many-to-English Machine Translation Tools, Data, and Pretrained Models

While there are more than 7000 languages in the world, most translation research efforts have targeted a few high-resource languages. Commercial translation systems support only one hundred languages or fewer, and do not make these models available for transfer to low resource languages. In this work, we present useful tools for machine translation research: MTData, NLCodec, and RTG. We demonstrate their usefulness by creating a multilingual neural machine translation model capable of translating from 500 source languages to English. We make this multilingual model readily downloadable and usable as a service, or as a parent model for transfer-learning to even lower-resource languages.

cs.CL↗

arXiv2021

Barocaloric thermal batteries

Nowadays the world is facing a prominent paradox regarding thermal energy. The production of heat accounts for more than 50% of global final energy consumption while the waste heat potential analysis reveals that 72% of the global primary energy consumption is lost after conversion mainly in the form of heat. Towards global decarbonization, it is of vital importance to establish a solution to thermal energy utilization under full control. Here, we propose and realize an unprecedented concept -- barocaloric thermal batteries based on the inverse colossal barocaloric effect of NH4SCN. Thermal charging is initialized upon pressurization through an order-to-disorder phase transition below 364 K and in turn the discharging of 43 J g-1, which are eleven times more than the input mechanical energy, occurs on demand at depressurization at lower temperatures. The discharging is also manifested as a directly measured temperature rise of 12 K. The thermodynamic equilibrium nature of the pressure-restrained heat-carrying phase guarantees stable storage and/or transport over a variety of temporal and/or spatial scales. The barocaloric thermal batteries reinforced by their solid microscopic mechanism are expected to significantly advance the ability to take advantage of waste heat.

cond-mat.mtrl-sci↗

arXiv2021

Phase transition gravitational waves from pseudo-Nambu-Goldstone dark matter and two Higgs doublets

We investigate the potential stochastic gravitational waves from first-order electroweak phase transitions in a model with pseudo-Nambu-Goldstone dark matter and two Higgs doublets. The dark matter candidate can naturally evade direct detection bounds, and can achieve the observed relic abundance via the thermal mechanism. Three scalar fields in the model obtain vacuum expectation values, related to phase transitions at the early Universe. We search for the parameter points that can cause first-order phase transitions, taking into account the existed experimental constraints. The resulting gravitational wave spectra are further evaluated. Some parameter points are found to induce strong gravitational wave signals, which have the opportunity to be detected in future space-based interferometer experiments LISA, Taiji, and TianQin.

hep-ph↗

arXiv2021

DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking

Single image deraining (SID) is an important and challenging topic in emerging vision applications, and most of emerged deraining methods are supervised relying on the ground truth (i.e., paired images) in recent years. However, in practice it is rather common to have no un-paired images in real deraining task, in such cases how to remove the rain streaks in an unsupervised way will be a very challenging task due to lack of constraints between images and hence suffering from low-quality recovery results. In this paper, we explore the unsupervised SID task using unpaired data and propose a novel net called Attention-guided Deraining by Constrained CycleGAN (or shortly, DerainCycleGAN), which can fully utilize the constrained transfer learning abilitiy and circulatory structure of CycleGAN. Specifically, we design an unsu-pervised attention guided rain streak extractor (U-ARSE) that utilizes a memory to extract the rain streak masks with two constrained cycle-consistency branches jointly by paying attention to both the rainy and rain-free image domains. As a by-product, we also contribute a new paired rain image dataset called Rain200A, which is constructed by our network automatically. Compared with existing synthesis datasets, the rainy streaks in Rain200A contains more obvious and diverse shapes and directions. As a result, existing supervised methods trained on Rain200A can perform much better for processing real rainy images. Extensive experiments on synthesis and real datasets show that our net is superior to existing unsupervised deraining networks, and is also very competitive to other related supervised networks.

cs.CV↗

arXiv2021

Semi-DerainGAN: A New Semi-supervised Single Image Deraining Network

Removing the rain streaks from single image is still a challenging task, since the shapes and directions of rain streaks in the synthetic datasets are very different from real images. Although supervised deep deraining networks have obtained impressive results on synthetic datasets, they still cannot obtain satisfactory results on real images due to weak generalization of rain removal capacity, i.e., the pre-trained models usually cannot handle new shapes and directions that may lead to over-derained/under-derained results. In this paper, we propose a new semi-supervised GAN-based deraining network termed Semi-DerainGAN, which can use both synthetic and real rainy images in a uniform network using two supervised and unsupervised processes. Specifically, a semi-supervised rain streak learner termed SSRML sharing the same parameters of both processes is derived, which makes the real images contribute more rain streak information. To deliver better deraining results, we design a paired discriminator for distinguishing the real pairs from fake pairs. Note that we also contribute a new real-world rainy image dataset Real200 to alleviate the difference between the synthetic and real image do-mains. Extensive results on public datasets show that our model can obtain competitive performance, especially on real images.

cs.CV↗

arXiv2021

Dense Residual Network: Enhancing Global Dense Feature Flow for Character Recognition

Deep Convolutional Neural Networks (CNNs), such as Dense Convolutional Networks (DenseNet), have achieved great success for image representation by discovering deep hierarchical information. However, most existing networks simply stacks the convolutional layers and hence failing to fully discover local and global feature information among layers. In this paper, we mainly explore how to enhance the local and global dense feature flow by exploiting hierarchical features fully from all the convolution layers. Technically, we propose an efficient and effective CNN framework, i.e., Fast Dense Residual Network (FDRN), for text recognition. To construct FDRN, we propose a new fast residual dense block (f-RDB) to retain the ability of local feature fusion and local residual learning of original RDB, which can reduce the computing efforts at the same time. After fully learning local residual dense features, we utilize the sum operation and several f-RDBs to define a new block termed global dense block (GDB) by imitating the construction of dense blocks to learn global dense residual features adaptively in a holistic way. Finally, we use two convolution layers to construct a down-sampling block to reduce the global feature size and extract deeper features. Extensive simulations show that FDRN obtains the enhanced recognition results, compared with other related models.

cs.CV↗

arXiv2021

A Survey on Concept Factorization: From Shallow to Deep Representation Learning

The quality of learned features by representation learning determines the performance of learning algorithms and the related application tasks (such as high-dimensional data clustering). As a relatively new paradigm for representation learning, Concept Factorization (CF) has attracted a great deal of interests in the areas of machine learning and data mining for over a decade. Lots of effective CF based methods have been proposed based on different perspectives and properties, but note that it still remains not easy to grasp the essential connections and figure out the underlying explanatory factors from exiting studies. In this paper, we therefore survey the recent advances on CF methodologies and the potential benchmarks by categorizing and summarizing the current methods. Specifically, we first re-view the root CF method, and then explore the advancement of CF-based representation learning ranging from shallow to deep/multilayer cases. We also introduce the potential application areas of CF-based methods. Finally, we point out some future directions for studying the CF-based representation learning. Overall, this survey provides an insightful overview of both theoretical basis and current developments in the field of CF, which can also help the interested researchers to understand the current trends of CF and find the most appropriate CF techniques to deal with particular applications.

cs.LG↗

arXiv2020

Prime Suspects in a Quantum Ladder

In this Letter we set up a suggestive number theory interpretation of a quantum ladder system made of N coupled chains of spin 1/2. Using the hard-core boson representation and a leg-Hamiltonian made of a magnetic field and a hopping term, we can associate to the spins $s_a$ the prime numbers $p_a$ so that the chains become quantum registers for square-free integers. The rung Hamiltonian involves permutation terms between next neighborhood chains and a coprime repulsive interaction. The system has various phases; in particular there is one whose ground state is a coherent superposition of the first N prime numbers. We also discuss the realization of such a model in terms of an open quantum system with a dissipative Lindblad dynamics.

cond-mat.stat-mech↗

arXiv2020

Gradient-Induced Co-Saliency Detection

Co-saliency detection (Co-SOD) aims to segment the common salient foreground in a group of relevant images. In this paper, inspired by human behavior, we propose a gradient-induced co-saliency detection (GICD) method. We first abstract a consensus representation for the grouped images in the embedding space; then, by comparing the single image with consensus representation, we utilize the feedback gradient information to induce more attention to the discriminative co-salient features. In addition, due to the lack of Co-SOD training data, we design a jigsaw training strategy, with which Co-SOD networks can be trained on general saliency datasets without extra pixel-level annotations. To evaluate the performance of Co-SOD methods on discovering the co-salient object among multiple foregrounds, we construct a challenging CoCA dataset, where each image contains at least one extraneous foreground along with the co-salient object. Experiments demonstrate that our GICD achieves state-of-the-art performance. Our codes and dataset are available at https://mmcheng.net/gicd/.

cs.CV↗

arXiv2020

Ultrafast ferromagnetic fluctuations preceding magnetoelastic first-order transitions

First-order magnetic transitions are of both fundamental and technological interest given that a number of emergent phases and functionalities are thereby created. Of particular interest are giant magnetocaloric effects, which are attributed to first-order magnetic transitions and have attracted broad attention for solid-state refrigeration applications. While the conventional wisdom is that atomic lattices play an important role in first-order magnetic transitions, a coherent microscopic description of the lattice and spin degrees of freedom is still lacking. Here, we study the magnetic phase transition dynamics on the intermetallic LaFe13-xSix, which is one of the most classical giant magnetocaloric systems, in both frequency and time domains utilizing neutron scattering and ultrafast X-ray diffraction. We have observed a strong magnetic diffuse scattering in the paramagnetic state preceding the first-order magnetic transition, corresponding to picosecond ferromagnetic fluctuations. Upon photon-excitation, the ferromagnetic state is completely suppressed in 0.9 ps and recovered in 20 ps. The ultrafast dynamics suggest that the magnetic degree of freedom dominates this magnetoelastic transition and ferromagnetic fluctuations might be universally relevant for this kind of compounds.

cond-mat.mtrl-sci↗

arXiv2020

Kernelized Multiview Subspace Analysis by Self-weighted Learning

With the popularity of multimedia technology, information is always represented or transmitted from multiple views. Most of the existing algorithms are graph-based ones to learn the complex structures within multiview data but overlooked the information within data representations. Furthermore, many existing works treat multiple views discriminatively by introducing some hyperparameters, which is undesirable in practice. To this end, abundant multiview based methods have been proposed for dimension reduction. However, there are still no research to leverage the existing work into a unified framework. To address this issue, in this paper, we propose a general framework for multiview data dimension reduction, named Kernelized Multiview Subspace Analysis (KMSA). It directly handles the multi-view feature representation in the kernel space, which provides a feasible channel for direct manipulations on multiview data with different dimensions. Meanwhile, compared with those graph-based methods, KMSA can fully exploit information from multiview data with nothing to lose. Furthermore, since different views have different influences on KMSA, we propose a self-weighted strategy to treat different views discriminatively according to their contributions. A co-regularized term is proposed to promote the mutual learning from multi-views. KMSA combines self-weighted learning with the co-regularized term to learn appropriate weights for all views. We also discuss the influence of the parameters in KMSA regarding the weights of multi-views. We evaluate our proposed framework on 6 multiview datasets for classification and image retrieval. The experimental results validate the advantages of our proposed method.

cs.LG↗