arXiv Science⌕ Search

arXiv subjects

Jangho Kim

Publications and source records attributed to Jangho Kim.

At least 55 records · Page 3Linked to original sources
arXiv2020

Paraphrasing Complex Network: Network Compression via Factor Transfer

Many researchers have sought ways of model compression to reduce the size of a deep neural network (DNN) with minimal performance degradation in order to use DNNs in embedded systems. Among the model compression methods, a method called knowledge transfer is to train a student network with a stronger teacher network. In this paper, we propose a novel knowledge transfer method which uses convolutional operations to paraphrase teacher's knowledge and to translate it for the student. This is done by two convolutional modules, which are called a paraphraser and a translator. The paraphraser is trained in an unsupervised manner to extract the teacher factors which are defined as paraphrased information of the teacher network. The translator located at the student network extracts the student factors and helps to translate the teacher factors by mimicking them. We observed that our student network trained with the proposed factor transfer method outperforms the ones trained with conventional knowledge transfer methods.

cs.CV↗
arXiv2020

Feature Fusion for Online Mutual Knowledge Distillation

We propose a learning framework named Feature Fusion Learning (FFL) that efficiently trains a powerful classifier through a fusion module which combines the feature maps generated from parallel neural networks. Specifically, we train a number of parallel neural networks as sub-networks, then we combine the feature maps from each sub-network using a fusion module to create a more meaningful feature map. The fused feature map is passed into the fused classifier for overall classification. Unlike existing feature fusion methods, in our framework, an ensemble of sub-network classifiers transfers its knowledge to the fused classifier and then the fused classifier delivers its knowledge back to each sub-network, mutually teaching one another in an online-knowledge distillation manner. This mutually teaching system not only improves the performance of the fused classifier but also obtains performance gain in each sub-network. Moreover, our model is more beneficial because different types of network can be used for each sub-network. We have performed a variety of experiments on multiple datasets such as CIFAR-10, CIFAR-100 and ImageNet and proved that our method is more effective than other alternative methods in terms of performance of both sub-networks and the fused classifier.

cs.CV↗
arXiv2020

Feature-map-level Online Adversarial Knowledge Distillation

Feature maps contain rich information about image intensity and spatial correlation. However, previous online knowledge distillation methods only utilize the class probabilities. Thus in this paper, we propose an online knowledge distillation method that transfers not only the knowledge of the class probabilities but also that of the feature map using the adversarial training framework. We train multiple networks simultaneously by employing discriminators to distinguish the feature map distributions of different networks. Each network has its corresponding discriminator which discriminates the feature map from its own as fake while classifying that of the other network as real. By training a network to fool the corresponding discriminator, it can learn the other network's feature map distribution. We show that our method performs better than the conventional direct alignment method such as L1 and is more suitable for online distillation. Also, we propose a novel cyclic learning scheme for training more than two networks together. We have applied our method to various network architectures on the classification task and discovered a significant improvement of performance especially in the case of training a pair of a small network and a large one.

cs.LG↗
arXiv2020

StackNet: Stacking Parameters for Continual learning

Training a neural network for a classification task typically assumes that the data to train are given from the beginning. However, in the real world, additional data accumulate gradually and the model requires additional training without accessing the old training data. This usually leads to the catastrophic forgetting problem which is inevitable for the traditional training methodology of neural networks. In this paper, we propose a continual learning method that is able to learn additional tasks while retaining the performance of previously learned tasks by stacking parameters. Composed of two complementary components, the index module and the StackNet, our method estimates the index of the corresponding task for an input sample with the index module and utilizes a particular portion of StackNet with this index. The StackNet guarantees no degradation in the performance of the previously learned tasks and the index module shows high confidence in finding the origin of an input sample. Compared to the previous work of PackNet, our method is competitive and highly intuitive.

cs.LG↗
arXiv2020

Gauge Corrections to Strong Coupling Lattice QCD on Anisotropic Lattices

Lattice QCD with staggered fermions can be formulated in dual variables to address the finite baryon density sign problem. In the past we have performed simulations in the strong coupling regime, including leading order gauge corrections. In order to vary the temperature for fixed $β$ it was necessary to introduce a bare anisotropy. In this talk we will extend our work to include results from a non-perturbative determination of the physical anisotropy $a_σ$/$a_τ= ξ(γ, β)$, which is necessary to unambiguously locate the critical end point and the first order line of the chiral transition.

hep-lat↗
arXiv2019

The Yang-Mills deconfinement transition from a high temperature expansion

The high temperature expansion is an analytical tool to study critical phenomena in statistical mechanics. We apply this method to 3d effective theories of Polyakov loops, which have been derived from 4d lattice Yang-Mills by means of resummed strong coupling expansions. In particular, the Polyakov loop susceptibility is computed as a power series in the effective couplings. A Padé analysis then provides the location of the phase transition in the effective theory, which can be mapped back to the parameters of 4d Yang-Mills. Our purely analytical results for the critical couplings $β_c(N_τ)$ agree to better than $10\%$ with those from Monte Carlo simulations. For the case of $SU(2)$, also the critical exponent $γ$ is predicted accurately, while a first-order nature as for $SU(3)$ cannot be identified by a Padé analysis. The method can be generalized to include fermions and finite density.

hep-lat↗
arXiv2019

On the $β$- and quark mass dependence of the nuclear transition in the strong coupling regime

Lattice QCD in a dual formulation with staggered fermions is well established in the strong coupling limit and allows to perform Monte Carlo simulations at finite baryon chemical potential. We have recently addressed the dependence of the nuclear critical end point as a function of the quark mass $am_q$, and separately as a function of the lattice gauge coupling $β$ in the chiral limit. Here we proceed to determine the dependence of the nuclear transition on both, $am_q$ and $β$, on isotropic lattices and attempt to pinpoint the critical end point for various $β$ where the sign problem is still manageable.

hep-lat↗
arXiv2019

QKD: Quantization-aware Knowledge Distillation

Quantization and Knowledge distillation (KD) methods are widely used to reduce memory and power consumption of deep neural networks (DNNs), especially for resource-constrained edge devices. Although their combination is quite promising to meet these requirements, it may not work as desired. It is mainly because the regularization effect of KD further diminishes the already reduced representation power of a quantized model. To address this short-coming, we propose Quantization-aware Knowledge Distillation (QKD) wherein quantization and KD are care-fully coordinated in three phases. First, Self-studying (SS) phase fine-tunes a quantized low-precision student network without KD to obtain a good initialization. Second, Co-studying (CS) phase tries to train a teacher to make it more quantizaion-friendly and powerful than a fixed teacher. Finally, Tutoring (TU) phase transfers knowledge from the trained teacher to the student. We extensively evaluate our method on ImageNet and CIFAR-10/100 datasets and show an ablation study on networks with both standard and depthwise-separable convolutions. The proposed QKD outperformed existing state-of-the-art methods (e.g., 1.3% improvement on ResNet-18 with W4A4, 2.6% on MobileNetV2 with W4A4). Additionally, QKD could recover the full-precision accuracy at as low as W3A3 quantization on ResNet and W6A6 quantization on MobilenetV2.

cs.CV↗
arXiv2018

Towards a determination of the nucleon EDM from the quark chromo-EDM operator with the gradient flow

In this proceedings, we lay the foundation for computing the contribution of quark chromo-electric dipole moment (qCEDM) operator to the nucleon electric dipole moment. By applying the gradient flow technique, we can parameterize the renormalization and operator mixing issues associated with the qCEDM operator on the lattice. As the nucleon mixing angle $α_N$ is a key component for determining the neutron and proton electric dipole moments induced by the qCEDM operator, we present the formalism and preliminary results for $α_N$ with respect to the gradient flow time $t_f$. The results are computed on $N_f=2+1$ Wilson-clover lattices provided by PACS-CS. The 3 ensembles have lattice spacing values of $a=\lbrace 0.1095,\,0.0936,\,0.0684\rbrace $fm, whilst keeping a similar $m_π\approx701$MeV, and a fixed box size of $L\approx1.9$fm.

hep-lat↗
arXiv2017

How to identify zero modes for improved staggered fermions

We present results of the eigenvalue spectrum for the staggered Diräc operator obtained using a modified Lanczos algorithm. We identify zero modes and non-zero modes. We derive the chiral Ward identity derived from the conserved $U(1)_A$ symmetry, and check it numerically. This is the first step toward construction of an improved method to identify zero modes reliably with staggered fermions.

hep-lat↗
arXiv2017

Dual Formulation and Phase Diagram of Lattice QCD in the Strong Coupling Regime

We present the computation of invariants that arise in the strong coupling expansion of lattice QCD. These invariants are needed for Monte Carlo simulations of Lattice QCD with staggered fermions in a dual, color singlet representation. This formulation is in particular useful to tame the finite density sign problem. The gauge integrals in this limiting case $β\rightarrow 0$ are well known, but the gauge integrals needed to study the gauge corrections are more involved. We discuss a method to evaluate such integrals. The phase boundary of lattice QCD for staggered fermions in the $μ_B-T$ plane has been established in the strong coupling limit. We present numerical simulations away from the strong coupling limit, taking into account the higher order gauge corrections via plaquette occupation numbers. This allows to study the nuclear and chiral transition as a function of $β$.

hep-lat↗
arXiv2016

Quark Mass Dependence of the QCD Critical End Point in the Strong Coupling Limit

Strong coupling lattice QCD in the dual representation allows to study the full $μ$-$T$ phase diagram, due to the mildness of the finite density sign problem. Such simulations have been performed in the chiral limit, both at finite $N_t$ and in the continuous time limit. Here we extend the phase diagram to finite quark masses, with an emphasis on the low temperature first order transition. We present our results on the quark mass dependence of the critical end point and the first order line obtained by Monte Carlo via the worm algorithm.

hep-lat↗
arXiv2015

Kaon BSM B-parameters using improved staggered fermions from $N_f=2+1$ unquenched QCD

We present results for the matrix elements of the additional $ΔS=2$ operators that appear in models of physics beyond the Standard Model (BSM), expressed in terms of four BSM $B$-parameters. Combined with experimental results for $ΔM_K$ and $ε_K$, these constrain the parameters of BSM models. We use improved staggered fermions, with valence HYP-smeared quarks and $N_f=2+1$ flavors of "asqtad" sea quarks. The configurations have been generated by the MILC collaboration. The matching between lattice and continuum four-fermion operators and bilinears is done perturbatively at one-loop order. We use three lattice spacings for the continuum extrapolation: $a\approx 0.09$, $0.06$ and $0.045\;$fm. Valence light-quark masses range down to $\approx m_s^{\rm phys}/13$ while the light sea-quark masses range down to $\approx m_s^{\rm phys}/20$. Compared to our previous published work, we have added four additional lattice ensembles, leading to better controlled extrapolations in the lattice spacing and sea-quark masses. We report final results for two renormalization scales, $μ=2\;\text{GeV}$ and $3\;\text{GeV}$, and compare them to those obtained by other collaborations. Agreement is found for two of the four BSM $B$-parameters ($B_2$ and $B_3^\text{SUSY}$). The other two ($B_4$ and $B_5$) differ significantly from those obtained using RI-MOM renormalization as an intermediate scheme, but are in agreement with recent preliminary results obtained by the RBC-UKQCD collaboration using RI-SMOM intermediate schemes.

hep-lat↗
arXiv2015

Nonperturbative Renormalization in the RI-SMOM Scheme and Gribov Uncertainty in the RI-MOM Scheme for Staggered Bilinears

We present results of renormalization factors for bilinear operators obtained using the nonperturbative renormalization method (NPR) in the RI-SMOM schemes. The operators are constructed using HYP staggered quarks on the MILC asqtad lattice ($N_f=2+1$). We compare results in the RI-SMOM schemes with those in the RI-MOM scheme for the $V\otimes S$ and $S\otimes S$ operators. Since we use Landau gauge fixing, we study the effect of Gribov ambiguity on the wave function renormalization $Z_q$ in the RI-MOM scheme. We find that the Gribov uncertainty is negligibly small for $Z_q$ in the RI-MOM scheme.

hep-lat↗
arXiv2015

Non-perturbative Renormalization of Bilinear Operators on Fine Lattice

We present results of the wave function renormalization factor $Z_q$ and mass renormalization factor $Z_m$ obtained using non-perturbative renormalization (NPR) method in the RI-MOM scheme with HYP improved staggered quarks. We use fine ensembles of MILC asqtad lattices ($N_f = 2+1$) with $28^3 \times 96$ geometry, $a \approx 0.09$\,fm, and $am_\ell/am_s = 0.0062/0.031 $. We also study on scalability of $Z_q$ and $Z_m$ by comparing the results on the coarse and fine ensembles.

hep-lat↗
arXiv2015

Performance of GTX Titan X GPUs and Code Optimization

Recently Nvidia has released a new GPU model: GTX Titan X (TX) in a linage of the Maxwell architecture. We use our conjugate gradient code and non-perturbative renormalization code to measure the performance of TX. The results are compared with those of GTX Titan Black (TB) in a lineage of the Kepler architecture. We observe a significant gain in the single and double precision calculations much greater than the theoretical expectation.

hep-lat↗
arXiv2014

Code Optimization on Kepler GPUs and Xeon Phi

Kepler GTX Titan Black and Kepler Tesla K40 are still the best GPUs for high performance computing, although Maxwell GPUs such as GTX 980 are available in the market. Hence, we measure the performance of our lattice QCD codes using the Kepler GPUs. We also upgrade our code to use the latest CPS (Columbia Physics System) library along with the most recent QUDA (QCD CUDA) library for lattice QCD. These new libraries improve the performance of our conjugate gradient (CG) inverter so that it runs twice faster than before. We also investigate the performance of Xeon Phi 7120P coprocessor. It has similar computing power with the Kepler GPUs in principle. However, its performance for our CG code is significantly inferior to that of the GTX Titan Black GPUs at present.

hep-lat↗