arXiv subjects

Jangho Kim

Publications and source records attributed to Jangho Kim.

At least 37 records · Page 2Linked to original sources

arXiv2023

$U(N)$ gauge theory in the strong coupling limit on a quantum annealer

Lattice QCD in the strong coupling regime can be formulated in dual variables which are integer-valued. It can be efficiently simulated for modest finite temperatures and finite densities via the worm algorithm, circumventing the finite density sign problem in this regime. However, the low temperature regime is more expensive to address. As the partition function is solely expressed in terms of integers, it can be cast as a combinatorial optimization problem that can be solved on a quantum annealer. We will first explain the setup of the system we want to study, and then present its reformulation suitable for a quantum annealer, and in particular the D-Wave. As a proof of concept, we present first results obtained on D-Wave for gauge group $U(1)$ and $U(3)$, and outline the next steps towards gauge groups $SU(3)$. We find that in addition, histogram reweighting greatly improves the accuracy of our observables when compared to analytic results.

hep-lat↗

arXiv2023

Magnitude Attention-based Dynamic Pruning

Existing pruning methods utilize the importance of each weight based on specified criteria only when searching for a sparse structure but do not utilize it during training. In this work, we propose a novel approach - \textbf{M}agnitude \textbf{A}ttention-based Dynamic \textbf{P}runing (MAP) method, which applies the importance of weights throughout both the forward and backward paths to explore sparse model structures dynamically. Magnitude attention is defined based on the magnitude of weights as continuous real-valued numbers enabling a seamless transition from a redundant to an effective sparse network by promoting efficient exploration. Additionally, the attention mechanism ensures more effective updates for important layers within the sparse network. In later stages of training, our approach shifts from exploration to exploitation, exclusively updating the sparse model composed of crucial weights based on the explored structure, resulting in pruned models that not only achieve performance comparable to dense models but also outperform previous pruning methods on CIFAR-10/100 and ImageNet.

cs.CV↗

arXiv2023

Nuclear Liquid-Gas Transition in the Strong Coupling Regime of Lattice QCD

The nuclear liquid-gas transition from a gas of hadrons to a nuclear phase cannot be determined numerically from conventional lattice QCD due to the severe sign problem at large values of the baryon chemical potential. In the strong coupling regime of lattice QCD with staggered quarks, the dual formulation is suitable to address the nuclear liquid gas transition. We determine this first order transition at low temperatures and as a function of the quark mass and the inverse gauge coupling $β$. We also determine the baryon mass and discuss the nuclear interactions as a function of the quark mass, and compare to mean field results.

hep-lat↗

arXiv2022

Nuclear Transition in the Strong Coupling Limit

Lattice QCD at finite baryon chemical potential has the infamous sign problem which hinders Monte Carlo simulations. This can be remedied by a dual representation that makes the sign problem mild. In the strong coupling limit, the dual formulation with staggered quarks is well established. We have used this formulation to study the quark mass dependence of the baryon mass and the nuclear transition. This allows us to quantify the nuclear interaction. We have also compared our Monte Carlo results with mean field predictions.

hep-lat↗

arXiv2022

Error reduction using machine learning on Ising worm simulation

We develop a method to improve on the statistical errors for higher moments using machine learning techniques. We present here results for the dual representation of the Ising model with an external field, derived via the high temperature expansion and simulated by the worm algorithm. We compare two ways of measuring the same set of observables, without and with machine learning: moments of the magnetization and the susceptibility can be improved by using the decision tree method to train the correlations between the higher moments and the second moment obtained from an integrated 2-point function. Those results are compared in small volumes to analytic predictions.

hep-lat↗

arXiv2022

Self-Distilled Self-Supervised Representation Learning

State-of-the-art frameworks in self-supervised learning have recently shown that fully utilizing transformer-based models can lead to performance boost compared to conventional CNN models. Striving to maximize the mutual information of two views of an image, existing works apply a contrastive loss to the final representations. Motivated by self-distillation in the supervised regime, we further exploit this by allowing the intermediate representations to learn from the final layer via the contrastive loss. Through self-distillation, the intermediate layers are better suited for instance discrimination, making the performance of an early-exited sub-network not much degraded from that of the full network. This renders the pretext task easier also for the final layer, leading to better representations. Our method, Self-Distilled Self-Supervised Learning (SDSSL), outperforms competitive baselines (SimCLR, BYOL and MoCo v3) using ViT on various tasks and datasets. In the linear evaluation and k-NN protocol, SDSSL not only leads to superior performance in the final layers, but also in most of the lower layers. Furthermore, qualitative and quantitative analyses show how representations are formed more effectively along the transformer layers. Code is available at https://github.com/hagiss/SDSSL.

cs.CV↗

arXiv2022

QTI Submission to DCASE 2021: residual normalization for device-imbalanced acoustic scene classification with efficient design

This technical report describes the details of our TASK1A submission of the DCASE2021 challenge. The goal of the task is to design an audio scene classification system for device-imbalanced datasets under the constraints of model complexity. This report introduces four methods to achieve the goal. First, we propose Residual Normalization, a novel feature normalization method that uses instance normalization with a shortcut path to discard unnecessary device-specific information without losing useful information for classification. Second, we design an efficient architecture, BC-ResNet-Mod, a modified version of the baseline architecture with a limited receptive field. Third, we exploit spectrogram-to-spectrogram translation from one to multiple devices to augment training data. Finally, we utilize three model compression schemes: pruning, quantization, and knowledge distillation to reduce model complexity. The proposed system achieves an average test accuracy of 76.3% in TAU Urban Acoustic Scenes 2020 Mobile, development dataset with 315k parameters, and average test accuracy of 75.3% after compression to 61.0KB of non-zero parameters. We extend this work to [1].

cs.SD↗

arXiv2022

Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification

While using two-dimensional convolutional neural networks (2D-CNNs) in image processing, it is possible to manipulate domain information using channel statistics, and instance normalization has been a promising way to get domain-invariant features. However, unlike image processing, we analyze that domain-relevant information in an audio feature is dominant in frequency statistics rather than channel statistics. Motivated by our analysis, we introduce Relaxed Instance Frequency-wise Normalization (RFN): a plug-and-play, explicit normalization module along the frequency axis which can eliminate instance-specific domain discrepancy in an audio feature while relaxing undesirable loss of useful discriminative information. Empirically, simply adding RFN to networks shows clear margins compared to previous domain generalization approaches on acoustic scene classification and yields improved robustness for multiple audio devices. Especially, the proposed RFN won the DCASE2021 challenge TASK1A, low-complexity acoustic scene classification with multiple devices, with a clear margin, and RFN is an extended work of our technical report.

cs.SD↗

arXiv2022

Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation

Word-level adversarial attacks have shown success in NLP models, drastically decreasing the performance of transformer-based models in recent years. As a countermeasure, adversarial defense has been explored, but relatively few efforts have been made to detect adversarial examples. However, detecting adversarial examples may be crucial for automated tasks (e.g. review sentiment analysis) that wish to amass information about a certain population and additionally be a step towards a robust defense system. To this end, we release a dataset for four popular attack methods on four datasets and four models to encourage further research in this field. Along with it, we propose a competitive baseline based on density estimation that has the highest AUC on 29 out of 30 dataset-attack-model combinations. Source code is available in https://github.com/anoymous92874838/text-adv-detection.

cs.CL↗

arXiv2022

Deconfinement critical point of heavy quark effective lattice theories

Effective three-dimensional Polyakov loop theories derived from QCD by strong coupling and hopping expansions are valid for heavy quarks and can also be applied to finite chemical potential $μ$, due to their considerably milder sign problem. We apply the Monte-Carlo method to the $N_f=1,2$ effective theories up to $\mathcal{O}(κ^4)$ in the hopping parameter at $μ=0$ to determine the critical quark mass, at which the first-order deconfinement phase transition terminates. The critical end point obtained from the effective theory to order $\mathcal{O}(κ^2)$ agrees well with 4-dimensional QCD simulations with a hopping expanded determinant by the WHOT-QCD collaboration. We also compare with full QCD simulations and thus obtain a measure for the validity of both the strong coupling and the hopping expansion in this regime.

hep-lat↗

arXiv2021

Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization

It is a practical research topic how to deal with multi-device audio inputs by a single acoustic scene classification system with efficient design. In this work, we propose Residual Normalization, a novel feature normalization method that uses frequency-wise normalization % instance normalization with a shortcut path to discard unnecessary device-specific information without losing useful information for classification. Moreover, we introduce an efficient architecture, BC-ResNet-ASC, a modified version of the baseline architecture with a limited receptive field. BC-ResNet-ASC outperforms the baseline architecture even though it contains the small number of parameters. Through three model compression schemes: pruning, quantization, and knowledge distillation, we can reduce model complexity further while mitigating the performance degradation. The proposed system achieves an average test accuracy of 76.3% in TAU Urban Acoustic Scenes 2020 Mobile, development dataset with 315k parameters, and average test accuracy of 75.3% after compression to 61.0KB of non-zero parameters. The proposed method won the 1st place in DCASE 2021 challenge, TASK1A.

cs.SD↗

arXiv2021

Nonperturbative Renormalization of the Quark Chromoelectric Dipole Moment with the Gradient Flow:Power Divergences

The CP-violating quark chromoelectric dipole moment (qCEDM) operator, contributing to the electric dipole moment (EDM), mixes under renormalization and particularly on the lattice with the pseudoscalar density. The mixing coefficient is power-divergent with the inverse lattice spacing squared, $1/a^2$, regardless of the lattice action used. We use the gradient flow to define a multiplicatively renormalized qCEDM operator and study its behavior at small flow time. We determine nonperturbatively the linearly divergent coefficient with the flow time, $1/t$, up to subleading logarithmic corrections, and compare it with the 1-loop perturbative expansion in the bare and renormalized strong coupling. We also discuss the O($a$) improvement of the qCEDM defined at positive flow time.

hep-lat↗

arXiv2021

Vehicle Image Generation Going Well with The Surroundings

Since the generative neural networks have made a breakthrough in the image generation problem, lots of researches on their applications have been studied such as image restoration, style transfer and image completion. However, there has been few research generating objects in uncontrolled real-world environments. In this paper, we propose a novel approach for vehicle image generation in real-world scenes. Using a subnetwork based on a precedent work of image completion, our model makes the shape of an object. Details of objects are trained by an additional colorization and refinement subnetwork, resulting in a better quality of generated objects. Unlike many other works, our method does not require any segmentation layout but still makes a plausible vehicle in the image. We evaluate our method by using images from Berkeley Deep Drive (BDD) and Cityscape datasets, which are widely used for object detection and image segmentation problems. The adequacy of the generated images by the proposed method has also been evaluated using a widely utilized object detection algorithm and the FID score.

cs.CV↗

arXiv2021

PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation

As edge devices become prevalent, deploying Deep Neural Networks (DNN) on edge devices has become a critical issue. However, DNN requires a high computational resource which is rarely available for edge devices. To handle this, we propose a novel model compression method for the devices with limited computational resources, called PQK consisting of pruning, quantization, and knowledge distillation (KD) processes. Unlike traditional pruning and KD, PQK makes use of unimportant weights pruned in the pruning process to make a teacher network for training a better student network without pre-training the teacher model. PQK has two phases. Phase 1 exploits iterative pruning and quantization-aware training to make a lightweight and power-efficient model. In phase 2, we make a teacher network by adding unimportant weights unused in phase 1 to a pruned network. By using this teacher network, we train the pruned network as a student network. In doing so, we do not need a pre-trained teacher network for the KD framework because the teacher and the student networks coexist within the same network. We apply our method to the recognition model and verify the effectiveness of PQK on keyword spotting (KWS) and image recognition.

cs.LG↗

arXiv2021

Chiral symmetry and taste symmetry from the eigenvalue spectrum of staggered Dirac operators

We investigate general properties of the eigenvalue spectrum for improved staggered quarks. We introduce a new chirality operator $[γ_5 \otimes 1]$ and a new shift operator $[1 \otimes ξ_5]$, which respect the same recursion relation as the $γ_5$ operator in the continuum. Then we show that matrix elements of the chirality operator sandwiched between two eigenstates of the staggered Dirac operator are related to those of the shift operator by the Ward identity of the conserved $U(1)_A$ symmetry of staggered fermion actions. We perform a numerical study in quenched QCD using HYP staggered quarks to demonstrate the Ward identity. We introduce a new concept of leakage patterns which collectively represent the matrix elements of the chirality operator and the shift operator sandwiched between two eigenstates of the staggered Dirac operator. The leakage pattern provides a new method to identify zero modes and non-zero modes in the Dirac eigenvalue spectrum. This method is as robust as the spectral flow method but requires much less computing power. Analysis using a machine learning technique confirms that the leakage pattern is universal, since the staggered Dirac eigenmodes on normal gauge configurations respect it. In addition, the leakage pattern can be used to determine a ratio of renormalization factors as a by-product. We conclude that it might be possible and realistic to measure the topological charge $Q$ using the Atiya-Singer index theorem and the leakage pattern of the chirality operator in the staggered fermion formalism.

hep-lat↗

arXiv2021

Prototype-based Personalized Pruning

Nowadays, as edge devices such as smartphones become prevalent, there are increasing demands for personalized services. However, traditional personalization methods are not suitable for edge devices because retraining or finetuning is needed with limited personal data. Also, a full model might be too heavy for edge devices with limited resources. Unfortunately, model compression methods which can handle the model complexity issue also require the retraining phase. These multiple training phases generally need huge computational cost during on-device learning which can be a burden to edge devices. In this work, we propose a dynamic personalization method called prototype-based personalized pruning (PPP). PPP considers both ends of personalization and model efficiency. After training a network, PPP can easily prune the network with a prototype representing the characteristics of personal data and it performs well without retraining or finetuning. We verify the usefulness of PPP on a couple of tasks in computer vision and Keyword spotting.

cs.LG↗

arXiv2020

Position-based Scaled Gradient for Model Quantization and Pruning

We propose the position-based scaled gradient (PSG) that scales the gradient depending on the position of a weight vector to make it more compression-friendly. First, we theoretically show that applying PSG to the standard gradient descent (GD), which is called PSGD, is equivalent to the GD in the warped weight space, a space made by warping the original weight space via an appropriately designed invertible function. Second, we empirically show that PSG acting as a regularizer to a weight vector is favorable for model compression domains such as quantization and pruning. PSG reduces the gap between the weight distributions of a full-precision model and its compressed counterpart. This enables the versatile deployment of a model either as an uncompressed mode or as a compressed mode depending on the availability of resources. The experimental results on CIFAR-10/100 and ImageNet datasets show the effectiveness of the proposed PSG in both domains of pruning and quantization even for extremely low bits. The code is released in Github.

cs.CV↗

arXiv2020

The SU(3) spin model with chemical potential by series expansion techniques

The $SU(3)$ spin model with chemical potential corresponds to a simplified version of QCD with static quarks in the strong coupling regime. It has been studied previously as a testing ground for new methods aiming to overcome the sign problem of lattice QCD. In this work we show that the equation of state and the phase structure of the model can be determined to reasonable accuracy by a linked cluster expansion. In particular, we compute the free energy to 14-th order in the nearest neighbour coupling. The resulting predictions for the equation of state and the location of the critical end point agree with numerical determinations to ${\cal O}(1\%)$ and ${\cal O}(10\%)$, respectively. While the accuracy for the critical couplings is still limited at the current series depth, the approach is equally applicable at zero and non-zero imaginary or real chemical potential, as well as to effective QCD Hamiltonians obtained by strong coupling and hopping expansions.

hep-lat↗