arXiv subjects

Chen Sun

Publications and source records attributed to Chen Sun.

At least 163 records · Page 9Linked to original sources

arXiv2023

Eliminating Leakage in Volatile Memory with Anti-Ferroelectric Transistors

Cache serves as a temporary data memory module in many general-purpose processors and domain-specific accelerators. Its density, power, speed, and reliability play a critical role in enhancing the overall system performance and quality of service. Conventional volatile memories, including static random-access memory (SRAM) and embedded dynamic random-access memory (eDRAM) in the complementary metal-oxide-semiconductor technology, have high performance and good reliability. However, the inherent leakage in both SRAM and eDRAM hinders further improvement towards smaller feature sizes and higher energy efficiency. Although the emerging nonvolatile memories can eliminate the leakage efficiently, the penalties of lower speed and degraded reliability are significant. This article reveals a new opportunity towards leakage-free volatile static memory beyond the known paradigms of existing volatile and nonvolatile memories. By engineering a double-well energy landscape with the assistance of a clamping voltage bias, leakage-free and refresh-free state retention of volatile memory is achieved for the first time. This new memory is highlighted by both the ultra-low leakage of nonvolatile memories and the speed, energy, and reliability advantages of volatile memories. A proof-of-concept memory is demonstrated using in-house anti-ferroelectric field-effect transistors (AFeFETs), delivering an extrapolated endurance of about 1012 cycles, a retention time of over 10 years, and no subthreshold channel leakage current. Such a new concept of AFeFET-based memory enables an improved balance between density, power, and reliability beyond all existing memory solutions.

cs.ET↗

arXiv2022

Computational Associative Memory with Amorphous InGaZnO Channel 3D NAND-Compatible FG Transistors

3D NAND enables continuous NAND density and cost scaling beyond conventional 2D NAND. However, its poly-Si channel suffers from low mobility, large device variations, and instability caused by grain boundaries. Here, we overcome these drawbacks by introducing an amorphous indium-gallium-zinc-oxide (a-IGZO) channel, which has the advantages of ultra-low OFF current, back-end-of-line compatibility, higher mobility and better uniformity than poly-Si, and free of grain boundaries due to the amorphous nature. Ultra-scaled floating-gate (FG) transistors with a channel length of 60 nm are reported, achieving the highest ON current of 127 uA/um among all reported a-IGZO-based flash devices for high-density, low-power, and high-performance 3D NAND applications. Furthermore, a non-volatile and area-efficient ternary content-addressable memory (TCAM) with only two a-IGZO FG transistors is experimentally demonstrated. Array-level simulations using experimentally calibrated models show that this design achieves at least 240x array-size scalability and 2.7-fold reduction in search energy than 16T-CMOS, 2T2R, and 2FeFET TCAMs.

cond-mat.mes-hall↗

arXiv2022

Many-body slow quench dynamics and nonadiabatic characterization of topological phases

Previous studies have shown that the bulk topology of single-particle systems can be captured by the band inversion surface or by the spin inversion surface emerged on the time-averaged spin polarization. Most of the studies, however, are based on the single-particle picture even though the systems are fermionic and of multi-bands. Here, we study the many-body quench dynamics of topological systems with all the valence bands fully occupied, and show that the concepts of band inversion surface and spin inversion surface are still valid. More importantly, the many-body quench dynamics is shown to be reduced to a nontrivial three-level Landau-Zener model, which can be solved exactly. Based on the analytical results, the topological spin texture revealed by the time-averaged spin polarization can be applied to characterize the bulk topology and thus provides a direct comparison for future experiments.

cond-mat.str-el↗

arXiv2022

Attention Bottlenecks for Multimodal Fusion

Humans perceive the world by concurrently processing and fusing high-dimensional inputs from multiple modalities such as vision and audio. Machine perception models, in stark contrast, are typically modality-specific and optimised for unimodal benchmarks, and hence late-stage fusion of final representations or predictions from each modality (`late-fusion') is still a dominant paradigm for multimodal video classification. Instead, we introduce a novel transformer based architecture that uses `fusion bottlenecks' for modality fusion at multiple layers. Compared to traditional pairwise self-attention, our model forces information between different modalities to pass through a small number of bottleneck latents, requiring the model to collate and condense the most relevant information in each modality and only share what is necessary. We find that such a strategy improves fusion performance, at the same time reducing computational cost. We conduct thorough ablation studies, and achieve state-of-the-art results on multiple audio-visual classification benchmarks including Audioset, Epic-Kitchens and VGGSound. All code and models will be released.

cs.CV↗

arXiv2022

Dark Matter In Extreme Astrophysical Environments

Exploring dark matter via observations of extreme astrophysical environments -- defined here as heavy compact objects such as white dwarfs, neutron stars, and black holes, as well as supernovae and compact object merger events -- has been a major field of growth since the last Snowmass process. Theoretical work has highlighted the utility of current and near-future observatories to constrain novel dark matter parameter space across the full mass range. This includes gravitational wave instruments and observatories spanning the electromagnetic spectrum, from radio to gamma-rays. While recent searches already provide leading sensitivity to various dark matter models, this work also highlights the need for theoretical astrophysics research to better constrain the properties of these extreme astrophysical systems. The unique potential of these search signatures to probe dark matter adds motivation to proposed next-generation astronomical and gravitational wave instruments.

hep-ph↗

arXiv2022

A Novel Non-Volatile Inverter-based CiM: Continuous Sign Weight Transition and Low Power on-Chip Training

In this work, we report a novel design, one-transistor-one-inverter (1T1I), to satisfy high speed and low power on-chip training requirements. By leveraging doped HfO2 with ferroelectricity, a non-volatile inverter is successfully demonstrated, enabling desired continuous weight transition between negative and positive via the programmable threshold voltage (VTH) of ferroelectric field-effect transistors (FeFETs). Compared with commonly used designs with the similar function, 1T1I uniquely achieves pure on-chip-based weight transition at an optimized working current without relying on assistance from off-chip calculation units for signed-weight comparison, facilitating high-speed training at low power consumption. Further improvements in linearity and training speed can be obtained via a two-transistor-one-inverter (2T1I) design. Overall, focusing on energy and time efficiencies, this work provides a valuable design strategy for future FeFET-based computing-in-memory (CiM).

cond-mat.mes-hall↗

arXiv2022

No-go rules for multitime Landau-Zener models

Multitime Landau-Zener (MTLZ) model is a class of exactly solvable quantum many-body models which is multitstate and multitime generalization of the two-state Landau-Zener model. Currently discovered MTLZ models include the "hypercubes", the "fans" and their direct product models. In this work, we prove two no-go rules, named the "no $K_{3,3}$" rule and the "no $1221$" rule, which forbid the existence of exact solutions for models with certain structures of interactions. We further apply these rules to show that for models with no more than $9$ states, besides the models mentioned above there are no other MTLZ models. We also propose a scheme to systematically classify cases that could possibly host MTLZ models. Our work could serve as a guideline to search for new exactly solvable models within the MTLZ class.

quant-ph↗

arXiv2022

A New Knowledge Distillation Network for Incremental Few-Shot Surface Defect Detection

Surface defect detection is one of the most essential processes for industrial quality inspection. Deep learning-based surface defect detection methods have shown great potential. However, the well-performed models usually require large training data and can only detect defects that appeared in the training stage. When facing incremental few-shot data, defect detection models inevitably suffer from catastrophic forgetting and misclassification problem. To solve these problems, this paper proposes a new knowledge distillation network, called Dual Knowledge Align Network (DKAN). The proposed DKAN method follows a pretraining-finetuning transfer learning paradigm and a knowledge distillation framework is designed for fine-tuning. Specifically, an Incremental RCNN is proposed to achieve decoupled stable feature representation of different categories. Under this framework, a Feature Knowledge Align (FKA) loss is designed between class-agnostic feature maps to deal with catastrophic forgetting problems, and a Logit Knowledge Align (LKA) loss is deployed between logit distributions to tackle misclassification problems. Experiments have been conducted on the incremental Few-shot NEU-DET dataset and results show that DKAN outperforms other methods on various few-shot scenes, up to 6.65% on the mean Average Precision metric, which proves the effectiveness of the proposed method.

cs.CV↗

arXiv2022

TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency

YouTube users looking for instructions for a specific task may spend a long time browsing content trying to find the right video that matches their needs. Creating a visual summary (abridged version of a video) provides viewers with a quick overview and massively reduces search time. In this work, we focus on summarizing instructional videos, an under-explored area of video summarization. In comparison to generic videos, instructional videos can be parsed into semantically meaningful segments that correspond to important steps of the demonstrated task. Existing video summarization datasets rely on manual frame-level annotations, making them subjective and limited in size. To overcome this, we first automatically generate pseudo summaries for a corpus of instructional videos by exploiting two key assumptions: (i) relevant steps are likely to appear in multiple videos of the same task (Task Relevance), and (ii) they are more likely to be described by the demonstrator verbally (Cross-Modal Saliency). We propose an instructional video summarization network that combines a context-aware temporal video encoder and a segment scoring transformer. Using pseudo summaries as weak supervision, our network constructs a visual summary for an instructional video given only video and transcribed speech. To evaluate our model, we collect a high-quality test set, WikiHow Summaries, by scraping WikiHow articles that contain video demonstrations and visual depictions of steps allowing us to obtain the ground-truth summaries. We outperform several baselines and a state-of-the-art video summarization model on this new benchmark.

cs.CV↗

arXiv2022

Beyond Transfer Learning: Co-finetuning for Action Localisation

Transfer learning is the predominant paradigm for training deep networks on small target datasets. Models are typically pretrained on large ``upstream'' datasets for classification, as such labels are easy to collect, and then finetuned on ``downstream'' tasks such as action localisation, which are smaller due to their finer-grained annotations. In this paper, we question this approach, and propose co-finetuning -- simultaneously training a single model on multiple ``upstream'' and ``downstream'' tasks. We demonstrate that co-finetuning outperforms traditional transfer learning when using the same total amount of data, and also show how we can easily extend our approach to multiple ``upstream'' datasets to further improve performance. In particular, co-finetuning significantly improves the performance on rare classes in our downstream task, as it has a regularising effect, and enables the network to learn feature representations that transfer between different datasets. Finally, we observe how co-finetuning with public, video classification datasets, we are able to achieve state-of-the-art results for spatio-temporal action localisation on the challenging AVA and AVA-Kinetics datasets, outperforming recent works which develop intricate models.

cs.CV↗

arXiv2022

Do Trajectories Encode Verb Meaning?

Distributional models learn representations of words from text, but are criticized for their lack of grounding, or the linking of text to the non-linguistic world. Grounded language models have had success in learning to connect concrete categories like nouns and adjectives to the world via images and videos, but can struggle to isolate the meaning of the verbs themselves from the context in which they typically occur. In this paper, we investigate the extent to which trajectories (i.e. the position and rotation of objects over time) naturally encode verb semantics. We build a procedurally generated agent-object-interaction dataset, obtain human annotations for the verbs that occur in this data, and compare several methods for representation learning given the trajectories. We find that trajectories correlate as-is with some verbs (e.g., fall), and that additional abstraction via self-supervised pretraining can further capture nuanced differences in verb meaning (e.g., roll vs. slide).

cs.CL↗

arXiv2022

AVATAR: Unconstrained Audiovisual Speech Recognition

Audio-visual automatic speech recognition (AV-ASR) is an extension of ASR that incorporates visual cues, often from the movements of a speaker's mouth. Unlike works that simply focus on the lip motion, we investigate the contribution of entire visual frames (visual actions, objects, background etc.). This is particularly useful for unconstrained videos, where the speaker is not necessarily visible. To solve this task, we propose a new sequence-to-sequence AudioVisual ASR TrAnsformeR (AVATAR) which is trained end-to-end from spectrograms and full-frame RGB. To prevent the audio stream from dominating training, we propose different word-masking strategies, thereby encouraging our model to pay attention to the visual stream. We demonstrate the contribution of the visual modality on the How2 AV-ASR benchmark, especially in the presence of simulated noise, and show that our model outperforms all other prior work by a large margin. Finally, we also create a new, real-world test bed for AV-ASR called VisSpeech, which demonstrates the contribution of the visual modality under challenging audio conditions.

cs.CV↗

arXiv2022

Spin Accumulation and Longitudinal Spin Diffusion of Magnets

We extend to the longitudinal component of the magnetization the spintronics idea that a magnet near equilibrium can be described by two magnetic variables. One is the usual magnetization $\vec{M}$. The other is the non-equilibrium quantity $\vec{m}$, called the spin accumulation, by which the non-equilibrium spin current can be transported. $\vec{M}$ represents a correlated distribution of a very large number of degrees of freedom, as expressed in some equilibrium distribution function for the excitations; we therefore forbid $\vec{M}$ to diffuse, but we permit $\vec{M}$ to decay. On the other hand, we permit $\vec{m}$, due to spin excitations, to both diffuse and decay. For this physical picture, diffusion from a given region occurs by decay of $\vec{M}$ to $\vec{m}$, then by diffusion of $\vec{m}$, and finally by decay of $\vec{m}$ to $\vec{M}$ in another region. This somewhat slows down the diffusion process. Restricting ourselves to the longitudinal variables $M$ and $m$ with equilibrium properties $M_{eq}=M_{0}+χ_{M\parallel}H$ and $m_{eq}=0$, we argue that the effective energy density must include a new, thermodynamically required exchange constant $λ_{M}=-1/χ_{M\parallel}$. We then develop the macroscopic equations by applying Onsager's irreversible thermodynamics, and use the resulting equations to study the space and time response. At fixed real frequency $ω$ there is, as usual, a single pair of complex wavevectors $\pm k$ but with an unusual dependence on $ω$. At fixed real wavevector, there are two decay constants, as opposed to one in the usual case. Extending the idea that non-equilibrium diffusion in other ordered systems involves a non-equilibrium quantity, this work suggests that in a superconductor the order parameter $Δ$ can decay but not diffuse, but a non-equilibrium gap-like $δ$, due to pair excitations, can both decay and diffuse.

cond-mat.mes-hall↗

arXiv2022

Multiview Transformers for Video Recognition

Video understanding requires reasoning at multiple spatiotemporal resolutions -- from short fine-grained motions to events taking place over longer durations. Although transformer architectures have recently advanced the state-of-the-art, they have not explicitly modelled different spatiotemporal resolutions. To this end, we present Multiview Transformers for Video Recognition (MTV). Our model consists of separate encoders to represent different views of the input video with lateral connections to fuse information across views. We present thorough ablation studies of our model and show that MTV consistently performs better than single-view counterparts in terms of accuracy and computational cost across a range of model sizes. Furthermore, we achieve state-of-the-art results on six standard datasets, and improve even further with large-scale pretraining. Code and checkpoints are available at: https://github.com/google-research/scenic/tree/main/scenic/projects/mtv.

cs.CV↗

arXiv2022

Galactic rotation curves versus ultralight dark matter: A systematic comparison with SPARC data

We look for and place observational constraints on the imprint of ultralight dark matter (ULDM) soliton cores in rotation-dominated galaxies. Extending previous analyses, we find a conservative constraint which disfavors the soliton-host halo relation found in some numerical simulations over a broad range in the ULDM particle mass $m$. Combining the observational constraints with theoretical arguments for the efficiency of soliton formation via gravitational dynamical relaxation, and assuming that the soliton-halo relation is correct, our results disfavor ULDM from comprising 100\% of the total cosmological dark matter in the range $10^{-24}~{\rm eV}\lesssim m\lesssim10^{-20}~{\rm eV}$. The constraints probe the ULDM fraction down to $f\lesssim0.3$ of the total dark matter.

hep-ph↗

arXiv2022

Axion Echos from the Supernova Graveyard

Stimulated decays of axion dark matter, triggered by a source in the sky, could produce a photon flux along the continuation of the line of sight, pointing backward to the source. The strength of this so-called axion "echo" signal depends on the entire history of the source and could still be strong from sources that are dim today but had a large flux density in the past, such as supernova remnants (SNRs). This echo signal turns out to be most observable in the radio band. We study the sensitivity of radio telescopes such as the Square Kilometer Array (SKA) to echo signals generated by SNRs that have already been observed, and show that SKA could reach axion-photon couplings of order $g_{aγγ} \sim \mathcal{O}(10^{-11}) \,\mathrm{GeV}^{-1}$ for axion masses $m_a \lesssim 10^{-5}\;\mathrm{eV}$. In addition, we show projections of the detection reach for signals coming from old SNRs and from newly born supernovae that could be detected in the future. Intriguingly, an observable echo signal could come from old "ghost" SNRs which were very bright in the past but are now so dim that they haven't been observed.

hep-ph↗

arXiv2022

Learning Audio-Video Modalities from Image Captions

A major challenge in text-video and text-audio retrieval is the lack of large-scale training data. This is unlike image-captioning, where datasets are in the order of millions of samples. To close this gap we propose a new video mining pipeline which involves transferring captions from image captioning datasets to video clips with no additional manual effort. Using this pipeline, we create a new large-scale, weakly labelled audio-video captioning dataset consisting of millions of paired clips and captions. We show that training a multimodal transformed based model on this data achieves competitive performance on video retrieval and video captioning, matching or even outperforming HowTo100M pretraining with 20x fewer clips. We also show that our mined clips are suitable for text-audio pretraining, and achieve state of the art results for the task of audio retrieval.

cs.CV↗

arXiv2022

Gravitational Waves from Incomplete Inflationary Phase Transitions

We study the observable implications of an incomplete first order phase transition during inflation. In such a phase transition, the nucleated bubbles do not percolate and instead are continuously produced until the onset of reheating. The process creates an inhomogeneity with a distinct power spectrum that depends on both the physics of the phase transition and the inflationary dynamics. Upon horizon re-entry, this spectrum generates gravitational waves through non-linear effects. This stochastic gravitational wave background is predicted to have unique signatures that may be detectable by future experiments spanning a wide frequency range. The discovery of such a gravitational wave signal would shed a light on the detailed dynamics of inflation.

hep-ph↗