arXiv subjects

Shimeng Yu

Publications and source records attributed to Shimeng Yu.

At least 19 recordsLinked to original sources

arXiv2026

p-DCIM: A 28nm Probabilistic SRAM DCIM based Ising Machine for Near-Memory Context Selection in LLM-RAG Pipeline

We present a fabricated SRAM DCIM Ising machine that performs set-level context selection for large language model (LLM) with retrieval-augmented generation (RAG). In RAG, the evidence-selection stage is increasingly delegated to repeated language-model reranking, which adds substantial energy to the inference pipeline. We reformulate evidence selection as a quadratic unconstrained binary optimization (QUBO) problem and solve it on a dual-mode digital compute-in-memory (DCIM) macro: the same multiply-and-accumulate datapath computes the embedding-similarity scores that populate the QUBO coefficients and evaluates the per-spin Hamiltonian differences that drive the anneal. The annealing temperature is set by a digital plane-count control along with the SRAM pseudo-read random bit flips, giving graded cooling entirely from on-chip digital configuration. The macro is taped-out in a 28 nm process [0.16 mm2, 0.41 mW] and is proposed to be integrated as part of the logic base-die in hybrid high-bandwidth memory (HBM) and high-bandwidth flash (HBF) stacks in a GPU package, where only the selected passage identifiers return to the host. Replacing the language-model selection pass with the near-memory Ising solver yields a macro normalized selection-energy estimate more than six orders of magnitude lower than the LLM-based baseline and reduces the evaluated total energy per query by 67%, while maintaining comparable token-level F1 on the evaluated examples.

cs.AR↗

arXiv2026

An Optical KV Cache Network for Disaggregated LLM Serving

In disaggregated LLM serving, each request's KV cache moves from a prefill GPU to a decode GPU before the response begins, and a coding-agent turn often moves the KV cache of its whole long context. Behind an oversubscribed packet fabric, this transfer adds to the time to first token (TTFT). An optical circuit switch (OCS) can give every transfer a full-rate circuit for less than an oversubscribed packet fabric costs, but a mechanical OCS needs tens of milliseconds to reconfigure. We hide this delay inside the prefill. A reservation scheduler commits each circuit while the prefill is still computing, so the circuit is ready when the KV cache is, and each GPU reaches the OCS through its own wavelength-multiplexed (WDM) optics on one fiber pair. In simulations of GPU clusters on real serving traces with prefill times measured on H100 GPUs, this optical KV cache network matches 400G packet fabrics with adaptive routing at 0.55 to 0.78 times the cost of a 2:1 fabric and tolerates the 50 ms reconfiguration delay of a 384-port mechanical OCS.

cs.NI↗

arXiv2026

Direct Observation of Nanoscale Polarization Switching in HZO Ferroelectrics Using STEM-EBIC

We report the experimental observation of discrete ferroelectric switching down to 0.475 nm x 0.475 nm in HfO2-based ferroelectrics using correlated STEM electron beam induced current (EBIC) imaging and nanoscale voltage pulsing. These high-resolution measurements reveal single-domain switching events characterized by abrupt, bi-state transitions with no intermediate polarization states. By mapping polarization-voltage behavior and switching kinetics across thousands of domains, we extract spatially resolved switching thresholds (Emin), switching speeds (tsw), and remanent polarization (2Pr), uncovering robust scaling trends. Larger domains exhibit nearly twice the polarization and faster switching despite higher Emin, while smaller domains switch more slowly at the same field. Sub-domain imaging enables quantification of variation scaling: polarization variation increases linearly logarithmically with area, with inter-domain variation dominating intra-domain variation. Importantly, we show that multi-bit operation arises from the statistical distribution of domain-level switching, rather than partial polarization, enabling distinct and reliable states in devices as small as 0.01 um2. These findings provide a foundational experimental framework for designing scalable, multi-bit ferroelectric memory based on grain- and domain-level heterogeneity.

cond-mat.mes-hall↗

arXiv2026

Multi-bit Ferroelectric-NAND for High-throughput Massive Database Search

The growing demand for large-scale database search in data-intensive applications, ranging from proteomics to autonomous systems, has exposed fundamental limitations in von Neumann architectures due to memory bandwidth and energy constraints. Hyperdimensional (HD) computing offers a robust and parallelizable framework for such tasks, but its practical implementation remains challenged by high memory demands. Ultra-high-density, energy-efficient ferroelectric NAND (FE-NAND) memory provides a potential solution by enabling in-situ computation. We fabricate quad-level FE-NAND strings with wide memory windows, disturb resilience, and robust retention. Using these planar FE-NAND strings as building blocks, we experimentally demonstrate in-situ multi-level cell (MLC) dot product operations at the single-cell level and, using experimentally calibrated physics-based simulations, demonstrate Hamming similarity calculations between reference and query hypervectors (HV). This platform leverages the inherent error tolerance of HD computing to achieve >90% search accuracy even at high logic levels (TLC, QLC). When benchmarked on Open Modification Search (OMS) tasks in proteomics with TB-scale datasets, our system shows nearly 1,000x speedup and over 10,000x energy efficiency improvement compared to incumbent solutions. These results establish FE-NAND as a viable in-storage compute architecture for large-scale, high-dimensional data processing.

cond-mat.mes-hall↗

arXiv2026

Cross-Layer Analysis of Thermal Tuning Stalls in Wafer-Scale Optical Interconnects for LLM MoE Training

Mixture-of-experts (MoE) training depends heavily on all-to-all communication, which wafer-scale optical interconnects with dense wavelength-division multiplexing can serve. Their microring resonators rely on thermo-optic tuning, while MoE compute bursts swing the photonic-layer temperature by about 10 K within tens of milliseconds. This paper quantifies the communication stall by coupling packet-level network simulation, transient thermal simulation of a 3D-stacked GPU and photonic die, and a ring detuning criterion, and by feeding the stall back into the network timeline until both agree. Including this feedback, a tracking loop at the measured 5 nm/s lengthens the training iteration of Mixtral 8x7B and LLaMA-MoE 6.7B by factors of about 1.16 and 1.25 at full model depth. Measured H100 die temperatures match the modeled swing of 300 ms bursts within 15%. A loop slewing eight times faster, a heater driven at each kernel launch, or a dummy load near 75% of peak power removes the stall. An athermalized lithium-niobate ring with a non-volatile ferroelectric setpoint removes it with no holding power or fast loop, which makes it suitable for wafer-scale optical interconnects.

cs.AR↗

arXiv2026

Experiments and Modeling of Defect Dynamics and BTI Behavior in Doped InO TFTs during $400^\circ$C Post-Processing Forming Gas Annealing

We investigate the impact of a monolithic three-dimensional (M3D) integration process-critical $400^\circ$C post-processing forming gas anneal (FGA) on the electrical performance, reliability, and defect evolution of oxide-channel thin-film transistors (TFTs), combining systematic experiments with density-functional-theory (DFT)-based liquid-quench molecular-dynamics (MD) simulations. Indium tungsten oxide (IWO) TFTs are employed as a model system and encapsulated with a thin 3 nm Al$_2$O$_3$ / 3 nm HfO$_2$ hybrid layer that effectively suppresses external hydrogen ingress. We reveal a non-monotonic evolution of device behavior during FGA, governed by initial densification followed by partial crystallization. Short-duration FGA (10 min) induces channel densification and the formation of shallow, delocalized defect states, leading to pronounced positive bias temperature instability (PBTI) degradation and the emergence of a characteristic transfer-curve "kink." With prolonged annealing (>40 min), partial crystallization of the oxide channel occurs, stabilizing hydrogen in deep, localized defect states, suppressing hydrogen mobility, and restoring device reliability. As a result, the PBTI shift is reduced to 10.4 mV after 2000 seconds of stress, accompanied by complete elimination of the transfer-curve kink. These findings provide a mechanistic understanding of hydrogen-defect interactions during high-temperature post-processing FGA and demonstrate that appropriate hydrogen-blocking encapsulation enables oxide-channel TFT integration without compromising electrical performance or reliability.

cond-mat.mtrl-sci↗

arXiv2026

Agentic TCAD Calibration Workflow for Oxide Semiconductor Transistors

Experimental TCAD calibration is essential for predictive technology modeling of emerging oxide semiconductor transistors. However, it remains time-consuming and expert dependent because of model ambiguity. Multiple physical models and parameter sets can reproduce the same measured transfer characteristics, while local fitting alone cannot uniquely identify the underlying device physics. We present the first demonstration of an agentic TCAD calibration workflow for a fabricated bottom-gate In--W--O (BG-IWO) transistor. Starting from the measured transfer curve and device information, the workflow uses measurement--TCAD residuals and local sensitivity tests to select bounded parameter corrections or evaluate additional physical models, and accept only updates that improve device metrics. The LLM agent orchestrates the workflow, while Sentaurus governs the device physics. For the 2\%-W reference device, five agent-suggested updates yield a fixed calibrated model, reducing the multi-metric device objective $J$ by 14.3$\times$. Maximum $V_{\mathrm{th}}$/$I_{\mathrm{on}}$ errors are 36.1~mV/0.022 decade for varying-drain-bias tests and 46.2~mV/0.062 decade for varying-channel-length tests, demonstrating model transferability across bias and geometry rather than a local parameter fit. W-composition tests provide process-sensitive insight. This agentic workflow provides a faster route to model development for emerging device technologies.

physics.ins-det↗

arXiv2026

Hardware Acceleration of Block-Diffusion LLM for Edge Devices

Single-stream (batch-one) edge inference cannot amortize weight traffic across requests. Full-attention diffusion LLMs recompute the entire sequence at every step; native block diffusion makes completed blocks immutable and exactly cacheable, yet refinement still streams prefix KV and FFN weights. We co-design WIFiV-LPDDR, a wide-I/O LPDDR system for precision-tagged reads, BRQ-KV for a canonical low-rank-plus-INT8-residual prefix with query-dependent per-entry precision, and DAT-FFN for drift-mapped canonical replacement, adjacent-stage-corrected low-bit delta, or cached-state carry while keeping live activations unquantized. Both map to an input-stationary mixed-precision systolic array. For the evaluated 1.5B/7B models on modeled Jetson-class platforms, the full stack provides arithmetic-mean energy-reduction factors of 3.79x/3.96x and arithmetic-mean latency speedups of 2.88x/4.44x at the reported DAT-FFN settings; every corresponding compressed model-benchmark score drops by less than one absolute percentage point from its baseline.

cs.AR↗

arXiv2026

LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving

The energy consumption of Large Language Model (LLM) serving is becoming a major system challenge as deployment scales, driven by hardware power and thermal constraints and rising electricity costs. A key contributor to chip energy dissipation is data movement between limited on-chip cache and off-chip High Bandwidth Memory (HBM). Meanwhile, emerging memory technologies such as monolithic 3D (M3D) integration of cache memories at the Back-End-Of-Line (BEOL) of logic chips enable larger and denser on-chip memories, creating new opportunities to reduce costly off-chip traffic. However, it remains unclear whether continuously scaling on-chip memory using emerging technologies can effectively improve the energy efficiency of LLM serving. To address this gap, we develop LLMET (LLM with Emerging Technology), a validated cross-layer simulation framework, and conduct a comprehensive study on the impact of large-capacity on-chip memory technologies across a broad range of models, applications and platforms. Utilizing M3D technology to expand the L2 cache from 40MB to 1GB yields a 44% reduction in chip energy during the Llama3.1-70B prefill phase with a 16K context window, based on LLMET simulation on a dual NVIDIA A100 GPU setup. On the 8x NVIDIA B200-like platform, extending the L2 cache from 128MB to 4GB saves the prefill energy by up to 24%. For the edge platform and workloads, the decode energy saving reaches 30% when increasing the 8MB cache size to 256MB. These results highlight the promise of ultra-large on-chip memories for energy-efficient LLM serving systems.

cs.AR↗

arXiv2026

Cross-Domain Acceleration of Open Modification Search: From Commodity Platforms to Emerging Memory and Storage Devices

Open modification search (OMS) in mass spectrometry (MS) is a data-intensive workload whose performance is dominantly limited by reference data movement rather than computation. Prior OMS accelerators have largely been evaluated in isolation, making it difficult to understand system-level trade-offs across platforms. This paper presents the first workload-driven, cross-platform survey of accelerators for MS search by studying not only commodity platforms, but also emerging memory- and storage-centric architectures, including GPUs, near-storage FPGAs, DRAM near-memory processing, ReRAM/PCM in-memory processing, and 3D NAND/FeNAND in-storage processing, under consistent algorithmic and accuracy assumptions. Leveraging a binary hyperdimensional computing (HDC)-based OMS formulation that reduces similarity evaluation to lightweight bitwise primitives and tolerates device-level non-idealities, we enable a robust execution on memory-centric architectures despite device-level non-idealities and limited computing capability. Overall, this study identifies memory- and storage-centric architectures as a key architectural breakthrough for large-scale, high-speed search acceleration, delivering up to >100x speedup and >40,000x improvement in energy efficiency.

cs.AR↗

arXiv2026

Probabilistic Memory for Trustworthy Edge Intelligence

Probabilistic computation plays an important role in trustworthy edge intelligence to quantify uncertainty, enhance robustness, reconstruct data, and protect privacy, but its adoption is limited by the orders-of-magnitude data throughput gap between Gaussian random number generation (GRNG) and computation, as well as instruction overhead. This paper introduces probabilistic memory (p-MEM), a unified memory primitive that stores distribution parameters, such as mean and standard deviation, and samples directly at the native memory bandwidth, where deterministic data becomes the zero-variance special case. Using a layout-validated p-MEM simulator, we comprehensively explore device choices, memory specifications, and technology nodes, showing that p-MEM can achieve more than 1000 GSa/s/mm^2 GRNG throughput, including memory-array access. Integrated into CPU/GPU systems, p-MEM reduces instruction count by up to 2.19x/4.37x, sampling latency by 562x/3.45x, and energy by 295.5x/3.53x for Bayesian neural network workloads, providing a scalable hardware substrate for trustworthy probabilistic AI.

cs.AR↗

arXiv2026

Thermal Processing Limits in Oxide-Channel Ferroelectric Field Effect Transistors

In this work, we report a systematic study of the impact of high-temperature post-capping thermal annealing on the memory characteristics of Oxide-semiconductor channel ferroelectric field-effect transistors (OS-FeFETs). Using an identical engineered ferroelectric gate stack 8nm Hf0.5Zr0.5O2 (HZO) / 3 nm Al2O3 / 8 nm HZO (8/3/8) and a hybrid capping layer (3 nm HfO2 + 3 nm Al2O3), 10 percent Ga doped InO (IGO) channel and 4 percent W doped InO (IWO) channel FeFETs remain functional after annealing at temperatures up to 650 C for durations of up to 30 min and 10 min, respectively; further annealing results in irreversible loss of conduction and device failure. Detailed electrical analysis reveals that the MW enhancement originates from a preferential positive shift in the erased-state threshold voltage, while the programmed-state threshold voltage remains comparatively stable. Grazing-incidence X-ray diffraction measurements further indicate structural evolution in the IWO and IGO oxide channels with increasing annealing temperature, supporting the observed electrical trends.

cond-mat.mtrl-sci↗

arXiv2026

Physics-informed AI Accelerated Retention Analysis of Ferroelectric Vertical NAND: From Day-Scale TCAD to Second-Scale Surrogate Model

Ferroelectric field-effect transistors (FeFET)-based vertical NAND (Fe-VNAND) has emerged as a promising candidate to overcome z-scaling limitations with lower programming voltages. However, the data retention of 3D Fe-VNAND is hindered by the complex interaction between charge detrapping and ferroelectric depolarization. Developing optimized device designs requires exploring an extensive parameter space, but the high computational cost of conventional Technology Computer-Aided Design (TCAD) tools makes such wide-scale optimization impractical. To overcome these simulation barriers, we present a Physics-Informed Neural Operator (PINO)-based AI surrogate model designed for high-efficiency prediction of threshold voltage (Vth) shifts and retention behavior. By embedding fundamental physical principles into the learning architecture, our PINO framework achieves a speedup exceeding 10000x compared to TCAD while maintaining physical accuracy. The resulting surrogate provides a physics-consistent data engine for compact model parameter extraction and look-up-table (LUT) generation, directly supporting reliability-aware SPICE simulation of Fe-VNAND. This study demonstrates the model's effectiveness on a single FeFET configuration, serving as a pathway toward modeling the retention loss mechanisms.

cs.LG↗

arXiv2026

Proxima: Near-storage Acceleration for Graph-based Approximate Nearest Neighbor Search in 3D NAND

Approximate nearest neighbor search (ANNS) plays an indispensable role in a wide variety of applications, including recommendation systems, information retrieval, and semantic search. Among the cutting-edge ANNS algorithms, graph-based approaches provide superior accuracy and scalability on massive datasets. However, the best-performing graph-based ANN search solutions incur tens of hundreds of memory footprints as well as costly distance computation, thus hindering their efficient deployment at scale. The 3D NAND flash is emerging as a promising device for data-intensive applications due to its high density and nonvolatility. In this work, we present the near-storage processing (NSP)-based ANNS solution Proxima, to accelerate graph-based ANNS with algorithm-hardware co-design in 3D NAND flash. Proxima significantly reduces the complexity of graph search by leveraging the distance approximation and early termination. On top of the algorithmic enhancement, we implement Proxima search algorithm in 3D NAND flash using the heterogeneous integration technique. To maximize 3D NAND's bandwidth utilization, we present customized dataflow and optimized data allocation scheme. Our evaluation results show that: compared to graph ANNS on CPU and GPU, Proxima achieves a magnitude improvement in throughput or energy efficiency. Proxima yields 7x to 13x speedup over existing ASIC designs. Furthermore, Proxima achieves a good balance between accuracy, efficiency and storage density compared to previous NSP-based accelerators.

cs.AR↗

arXiv2026

NeuroSim V1.5: Improved Software Backbone for Benchmarking Compute-in-Memory Accelerators with Device and Circuit-level Non-idealities

The exponential growth of artificial intelligence (AI) applications has exposed the inefficiency of conventional von Neumann architectures, where frequent data transfers between compute units and memory create significant energy and latency bottlenecks. Analog Computing-in-Memory (ACIM) addresses this challenge by performing multiply-accumulate (MAC) operations directly in the memory arrays, substantially reducing data movement. However, designing robust ACIM accelerators requires accurate modeling of device- and circuit-level non-idealities. In this work, we present NeuroSim V1.5, introducing several key advances: (1) seamless integration with TensorRT's post-training quantization flow enabling support for more neural networks including transformers, (2) a flexible noise injection methodology built on pre-characterized statistical models, making it straightforward to incorporate data from SPICE simulations or silicon measurements, (3) expanded device support including emerging non-volatile capacitive memories, and (4) up to 6.5x faster runtime than NeuroSim V1.4 through optimized behavioral simulation. The combination of these capabilities uniquely enables systematic design space exploration across both accuracy and hardware efficiency metrics. Through multiple case studies, we demonstrate optimization of critical design parameters while maintaining network accuracy. By bridging high-fidelity noise modeling with efficient simulation, NeuroSim V1.5 advances the design and validation of next-generation ACIM accelerators. All NeuroSim versions are available open-source at https://github.com/neurosim/NeuroSim.

cs.AR↗

arXiv2026

System-Technology Co-Optimization of Bitline Routing and Bonding Pathways in Monolithic 3D DRAM Architectures

3D DRAM has emerged as a promising approach for continued density scaling, but its viability is limited by routing and hybrid bonding constraints to periphery, which may degrade sensing margin, latency, and array efficiency. With device characteristics and array parasitics extracted from TCAD, SPICE simulations are performed with peri logic in a CMOS-Bonded-Array (CBA). The analysis shows that the bitline strap architecture with amorphous oxide semiconductor (AOS) selectors is essential to manage routing congestion and parasitics. The optimized design achieves a bit density of 2.6 Gb/mm^2 (137 layers with Si access transistors or 87 layers with AOS), representing ~6x density scaling over D1b 2D DRAM. The design further demonstrates a nominal row cycle time (tRC) of 10.5 ns, compared to 21.3 ns in D1b, and a 60% reduction in read/write energy.

cs.AR↗

arXiv2026

ALD Oxidant as A Tuning Knob for Memory Window Expansion in Ferroelectric FETs for Vertical NAND Applications

Dielectric inserts are widely used to expand the memory window (MW) in ferroelectric FETs (FeFETs) for vertical NAND applications, with prior efforts focused primarily on material selection and stack positioning. Here, we demonstrate that the ALD oxidant used for the Al2O3 interlayer serves as a process-level tuning knob for MW engineering. H2O-grown Al2O3 yields a significantly larger MW (7-8 V) compared to O3 (4 V) for both gate-injection (12/3) and tunnel dielectric (8/3/8) configurations. While the tunnel dielectric (8/3/8) stack maintains robust retention up to 1e4s at 125C despite the larger MW, the gate-injection (12/3) configuration exhibits pronounced retention degradation for the H2O case. The enhanced MW is attributed to higher interlayer leakage associated with H2O-based ALD. These results establish oxidant choice as a key process parameter for co-optimizing MW and retention in ferroelectric NAND technologies.

cond-mat.mtrl-sci↗

arXiv2026

Scalable Digital Compute-in-Memory Ising Machines for Robustness Verification of Binary Neural Networks

Verification of binary neural network (BNN) robustness is NP-hard, as it can be formulated as a combinatorial search for an adversarial perturbation that induces misclassification. Exact verification methods therefore scale poorly with problem dimension, motivating the use of hardware-accelerated heuristics and unconventional computing platforms, such as Ising solvers, that can efficiently explore complex energy landscapes and discover high-quality solutions. In this work, we reformulate BNN robustness verification as a quadratic unconstrained binary optimization (QUBO) problem and solve it using a digital compute-in-memory (DCIM) SRAM-based Ising machine. Instead of requiring globally optimal solutions, we exploit imperfect solutions produced by the DCIM Ising machine to extract adversarial perturbations and thereby demonstrate the non-robustness of the BNN. The proposed architecture stores quantized QUBO coefficients in approximately 9.1~Mb of SRAM and performs annealing in memory via voltage-controlled pseudo-read dynamics, enabling iterative updates with minimal data movement. Experimental projections indicate that the proposed approach achieves a $178\times$ acceleration in convergence rate and a $1538\times$ improvement in power efficiency relative to conventional CPU-based implementations.

cs.ET↗