arXiv Science⌕ Search

arXiv subjects

Debadatta Mishra

Publications and source records attributed to Debadatta Mishra.

4 recordsLinked to original sources

LLMRunner: Closed Workload Model Performance Testing Framework for LLM Servings

Performance testing large language model (LLM) serving engines is a crucial requirement considering the criticality of the infrastructure for deploying conversational AI at scale. Most of the benchmarking tools used to carryout performance evaluation of LLM serving do not capture realistic workload orchestration unlike closed workload performance testing infrastructure used in other domains such as web and database engines. For example, state-of-the-art LLM benchmarks for performance testing model traffic as streams of independent single-turn and stand-alone requests omitting the end-user behavior such as think-time between conversational turns. We present LLMRunner, a performance tester for LLM serving that addresses this gap by providing a framework to generate realistic load consisting of multi-turn conversations with configurable end-user parameters such as number of users in the workload loop, think-time distributions etc. Moreover, LLMRunner provides declarative matrix-sweep experiment plans, integrated GPU and KV-cache monitoring, and an automated artifact collection and plotting subsystem. We use LLMRunner to comparatively analyze the performance of LLM serving when NVIDIA CUDA Unified Virtual Memory (UVM) is used as a drop-in replacement memory allocator for the Key-Value cache (KV-Cache). Our test demonstrates that UVM incurs no measurable throughput or latency penalty compared to the baseline memory allocation technique for non-oversubscription scenarios.

cs.PF↗

Motivating Next-Generation OS Physical Memory Management for Terabyte-Scale NVMMs

Software managed byte-addressable hybrid memory systems consisting of DRAMs and NVMMs offer a lot of flexibility to design efficient large scale data processing applications. Operating systems (OS) play an important role in enabling the applications to realize the integrated benefits of DRAMs' low access latency and NVMMs' large capacity along with its persistent characteristics. In this paper, we comprehensively analyze the performance of conventional OS physical memory management subsystems that were designed only based on the DRAM memory characteristics in the context of modern hybrid byte-addressable memory systems. To study the impact of high access latency and large capacity of NVMMs on physical memory management, we perform an extensive evaluation on Linux with Intel's Optane NVMM. We observe that the core memory management functionalities such as page allocation are negatively impacted by high NVMM media latency, while functionalities such as conventional fragmentation management are rendered inadequate. We also demonstrate that certain traditional memory management functionalities are affected by neither aspects of modern NVMMs. We conclusively motivate the need to overhaul fundamental aspects of traditional OS physical memory management in order to fully exploit terabyte-scale NVMMs.

cs.OS↗

Rescuing the End-user systems from Vulnerable Applications using Virtualization Techniques

In systems owned by normal end-users, many times security attacks are mounted by sneaking in malicious applications or exploiting existing software vulnerabilities through security non-conforming actions of users. Virtualization approaches can address this problem by providing a quarantine environment for applications, malicious devices, and device drivers, which are mostly used as entry points for security attacks. However, the existing methods to provide quarantine environments using virtualization are not transparent to the user, both in terms of application interface transparency and file system transparency. Further, software configuration level solutions like remote desktops and remote application access mechanisms combined with shared file systems do not meet the user transparency and security requirements. We propose qOS, a VM-based solution combined with certain OS extensions to meet the security requirements of end-point systems owned by normal users, in a transparent and efficient manner. We demonstrate the efficacy of qOS by empirically evaluating the prototype implementation in the Linux+KVM system in terms of efficiency, security, and user transparency.

cs.CR↗

A practical analysis of ROP attacks

Control Flow Hijacking attacks have posed a serious threat to the security of applications for a long time where an attacker can damage the control Flow Integrity of the program and execute arbitrary code. These attacks can be performed by injecting code in the program's memory or reusing already existing code in the program (also known as Code-Reuse Attacks). Code-Reuse Attacks in the form of Return-into-libc Attacks or Return-Oriented Programming Attacks are said to be Turing Complete, providing a guarantee that there will always exist code segments (also called ROP gadgets) within a binary allowing an attacker to perform any kind of function by building a suitable ROP chain (chain of ROP gadgets). Our goal is to study different techniques of performing ROP Attacks and find the difficulties encountered to perform such attacks. For this purpose, we have designed an automated tool which works on 64-bit systems and generates a ROP chain from ROP gadgets to execute arbitrary system calls.

cs.CR↗