arXiv Science⌕ Search

arXiv subjects

JeongGil Ko

Publications and source records attributed to JeongGil Ko.

22 records · Page 2Linked to original sources

On-device Training: A First Overview on Existing Systems

The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (ii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work targets to summarize and analyze state-of-the-art systems research that allows such on-device model training capabilities and provide a survey of on-device training from a systems perspective.

cs.LG↗

FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things

There is a significant relevance of federated learning (FL) in the realm of Artificial Intelligence of Things (AIoT). However, most existing FL works do not use datasets collected from authentic IoT devices and thus do not capture unique modalities and inherent challenges of IoT data. To fill this critical gap, in this work, we introduce FedAIoT, an FL benchmark for AIoT. FedAIoT includes eight datasets collected from a wide range of IoT devices. These datasets cover unique IoT modalities and target representative applications of AIoT. FedAIoT also includes a unified end-to-end FL framework for AIoT that simplifies benchmarking the performance of the datasets. Our benchmark results shed light on the opportunities and challenges of FL for AIoT. We hope FedAIoT could serve as an invaluable resource to foster advancements in the important field of FL for AIoT. The repository of FedAIoT is maintained at https://github.com/AIoT-MLSys-Lab/FedAIoT.

cs.LG↗

Fast Monte-Carlo Approximation of the Attention Mechanism

We introduce Monte-Carlo Attention (MCA), a randomized approximation method for reducing the computational cost of self-attention mechanisms in Transformer architectures. MCA exploits the fact that the importance of each token in an input sequence varies with respect to their attention scores; thus, some degree of error can be tolerable when encoding tokens with low attention. Using approximate matrix multiplication, MCA applies different error bounds to encode input tokens such that those with low attention scores are computed with relaxed precision, whereas errors of salient elements are minimized. MCA can operate in parallel with other attention optimization schemes and does not require model modification. We study the theoretical error bounds and demonstrate that MCA reduces attention complexity (in FLOPS) for various Transformer models by up to 11$\times$ in GLUE benchmarks without compromising model accuracy.

cs.LG↗

DynaChanAl: Dynamic Channel Allocation with Minimal End-to-end Delay for Wireless Sensor Networks

With recent advances in wireless communication, networking, and low power sensor technology, wireless sensor network (WSN) systems have begun to take significant roles in various applications ranging from environmental sensing to mobile healthcare sensing. While some WSN applications only require a lim- ited amount of bandwidth, new emerging applications operate with a notice- ably large amount of data transfers. One way to deal with such applications is to maximize the available capacity by utilizing the use of multiple wireless channels. This work proposes DynaChannAl, a distributed dynamic wireless channel algorithm with the goal of effectively distributing nodes on multiple wireless channels in WSN systems. Specifically, DynaChannAl targets applica- tions where mobile nodes connect to a pre-existing wireless backbone and takes the expected end-to-end queuing delay as its core metric. We use the link qual- ity indicator (LQI) values provided by IEEE 802.15.4 radios white-list potential links with good link quality and evaluate such links with the aggregated packet transmission latency at each hop. Our approach is useful for applications that require minimal end-to-end delay (i.e., healthcare applications). DynaChannAl is a light weight and highly adoptable scheme that can be easily incorporated with various pre-developed components and pre-deployed applications. We eval- uate DynaChannAl in on a 45 node WSN testbed. As the first study to consider end-to-end latency as the core metric for channel allocation in WSN systems, the experimental results indicate that DynaChannAl successfully distributes multi- ple (mobile) source nodes on different wireless channels and enables the nodes to select wireless channel and links that can minimize the end-to-end latency.

cs.NI↗