arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.33995

SGL: Towards a Bridging Model for Heterogeneous Hierarchical Platforms

Chong Li·Gaétan Hains
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper introduces the Scatter-Gather parallel-programming and parallel-execution model in the form of a simple imperative Scatter-Gather Language (SGL). Its design is based on past experience with bulk-synchronous parallel (BSP) programming and BSP language design. SGL's novel features are motivated by the last decade's move towards multi-level and heterogeneous parallel architectures involving multi-core processors, graphics accelerators and hierarchical routing networks in the largest multiprocessing systems. The design of SGL is coherent with L. Valiant's Multi-BSP while offering a programming interface that is even simpler than the primitives of Bulk-Synchronous Parallel ML (BSML). SGL appears to cover a large subset of all BSP algorithms while avoiding complex message-passing programming. Like all BSP-inspired systems, it supports predictable, portable and scalable performance. Moreover, SGL's explicit data distribution will allow automatic or programmable load-balancing.

Explore related subjects

cs.DCcs.PFcs.PL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Chong Li, Gaétan Hains. 2026-10-25. SGL: Towards a Bridging Model for Heterogeneous Hierarchical Platforms. https://doi.org/10.1504/ijhpcn.2012.046371

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

ActECC: Action Chunk Temporal Harnessing for Edge-Cloud Collaborative VLA Inference System

VLA models have emerged as the mainstream for embodied intelligence, which decouples reasoning and execution via action chunks. To enable real-time deployment of VLA models, existing studies investigate edge-cloud collaborative inference, which pose two challenges to the action chunk mechanism. (1) Edge-cloud collaboration complicates system latency components, hindering the determination of valid ranges for action chunks. (2) Latency fluctuations in cross-edge update links under edge-cloud scenarios make it difficult to schedule replanning requests. To address these, we analyze the key factors affecting action chunk usable ranges and replanning request timing, revealing phase dependence and edge-cloud latency constraints. Based on these insights, we propose ActECC, which enables Action Chunk Temporal Harnessing through kinematics-guided usable range assessment and delay-aware replanning scheduling. Experiments show that ActECC reduces per-action latency by up to 27.4% and cloud calls by up to 45.1%, while lowering average execution error compared with existing edge-cloud collaborative.

cs.DC↗
arXiv2026

KubePACS: Kubernetes Cluster Using Performant, Highly Available, and Cost Efficient Spot Instances

Cloud users aim to minimize cost while maximizing performance by selecting the most suitable instance types for their workloads. To reduce expenses, spot instances have been widely adopted due to their steep discounts compared to on-demand pricing. However, their use introduces reliability risks due to potential interruptions, and existing research has primarily focused on mitigating this trade-off from a cost or availability perspective alone. Despite the diversity in hardware capabilities among instance types, current provisioning systems tend to ignore performance variation, selecting nodes solely based on minimum resource requirements. In this paper, we present KubePACS, a Kubernetes-native spot instance provisioning system that constructs node pools optimized for both cost and performance while guaranteeing high availability. KubePACS formulates the node selection process as a multi-objective optimization problem, incorporating real-time data such as spot prices, performance benchmarks, and availability scores, including the multi-node Spot Placement Score (SPS). It solves this problem efficiently using an Integer Linear Programming (ILP) approach guided by the Golden Section Search (GSS) algorithm to find the optimal configuration. By integrating with the Karpenter node autoscaler, KubePACS jointly optimizes instance-type selection and node scaling decisions within a standard provisioning workflow. KubePACS also adopts a novel heuristic to support workload-specific preferences by scaling performance metrics for specialized instances. Through extensive evaluation across synthetic and real-world workloads, KubePACS demonstrates on average 55.09% and up to 81.06% higher performance per dollar over state-of-the-art solutions such as Karpenter, SpotVerse, and SpotKube, which only reference the spot instance prices and limited availability data.

cs.DC↗
arXiv2026

Democratizing LLM Training with the Edge: Challenges, Systems, and the Road Ahead

The growing computational power of modern GPU data centers has let a few industrial giants train and deploy large language models (LLMs) on massive amounts of data. Yet the sheer scale of these models and their training data makes centralized development increasingly hard to sustain, and at the same time shuts out smaller institutions that hold relevant private data but lack comparable infrastructure. In this paper, we organize our positions on distributed LLM training around the pretraining and fine-tuning paradigms, covering federated learning, split learning, and decentralized approaches, and we assess their readiness for geo-distributed settings with heterogeneous hardware and unreliable connectivity. We then present a {\em unified} system architecture built around a cloud-hosted parameter server that coordinates heterogeneous clients over outbound gRPC connections, reusing one communication, compression, and fault-tolerance stack for both pretraining and a family of fine-tuning methods. We validate the design with real geo-distributed experiments on GPT-2 Medium and Llama3-1B pretraining across multiple university sites, showing stable convergence under realistic conditions. Finally, we chart the road ahead, identifying open problems that span frontier model support, elastic fault tolerance, adaptive compression, federated alignment, privacy and trust, incentive design, and system--algorithm co-design, all of which must be addressed to make democratized LLM training a reality.

cs.DC↗