arXiv subjects

Haoyu Zhang

Publications and source records attributed to Haoyu Zhang.

185 records · Page 11Linked to original sources

arXiv2018

SLAQ: Quality-Driven Scheduling for Distributed Machine Learning

Training machine learning (ML) models with large datasets can incur significant resource contention on shared clusters. This training typically involves many iterations that continually improve the quality of the model. Yet in exploratory settings, better models can be obtained faster by directing resources to jobs with the most potential for improvement. We describe SLAQ, a cluster scheduling system for approximate ML training jobs that aims to maximize the overall job quality. When allocating cluster resources, SLAQ explores the quality-runtime trade-offs across multiple jobs to maximize system-wide quality improvement. To do so, SLAQ leverages the iterative nature of ML training algorithms, by collecting quality and resource usage information from concurrent jobs, and then generating highly-tailored quality-improvement predictions for future iterations. Experiments show that SLAQ achieves an average quality improvement of up to 73% and an average delay reduction of up to 44% on a large set of ML training jobs, compared to resource fairness schedulers.

cs.DC↗

arXiv2018

Smooth $q$-Gram, and Its Applications to Detection of Overlaps among Long, Error-Prone Sequencing Reads

We propose smooth $q$-gram, the first variant of $q$-gram that captures $q$-gram pair within a small edit distance. We apply smooth $q$-gram to the problem of detecting overlapping pairs of error-prone reads produced by single molecule real time sequencing (SMRT), which is the first and most critical step of the de novo fragment assembly of SMRT reads. We have implemented and tested our algorithm on a set of real world benchmarks. Our empirical results demonstrated the significant superiority of our algorithm over the existing $q$-gram based algorithms in accuracy.

cs.DS↗

arXiv2017

Size-dependent couple stress Timoshenko beam theory

In this paper, a new size-dependent Timoshenko beam model is developed based on the consistent couple stress theory. In the present formulation, the governing equations and corresponding boundary conditions are obtained. Afterwards, this formulation is used to investigate size-dependency for several elementary beam problems. Analytical solutions are obtained for pure bending of a beam and for a cantilever beam with partially and fully clamped boundary conditions. These analytical results are then compared to the numerical results from a two-dimensional finite element formulation for the corresponding couple stress continuum problem.

physics.gen-ph↗

arXiv2017

EmbedJoin: Efficient Edit Similarity Joins via Embeddings

We study the problem of edit similarity joins, where given a set of strings and a threshold value $K$, we want to output all pairs of strings whose edit distances are at most $K$. Edit similarity join is a fundamental problem in data cleaning/integration, bioinformatics, collaborative filtering and natural language processing, and has been identified as a primitive operator for database systems. This problem has been studied extensively in the literature. However, we have observed that all the existing algorithms fall short on long strings and large distance thresholds. In this paper we propose an algorithm named EmbedJoin which scales very well with string length and distance threshold. Our algorithm is built on the recent advance of metric embeddings for edit distance, and is very different from all of the previous approaches. We demonstrate via an extensive set of experiments that EmbedJoin significantly outperforms the previous best algorithms on long strings and large distance thresholds.

cs.DB↗

arXiv2016

Computing Skylines on Distributed Data

In this paper we study skyline queries in the distributed computational model, where we have $s$ remote sites and a central coordinator (the query node); each site holds a piece of data, and the coordinator wants to compute the skyline of the union of the $s$ datasets. The computation is in terms of rounds, and the goal is to minimize both the total communication cost and the round cost. Viewing data objects as points in the Euclidean space, we consider both the horizontal data partition case where each site holds a subset of points, and the vertical data partition case where each site holds one coordinate of all the points. We give a set of algorithms that have provable theoretical guarantees, and complement them with information theoretical lower bounds. We also demonstrate the superiority of our algorithms over existing heuristics by an extensive set of experiments on both synthetic and real world datasets.

cs.DB↗