arXiv subjects

MohammadHossein Bateni

Publications and source records attributed to MohammadHossein Bateni.

At least 37 records · Page 2Linked to original sources

arXiv2023

Agile Modeling: From Concept to Classifier in Minutes

The application of computer vision to nuanced subjective use cases is growing. While crowdsourcing has served the vision community well for most objective tasks (such as labeling a "zebra"), it now falters on tasks where there is substantial subjectivity in the concept (such as identifying "gourmet tuna"). However, empowering any user to develop a classifier for their concept is technically difficult: users are neither machine learning experts, nor have the patience to label thousands of examples. In reaction, we introduce the problem of Agile Modeling: the process of turning any subjective visual concept into a computer vision model through a real-time user-in-the-loop interactions. We instantiate an Agile Modeling prototype for image classification and show through a user study (N=14) that users can create classifiers with minimal effort under 30 minutes. We compare this user driven process with the traditional crowdsourcing paradigm and find that the crowd's notion often differs from that of the user's, especially as the concepts become more subjective. Finally, we scale our experiments with simulations of users training classifiers for ImageNet21k categories to further demonstrate the efficacy.

cs.LG↗

arXiv2023

Sequential Attention for Feature Selection

Feature selection is the problem of selecting a subset of features for a machine learning model that maximizes model quality subject to a budget constraint. For neural networks, prior methods, including those based on $\ell_1$ regularization, attention, and other techniques, typically select the entire feature subset in one evaluation round, ignoring the residual value of features during selection, i.e., the marginal contribution of a feature given that other features have already been selected. We propose a feature selection algorithm called Sequential Attention that achieves state-of-the-art empirical results for neural networks. This algorithm is based on an efficient one-pass implementation of greedy forward selection and uses attention weights at each step as a proxy for feature importance. We give theoretical insights into our algorithm for linear regression by showing that an adaptation to this setting is equivalent to the classical Orthogonal Matching Pursuit (OMP) algorithm, and thus inherits all of its provable guarantees. Our theoretical and empirical analyses offer new explanations towards the effectiveness of attention and its connections to overparameterization, which may be of independent interest.

cs.LG↗

arXiv2023

Optimal Fully Dynamic $k$-Center Clustering for Adaptive and Oblivious Adversaries

In fully dynamic clustering problems, a clustering of a given data set in a metric space must be maintained while it is modified through insertions and deletions of individual points. In this paper, we resolve the complexity of fully dynamic $k$-center clustering against both adaptive and oblivious adversaries. Against oblivious adversaries, we present the first algorithm for fully dynamic $k$-center in an arbitrary metric space that maintains an optimal $(2+ε)$-approximation in $O(k \cdot \mathrm{polylog}(n,Δ))$ amortized update time. Here, $n$ is an upper bound on the number of active points at any time, and $Δ$ is the aspect ratio of the metric space. Previously, the best known amortized update time was $O(k^2\cdot \mathrm{polylog}(n,Δ))$, and is due to Chan, Gourqin, and Sozio (2018). Moreover, we demonstrate that our runtime is optimal up to $\mathrm{polylog}(n,Δ)$ factors. In fact, we prove that even offline algorithms for $k$-clustering tasks in arbitrary metric spaces, including $k$-medians, $k$-means, and $k$-center, must make at least $Ω(n k)$ distance queries to achieve any non-trivial approximation factor. This implies a lower bound of $Ω(k)$ which holds even for the insertions-only setting. We also show deterministic lower and upper bounds for adaptive adversaries, demonstrate that an update time sublinear in $k$ is possible against oblivious adversaries for metric spaces which admit locally sensitive hash functions (LSH) and give the first fully dynamic $O(1)$-approximation algorithms for the closely related $k$-sum-of-radii and $k$-sum-of-diameter problems.

cs.DS↗

arXiv2021

Optimal Fully Dynamic $k$-Centers Clustering

We present the first algorithm for fully dynamic $k$-centers clustering in an arbitrary metric space that maintains an optimal $2+ε$ approximation in $O(k \cdot \operatorname{polylog}(n,Δ))$ amortized update time. Here, $n$ is an upper bound on the number of active points at any time, and $Δ$ is the aspect ratio of the data. Previously, the best known amortized update time was $O(k^2\cdot \operatorname{polylog}(n,Δ))$, and is due to Chan, Gourqin, and Sozio. We demonstrate that the runtime of our algorithm is optimal up to $\operatorname{polylog}(n,Δ)$ factors, even for insertion-only streams, which closes the complexity of fully dynamic $k$-centers clustering. In particular, we prove that any algorithm for $k$-clustering tasks in arbitrary metric spaces, including $k$-means, $k$-medians, and $k$-centers, must make at least $Ω(n k)$ distance queries to achieve any non-trivial approximation factor. Despite the lower bound for arbitrary metrics, we demonstrate that an update time sublinear in $k$ is possible for metric spaces which admit locally sensitive hash functions (LSH). Namely, we demonstrate a black-box transformation which takes a locally sensitive hash family for a metric space and produces a faster fully dynamic $k$-centers algorithm for that space. In particular, for a large class of metrics including Euclidean space, $\ell_p$ spaces, the Hamming Metric, and the Jaccard Metric, for any $c > 1$, our results yield a $c(4+ε)$ approximate $k$-centers solution in $O(n^{1/c} \cdot \operatorname{polylog}(n,Δ))$ amortized update time, simultaneously for all $k \geq 1$. Previously, the only known comparable result was a $O(c \log n)$ approximation for Euclidean space due to Schmidt and Sohler, running in the same amortized update time.

cs.DS↗

arXiv2021

Streaming Belief Propagation for Community Detection

The community detection problem requires to cluster the nodes of a network into a small number of well-connected "communities". There has been substantial recent progress in characterizing the fundamental statistical limits of community detection under simple stochastic block models. However, in real-world applications, the network structure is typically dynamic, with nodes that join over time. In this setting, we would like a detection algorithm to perform only a limited number of updates at each node arrival. While standard voting approaches satisfy this constraint, it is unclear whether they exploit the network information optimally. We introduce a simple model for networks growing over time which we refer to as streaming stochastic block model (StSBM). Within this model, we prove that voting algorithms have fundamental limitations. We also develop a streaming belief-propagation (StreamBP) approach, for which we prove optimality in certain regimes. We validate our theoretical findings on synthetic and real data.

stat.ML↗

arXiv2019

Distributed Weighted Matching via Randomized Composable Coresets

Maximum weight matching is one of the most fundamental combinatorial optimization problems with a wide range of applications in data mining and bioinformatics. Developing distributed weighted matching algorithms is challenging due to the sequential nature of efficient algorithms for this problem. In this paper, we develop a simple distributed algorithm for the problem on general graphs with approximation guarantee of $2+\varepsilon$ that (nearly) matches that of the sequential greedy algorithm. A key advantage of this algorithm is that it can be easily implemented in only two rounds of computation in modern parallel computation frameworks such as MapReduce. We also demonstrate the efficiency of our algorithm in practice on various graphs (some with half a trillion edges) by achieving objective values always close to what is achievable in the centralized setting.

cs.DC↗

arXiv2019

Categorical Feature Compression via Submodular Optimization

In the era of big data, learning from categorical features with very large vocabularies (e.g., 28 million for the Criteo click prediction dataset) has become a practical challenge for machine learning researchers and practitioners. We design a highly-scalable vocabulary compression algorithm that seeks to maximize the mutual information between the compressed categorical feature and the target binary labels and we furthermore show that its solution is guaranteed to be within a $1-1/e \approx 63\%$ factor of the global optimal solution. To achieve this, we introduce a novel re-parametrization of the mutual information objective, which we prove is submodular, and design a data structure to query the submodular function in amortized $O(\log n )$ time (where $n$ is the input vocabulary size). Our complete algorithm is shown to operate in $O(n \log n )$ time. Additionally, we design a distributed implementation in which the query data structure is decomposed across $O(k)$ machines such that each machine only requires $O(\frac n k)$ space, while still preserving the approximation guarantee and using only logarithmic rounds of computation. We also provide analysis of simple alternative heuristic compression methods to demonstrate they cannot achieve any approximation guarantee. Using the large-scale Criteo learning task, we demonstrate better performance in retaining mutual information and also verify competitive learning performance compared to other baseline methods.

cs.LG↗

arXiv2018

Coresets Meet EDCS: Algorithms for Matching and Vertex Cover on Massive Graphs

As massive graphs become more prevalent, there is a rapidly growing need for scalable algorithms that solve classical graph problems, such as maximum matching and minimum vertex cover, on large datasets. For massive inputs, several different computational models have been introduced, including the streaming model, the distributed communication model, and the massively parallel computation (MPC) model that is a common abstraction of MapReduce-style computation. In each model, algorithms are analyzed in terms of resources such as space used or rounds of communication needed, in addition to the more traditional approximation ratio. In this paper, we give a single unified approach that yields better approximation algorithms for matching and vertex cover in all these models. The highlights include: * The first one pass, significantly-better-than-2-approximation for matching in random arrival streams that uses subquadratic space, namely a $(1.5+ε)$-approximation streaming algorithm that uses $O(n^{1.5})$ space for constant $ε> 0$. * The first 2-round, better-than-2-approximation for matching in the MPC model that uses subquadratic space per machine, namely a $(1.5+ε)$-approximation algorithm with $O(\sqrt{mn} + n)$ memory per machine for constant $ε> 0$. By building on our unified approach, we further develop parallel algorithms in the MPC model that give a $(1 + ε)$-approximation to matching and an $O(1)$-approximation to vertex cover in only $O(\log\log{n})$ MPC rounds and $O(n/poly\log{(n)})$ memory per machine. These results settle multiple open questions posed in the recent paper of Czumaj~et.al. [STOC 2018].

cs.DS↗

arXiv2018

Fast Algorithms for Knapsack via Convolution and Prediction

The \Problem{knapsack} problem is a fundamental problem in combinatorial optimization. It has been studied extensively from theoretical as well as practical perspectives as it is one of the most well-known NP-hard problems. The goal is to pack a knapsack of size $t$ with the maximum value from a collection of $n$ items with given sizes and values. Recent evidence suggests that a classic $O(nt)$ dynamic-programming solution for the \Problem{knapsack} problem might be the fastest in the worst case. In fact, solving the \Problem{knapsack} problem was shown to be computationally equivalent to the \Problem{$(\min, +)$ convolution} problem, which is thought to be facing a quadratic-time barrier. This hardness is in contrast to the more famous \Problem{$(+, \cdot)$ convolution} (generally known as \Problem{polynomial multiplication}), that has an $O(n\log n)$-time solution via Fast Fourier Transform. Our main results are algorithms with near-linear running times (in terms of the size of the knapsack and the number of items) for the \Problem{knapsack} problem, if either the values or sizes of items are small integers. More specifically, if item sizes are integers bounded by $\smax$, the running time of our algorithm is $\tilde O((n+t)\smax)$. If the item values are integers bounded by $\vmax$, our algorithm runs in time $\tilde O(n+t\vmax)$. Best previously known running times were $O(nt)$, $O(n^2\smax)$ and $O(n\smax\vmax)$ (Pisinger, J. of Alg., 1999).

cs.GT↗

arXiv2018

Polynomial-time Approximation Scheme for Minimum k-cut in Planar and Minor-free Graphs

The $k$-cut problem asks, given a connected graph $G$ and a positive integer $k$, to find a minimum-weight set of edges whose removal splits $G$ into $k$ connected components. We give the first polynomial-time algorithm with approximation factor $2-ε$ (with constant $ε> 0$) for the $k$-cut problem in planar and minor-free graphs. Applying more complex techniques, we further improve our method and give a polynomial-time approximation scheme for the $k$-cut problem in both planar and minor-free graphs. Despite persistent effort, to the best of our knowledge, this is the first improvement for the $k$-cut problem over standard approximation factor of $2$ in any major class of graphs.

cs.DS↗

arXiv2018

Massively Parallel Dynamic Programming on Trees

Dynamic programming is a powerful technique that is, unfortunately, often inherently sequential. That is, there exists no unified method to parallelize algorithms that use dynamic programming. In this paper, we attempt to address this issue in the Massively Parallel Computations (MPC) model which is a popular abstraction of MapReduce-like paradigms. Our main result is an algorithmic framework to adapt a large family of dynamic programs defined over trees. We introduce two classes of graph problems that admit dynamic programming solutions on trees. We refer to them as "(polylog)-expressible" and "linear-expressible" problems. We show that both classes can be parallelized in $O(\log n)$ rounds using a sublinear number of machines and a sublinear memory per machine. To achieve this result, we introduce a series of techniques that can be plugged together. To illustrate the generality of our framework, we implement in $O(\log n)$ rounds of MPC, the dynamic programming solution of graph problems such as minimum bisection, $k$-spanning tree, maximum independent set, longest path, etc., when the input graph is a tree.

cs.DS↗

arXiv2017

Distributed Coverage Maximization via Sketching

Coverage problems are central in optimization and have a wide range of applications in data mining and machine learning. While several distributed algorithms have been developed for coverage problems, the existing methods suffer from several limitations, e.g., they all achieve either suboptimal approximation guarantees or suboptimal space and memory complexities. In addition, previous algorithms developed for submodular maximization assume oracle access, and ignore the computational complexity of communicating large subsets or computing the size of the union of the subsets in this subfamily. In this paper, we develop an improved distributed algorithm for the $k$-cover and the set cover with $λ$ outliers problems, with almost optimal approximation guarantees, almost optimal memory complexity, and linear communication complexity running in only four rounds of computation. Finally, we perform an extensive empirical study of our algorithms on a number of publicly available real data sets, and show that using sketches of size $30$ to $600$ times smaller than the input, one can solve the coverage maximization problem with quality very close to that of the state-of-the-art single-machine algorithm.

cs.DS↗

arXiv2015

Distributed Balanced Partitioning via Linear Embedding

Balanced partitioning is often a crucial first step in solving large-scale graph optimization problems, e.g., in some cases, a big graph can be chopped into pieces that fit on one machine to be processed independently before stitching the results together. In other cases, links between different parts may show up in the running time and/or network communications cost. We study a distributed balanced partitioning problem where the goal is to partition the vertices of a given graph into k pieces so as to minimize the total cut size. Our algorithm is composed of a few steps that are easily implementable in distributed computation frameworks. The algorithm first embeds nodes of the graph onto a line, and then processes nodes in a distributed manner guided by the linear embedding order. We examine various ways to find the first embedding, e.g., via a hierarchical clustering or Hilbert curves. Then we apply four different techniques including local swaps, minimum cuts on the boundaries of partitions, as well as contraction and dynamic programming. As our empirical study, we compare the above techniques with each other, and also to previous work in distributed graph algorithms, e.g., a label propagation method, FENNEL and Spinner. We report our results both on a private map graph and several public social networks, and show that our results beat previous distributed algorithms: e.g., compared to the label propagation algorithm, we report an improvement of 15-25% in the cut value. We also observe that our algorithms allow for scalable distributed implementation for any number of partitions. Finally, we apply our techniques for the Google Maps Driving Directions to minimize the number of multi-shard queries with the goal of saving in CPU usage. During live experiments, we observe an ~40% drop in the number of multi-shard queries when comparing our method with a standard geography-based method.

cs.DC↗

arXiv2015

Revenue Maximization for Selling Multiple Correlated Items

We study the problem of selling $n$ items to a single buyer with an additive valuation function. We consider the valuation of the items to be correlated, i.e., desirabilities of the buyer for the items are not drawn independently. Ideally, the goal is to design a mechanism to maximize the revenue. However, it has been shown that a revenue optimal mechanism might be very complicated and as a result inapplicable to real-world auctions. Therefore, our focus is on designing a simple mechanism that achieves a constant fraction of the optimal revenue. Babaioff et al. propose a simple mechanism that achieves a constant fraction of the optimal revenue for independent setting with a single additive buyer. However, they leave the following problem as an open question: "Is there a simple, approximately optimal mechanism for a single additive buyer whose value for $n$ items is sampled from a common base-value distribution?" Babaioff et al. show a constant approximation factor of the optimal revenue can be achieved by either selling the items separately or as a whole bundle in the independent setting. We show a similar result for the correlated setting when the desirabilities of the buyer are drawn from a common base-value distribution. It is worth mentioning that the core decomposition lemma which is mainly the heart of the proofs for efficiency of the mechanisms does not hold for correlated settings. Therefore we propose a modified version of this lemma which is applicable to the correlated settings as well. Although we apply this technique to show the proposed mechanism can guarantee a constant fraction of the optimal revenue in a very weak correlation, this method alone can not directly show the efficiency of the mechanism in stronger correlations.

cs.GT↗

arXiv2014

Network Cournot Competition

Cournot competition is a fundamental economic model that represents firms competing in a single market of a homogeneous good. Each firm tries to maximize its utility---a function of the production cost as well as market price of the product---by deciding on the amount of production. In today's dynamic and diverse economy, many firms often compete in more than one market simultaneously, i.e., each market might be shared among a subset of these firms. In this situation, a bipartite graph models the access restriction where firms are on one side, markets are on the other side, and edges demonstrate whether a firm has access to a market or not. We call this game \emph{Network Cournot Competition} (NCC). In this paper, we propose algorithms for finding pure Nash equilibria of NCC games in different situations. First, we carefully design a potential function for NCC, when the price functions for markets are linear functions of the production in that market. However, for nonlinear price functions, this approach is not feasible. We model the problem as a nonlinear complementarity problem in this case, and design a polynomial-time algorithm that finds an equilibrium of the game for strongly convex cost functions and strongly monotone revenue functions. We also explore the class of price functions that ensures strong monotonicity of the revenue function, and show it consists of a broad class of functions. Moreover, we discuss the uniqueness of equilibria in both of these cases which means our algorithms find the unique equilibria of the games. Last but not least, when the cost of production in one market is independent from the cost of production in other markets for all firms, the problem can be separated into several independent classical \emph{Cournot Oligopoly} problems. We give the first combinatorial algorithm for this widely studied problem.

cs.GT↗

arXiv2014

Multiplicative Bidding in Online Advertising

In this paper, we initiate the study of the multiplicative bidding language adopted by major Internet search companies. In multiplicative bidding, the effective bid on a particular search auction is the product of a base bid and bid adjustments that are dependent on features of the search (for example, the geographic location of the user, or the platform on which the search is conducted). We consider the task faced by the advertiser when setting these bid adjustments, and establish a foundational optimization problem that captures the core difficulty of bidding under this language. We give matching algorithmic and approximation hardness results for this problem; these results are against an information-theoretic bound, and thus have implications on the power of the multiplicative bidding language itself. Inspired by empirical studies of search engine price data, we then codify the relevant restrictions of the problem, and give further algorithmic and hardness results. Our main technical contribution is an $O(\log n)$-approximation for the case of multiplicative prices and monotone values. We also provide empirical validations of our problem restrictions, and test our algorithms on real data against natural benchmarks. Our experiments show that they perform favorably compared with the baseline.

cs.DS↗

arXiv2013

Improved Approximation Algorithms for (Budgeted) Node-weighted Steiner Problems

Moss and Rabani[12] study constrained node-weighted Steiner tree problems with two independent weight values associated with each node, namely, cost and prize (or penalty). They give an O(log n)-approximation algorithm for the prize-collecting node-weighted Steiner tree problem (PCST). They use the algorithm for PCST to obtain a bicriteria (2, O(log n))-approximation algorithm for the Budgeted node-weighted Steiner tree problem. Their solution may cost up to twice the budget, but collects a factor Omega(1/log n) of the optimal prize. We improve these results from at least two aspects. Our first main result is a primal-dual O(log h)-approximation algorithm for a more general problem, prize-collecting node-weighted Steiner forest, where we have (h) demands each requesting the connectivity of a pair of vertices. Our algorithm can be seen as a greedy algorithm which reduces the number of demands by choosing a structure with minimum cost-to-reduction ratio. This natural style of argument (also used by Klein and Ravi[10] and Guha et al.[8]) leads to a much simpler algorithm than that of Moss and Rabani[12] for PCST. Our second main contribution is for the Budgeted node-weighted Steiner tree problem, which is also an improvement to [12] and [8]. In the unrooted case, we improve upon an O(log^2(n))-approximation of [8], and present an O(log n)-approximation algorithm without any budget violation. For the rooted case, where a specified vertex has to appear in the solution tree, we improve the bicriteria result of [12] to a bicriteria approximation ratio of (1+eps, O(log n)/(eps^2)) for any positive (possibly subconstant) (eps). That is, for any permissible budget violation (1+eps), we present an algorithm achieving a tradeoff in the guarantee for prize. Indeed, we show that this is almost tight for the natural linear-programming relaxation used by us as well as in [12].

cs.DS↗

arXiv2010

Prize-collecting Network Design on Planar Graphs

In this paper, we reduce Prize-Collecting Steiner TSP (PCTSP), Prize-Collecting Stroll (PCS), Prize-Collecting Steiner Tree (PCST), Prize-Collecting Steiner Forest (PCSF) and more generally Submodular Prize-Collecting Steiner Forest (SPCSF) on planar graphs (and more generally bounded-genus graphs) to the same problems on graphs of bounded treewidth. More precisely, we show any $α$-approximation algorithm for these problems on graphs of bounded treewidth gives an $(α+ ε)$-approximation algorithm for these problems on planar graphs (and more generally bounded-genus graphs), for any constant $ε> 0$. Since PCS, PCTSP, and PCST can be solved exactly on graphs of bounded treewidth using dynamic programming, we obtain PTASs for these problems on planar graphs and bounded-genus graphs. In contrast, we show PCSF is APX-hard to approximate on series-parallel graphs, which are planar graphs of treewidth at most 2. This result is interesting on its own because it gives the first provable hardness separation between prize-collecting and non-prize-collecting (regular) versions of the problems: regular Steiner Forest is known to be polynomially solvable on series-parallel graphs and admits a PTAS on graphs of bounded treewidth. An analogous hardness result can be shown for Euclidian PCSF. This ends the common belief that prize-collecting variants should not add any new hardness to the problems.

cs.DS↗