arXiv subjects

Vardan Voskanyan

Publications and source records attributed to Vardan Voskanyan.

13 recordsLinked to original sources

arXiv2025

Evolving Excellence: Automated Optimization of LLM-based Agents

Agentic AI systems built on large language models (LLMs) offer significant potential for automating complex workflows, from software development to customer support. However, LLM agents often underperform due to suboptimal configurations; poorly tuned prompts, tool descriptions, and parameters that typically require weeks of manual refinement. Existing optimization methods either are too complex for general use or treat components in isolation, missing critical interdependencies. We present ARTEMIS, a no-code evolutionary optimization platform that jointly optimizes agent configurations through semantically-aware genetic operators. Given only a benchmark script and natural language goals, ARTEMIS automatically discovers configurable components, extracts performance signals from execution logs, and evolves configurations without requiring architectural modifications. We evaluate ARTEMIS on four representative agent systems: the \emph{ALE Agent} for competitive programming on AtCoder Heuristic Contest, achieving a \textbf{$13.6\%$ improvement} in acceptance rate; the \emph{Mini-SWE Agent} for code optimization on SWE-Perf, with a statistically significant \textbf{10.1\% performance gain}; and the \emph{CrewAI Agent} for cost and mathematical reasoning on Math Odyssey, achieving a statistically significant \textbf{$36.9\%$ reduction} in the number of tokens required for evaluation. We also evaluate the \emph{MathTales-Teacher Agent} powered by a smaller open-source model (Qwen2.5-7B) on GSM8K primary-level mathematics problems, achieving a \textbf{22\% accuracy improvement} and demonstrating that ARTEMIS can optimize agents based on both commercial and local models.

cs.SE↗

arXiv2025

Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approach

Recent advancements in Large Language Models (LLMs) for code optimization have enabled industrial platforms to automate software performance engineering at unprecedented scale and speed. Yet, organizations in regulated industries face strict constraints on which LLMs they can use - many cannot utilize commercial models due to data privacy regulations and compliance requirements, creating a significant challenge for achieving high-quality code optimization while maintaining cost-effectiveness. We address this by implementing a Mixture-of-Agents (MoA) approach that directly synthesizes code from multiple specialized LLMs, comparing it against TurinTech AI's vanilla Genetic Algorithm (GA)-based ensemble system and individual LLM optimizers using real-world industrial codebases. Our key contributions include: (1) First MoA application to industrial code optimization using real-world codebases; (2) Empirical evidence that MoA excels with open-source models, achieving 14.3% to 22.2% cost savings and 28.6% to 32.2% faster optimization times for regulated environments; (3) Deployment guidelines demonstrating GA's advantage with commercial models while both ensembles outperform individual LLMs; and (4) Real-world validation across 50 code snippets and seven LLM combinations, generating over 8,700 variants, addresses gaps in industrial LLM ensemble evaluation. This provides actionable guidance for organizations balancing regulatory compliance with optimization performance in production environments.

cs.SE↗

arXiv2025

Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective

There is a growing interest in leveraging multiple large language models (LLMs) for automated code optimization. However, industrial platforms deploying multiple LLMs face a critical challenge: prompts optimized for one LLM often fail with others, requiring expensive model-specific prompt engineering. This cross-model prompt engineering bottleneck severely limits the practical deployment of multi-LLM systems in production environments. We introduce Meta-Prompted Code Optimization (MPCO), a framework that automatically generates high-quality, task-specific prompts across diverse LLMs while maintaining industrial efficiency requirements. MPCO leverages metaprompting to dynamically synthesize context-aware optimization prompts by integrating project metadata, task requirements, and LLM-specific contexts. It is an essential part of the ARTEMIS code optimization platform for automated validation and scaling. Our comprehensive evaluation on five real-world codebases with 366 hours of runtime benchmarking demonstrates MPCO's effectiveness: it achieves overall performance improvements up to 19.06% with the best statistical rank across all systems compared to baseline methods. Analysis shows that 96% of the top-performing optimizations stem from meaningful edits. Through systematic ablation studies and meta-prompter sensitivity analysis, we identify that comprehensive context integration is essential for effective meta-prompting and that major LLMs can serve effectively as meta-prompters, providing actionable insights for industrial practitioners.

cs.SE↗

arXiv2025

Ensemble Learning for Large Language Models in Text and Code Generation: A Survey

Generative Pretrained Transformers (GPTs) are foundational Large Language Models (LLMs) for text generation. However, individual LLMs often produce inconsistent outputs and exhibit biases, limiting their representation of diverse language patterns. The closed-source nature of many powerful LLMs further restricts industry applications due to data privacy concerns. Inspired by successes in text generation, LLM ensemble techniques are now increasingly explored for code generation. This article reviews these emerging ensemble approaches to enhance understanding, encourage further research, and promote practical implementation in both text and code generation. We categorize LLM ensembles into seven main methods - weight merging, knowledge fusion, mixture-of-experts, reward ensemble, output ensemble, routing, and cascading - analyzing capabilities of those approaches. Our findings highlight key benefits such as improved diversity representation, enhanced output quality, and greater application flexibility. These insights aid model selection for real-world tasks and crucially, lay groundwork for extending ensemble strategies to multimodal LLMs.

cs.CL↗

arXiv2025

Weak-strong uniqueness for solutions to mean-field games

This paper addresses the crucial question of solution uniqueness in stationary first-order Mean-Field Games (MFGs). Despite well-established existence results, establishing uniqueness, particularly for weaker solutions in the sense of monotone operators, remains an open challenge. Building upon the framework of monotonicity methods, we introduce a linearization method that enables us to prove a weak-strong uniqueness result for stationary MFG systems on the d-dimensional torus. In particular, we give explicit conditions under which this uniqueness holds.

math.AP↗

arXiv2025

Language Models for Code Optimization: Survey, Challenges and Future Directions

Language models (LMs) built upon deep neural networks (DNNs) have recently demonstrated breakthrough effectiveness in software engineering tasks such as code generation, completion, and repair. This has paved the way for the emergence of LM-based code optimization techniques, which are crucial for enhancing the performance of existing programs, such as accelerating program execution time. However, a comprehensive survey dedicated to this specific application has been lacking. To fill this gap, we present a systematic literature review of over 50 primary studies, identifying emerging trends and addressing 11 specialized questions. Our findings reveal five critical open challenges, such as balancing model complexity with practical usability, cross-language/performance generalizability, and building trust in AI-driven solutions. Furthermore, we provide eight future research directions to facilitate more efficient, robust, and reliable LM-based code optimization. Thereby, this study aims to provide actionable insights and foundational references for both researchers and practitioners in this rapidly evolving field.

cs.SE↗

arXiv2022

evoML Yellow Paper: Evolutionary AI and Optimisation Studio

Machine learning model development and optimisation can be a rather cumbersome and resource-intensive process. Custom models are often more difficult to build and deploy, and they require infrastructure and expertise which are often costly to acquire and maintain. Machine learning product development lifecycle must take into account the need to navigate the difficulties of developing and deploying machine learning models. evoML is an AI-powered tool that provides automated functionalities in machine learning model development, optimisation, and model code optimisation. Core functionalities of evoML include data cleaning, exploratory analysis, feature analysis and generation, model optimisation, model evaluation, model code optimisation, and model deployment. Additionally, a key feature of evoML is that it embeds code and model optimisation into the model development process, and includes multi-objective optimisation capabilities.

cs.AI↗

arXiv2022

Sharp regularity for singular obstacle problems

We obtain sharp local $C^{1,\alpha}$ regularity of solutions for singular obstacle problems, Euler-Lagrange equation of which is given by $$ \Delta_p u=\gamma(u-\varphi)^{\gamma-1}\,\text{ in }\,\{u>\varphi\}, $$ for $0<\gamma<1$ and $p\ge2$. At the free boundary $\partial\{u>\varphi\}$, we prove optimal $C^{1,\tau}$ regularity of solutions, with $\tau$ given explicitly in terms of $p$, $\gamma$ and smoothness of $\varphi$, which is new even in the linear setting.

math.AP↗

arXiv2017

First-order, stationary mean-field games with congestion

Mean-field games (MFGs) are models for large populations of competing rational agents that seek to optimize a suitable functional. In the case of congestion, this functional takes into account the difficulty of moving in high-density areas. Here, we study stationary MFGs with congestion with quadratic or power-like Hamiltonians. First, using explicit examples, we illustrate two main difficulties: the lack of classical solutions and the existence of areas with vanishing density. Our main contribution is a new variational formulation for MFGs with congestion. This formulation was not previously known, and, thanks to it, we prove the existence and uniqueness of solutions. Finally, we consider applications to numerical methods.

math.AP↗

arXiv2015

Existence of positive solutions for an approximation of stationary mean-field games

Here, we consider a regularized mean-field game model that features a low-order regularization. We prove the existence of solutions with positive density. To do so, we combine a priori estimates with the continuation method. In contrast with high-order regularizations, the low-order regularizations are easier to implement numerically. Moreover, our methods give a theoretical foundation for this approach.

math.AP↗

arXiv2015

Short-time existence of solutions for mean-field games with congestion

We consider time-dependent mean-field games with congestion that are given by a system of a Hamilton-Jacobi equation coupled with a Fokker-Planck equation. The congestion effects make the Hamilton-Jacobi equation singular. These models are motivated by crowd dynamics where agents have difficulty moving in high-density areas. Uniqueness of classical solutions for this problem is well understood. However, existence of classical solutions, was only known in very special cases - stationary problems with quadratic Hamiltonians and some time-dependent explicit examples. Here, we prove short-time existence of $C^\infty$ solutions in the case of sub-quadratic Hamiltonians.

math.AP↗

arXiv2015

Regularity for second order stationary mean-field games

In this paper, we prove the existence of classical solutions for second order stationary mean-field game systems. These arise in ergodic (mean-field) optimal control, convex degenerate problems in calculus of variations, and in the study of long-time behavior of time-dependent mean-field games. Our argument is based on the interplay between the regularity of solutions of the Hamilton-Jacobi equation in terms of the solutions of the Fokker-Planck equation and vice-versa. Because we consider different classes of couplings, distinct techniques are used to obtain a priori estimates for the density. In the case of polynomial couplings, we recur to an iterative method. An integral method builds upon the properties of the logarithmic function in the setting of logarithmic nonlinearities. This work extends substantially previous results by allowing for more general classes of Hamiltonians and mean-field assumptions.

math.AP↗

arXiv2013

On the existence of classical solutions for stationary extended mean field games

In this paper we consider extended stationary mean field games, that is mean-field games which depend on the velocity field of the players. We prove various a-priori estimates which generalize the results for quasi-variational mean field games in [GPSM12]. In addition we use adjoint method techniques to obtain higher regularity bounds. Then we establish existence of smooth solutions under fairly general conditions by applying the continuity method. When applied to standard stationary mean-field games as in [LL06a], [GSM11] or [GPSM12] this paper yields various new estimates and regularity properties not available previously. We discuss additionally several examples where existence of classical solutions can be proved.

math.AP↗