arXiv Science⌕ Search

arXiv subjects

Pierre Pinson

Publications and source records attributed to Pierre Pinson.

At least 19 recordsLinked to original sources

Prescriptive Learning of Linear Policies with Feasibility Guarantees for Binary Decision-Making

Binary decision-making under uncertainty arises across a wide range of applications, including energy systems scheduling, transportation, and workforce planning. While machine learning has been increasingly applied to accelerate or replace mixed-integer solvers in such settings, existing approaches rely on predictive learning, training models to reproduce optimal decisions from labeled data, which lacks knowledge of counterfactual outcomes and provides no guarantees that the resulting decisions will be feasible. This limitation is further compounded under distributional shift, where policies trained on historical data may produce systematically infeasible decisions. We propose a prescriptive learning framework for binary decision-making that addresses both limitations. Rather than predicting optimal labels, the framework jointly optimizes over linear decision policies and feasible assignments across all training scenarios, allowing feasible alternatives and their operational performance to be considered directly during training. We further extend the framework to a distributionally robust chance-constrained formulation using a Wasserstein ambiguity set, which enforces feasibility with a prescribed probability under the worst-case distribution in the ambiguity set and enables a tunable trade-off between robustness and solution quality. We demonstrate the framework on two case studies, namely a unit commitment problem and a staff allocation problem. Out-of-sample experiments under test distributions that either match or differ from the training distribution show that the robust model is more conservative when the two distributions coincide, while under distributional shift it substantially reduces unmet demand and lowers total cost by up to 76.5% relative to classical predictive learning.

cs.LG↗

Probabilistic Prediction Markets with Intermittent Contributions

Although both data availability and the demand for accurate forecasts are increasing, collaboration between stakeholders is often constrained by data ownership and competitive interests. In contrast to recent proposals within cooperative game-theoretical frameworks, we place ourselves in a more general framework, based on prediction markets. There, independent agents trade forecasts of uncertain future events in exchange for rewards. We introduce and analyse a prediction market operating with a fixed pool of agents that (i) accounts for the historical performance, (ii) adapts to time-varying conditions, while (iii) accommodates intermittent forecast submissions. The proposed design employs robust regression models to learn the optimal forecasts' combination whilst handling missing submissions. Moreover, we introduce a pay-off allocation mechanism that considers both in-sample and out-of-sample performance while satisfying several desirable economic properties. Case studies using simulated and real-world data demonstrate the effectiveness and adaptability of the proposed market design.

cs.LG↗

Learning to Fix: Optimisation-Aware Machine Learning for Accelerated Unit Commitment

Unit Commitment is a computationally demanding mixed-integer linear optimisation problem requiring many binary commitment decisions across a scheduling horizon. To reduce this computational burden, machine learning approaches can predict a subset of these decisions, thereby shrinking the search space explored by the optimisation solver. Existing confidence-based approaches determine which variables to fix based on user-defined probability thresholds, which are agnostic to the downstream optimisation problem. We address this limitation by introducing an optimisation-aware framework that yields generator-specific confidence thresholds based on the impact of fixing errors on Unit Commitment solution quality. This approach won first place in the 2025 EPRI AI-ccelerating Unit Commitment competition. The proposed framework achieves a mean optimality gap below 0.5% while also delivering an average speed-up of more than 20x. It therefore provides a general mechanism for translating probabilistic predictions into controlled reductions of mixed-integer search spaces, directly linking learning decisions to their downstream computational and economic consequences.

eess.SY↗

Ancillary Services in High-Renewable Power Systems: Comparing Market Design, Emerging Trends, and Future Challenges

Rapid growth in renewable generation is changing power-system operation and increasing the importance of ancillary services for security and reliability. As inverter-based resources displace conventional synchronous generation, system operators must procure a broader set of services spanning five categories: frequency control and reserves, voltage and reactive power support, system stability, restoration, and congestion management. This paper compares ancillary-service market design in Great Britain, Germany, Denmark, Italy, California, Texas, and Australia. It examines how these systems define, procure, remunerate, and coordinate services as variable renewable generation increases. The review develops a consistent product-level classification, provides comparable timing visualisations of frequency-control and reserve products, and maps implementation modalities across services and systems. Four trends emerge: faster and more granular frequency-response products, more explicit procurement of stability capabilities including inertia, system strength, and dynamic voltage support, broader participation by storage, demand-side resources, and other inverter-based resources, and greater use of locational, targeted, and technology-neutral procurement. Challenges include limited harmonisation, incomplete valuation of flexibility and stability capabilities whose value depends on location and operating conditions, barriers to participation by distributed and emerging resources, and insufficient coordination between market and operational arrangements. The comparison finds no convergence towards a single market design and supports differentiated but coordinated ancillary-service frameworks reflecting each service's technical, temporal, and locational characteristics.

eess.SY↗

From Intraday Orderbook to Imbalance Price: Understanding Cross-Market Interaction

Power systems with increasing variable renewable generation face greater uncertainty in scheduling and balancing. Intraday and balancing electricity markets facilitate position adjustments and real-time balancing close to delivery. As delivery approaches, continuous intraday market participants exposed to imbalance settlement adjust their positions by trading additional volumes to reduce their imbalance exposure. We conjecture that positions remaining open after intraday trading, together with demand and supply uncertainties affecting physical market participants, influence price formation in the balancing market. This cross-market interaction is, however, rarely studied. To understand this interaction, this paper uses probabilistic modeling to examine how intraday orderbook information reflects subsequent imbalance price formation in Germany and Austria. We compare orderbook representations based on open, high, low, close, and volume, Volume-Weighted Average Price (VWAP), and last mid price across multiple horizons. Each representation is evaluated using the self product, neighboring products, and the product from the neighboring country. We then compare the best orderbook setting with fundamental feature sets and their combinations, followed by an ablation study of the available training history. We show that VWAP with neighboring products provides the best performance in both countries. Combining orderbook and fundamental information reduces testing loss in Germany but increases it in Austria. Using all available observations provides the best overall performance, while excluding 2022 can reduce loss for extreme price samples. These results reveal and help explain country-dependent interactions between the intraday and balancing markets.

q-fin.CP↗

Selling Information to a Competitor

Selling information to a competitor can generate revenue but may also help the rival choose better actions in the market they compete in. We study this trade-off when the buyer's demand for supplemental information depends on its own private information. The buyer selects from a menu of information products offered by the seller, who incurs a competitive loss proportional to the increase in the buyer's probability of choosing the correct downstream action. In a common-prior model with binary states and actions, we characterize profit-maximizing menus for two buyer types and for a continuum of types. We show that selling information can remain profitable despite the competitive loss, but withholding is optimal when this loss exceeds the buyer's valuation of the information. Stronger competition weakly reduces the profitability of selling information and the buyer's overall probability of choosing correctly downstream, though it need not make individual information products less informative.

cs.GT↗

Suboptimality Loss for Inverse Learning from Imperfect Equilibria

Many modern systems involve the strategic interaction of multiple agents. In such settings, observed actions typically reflect equilibrium behavior under utilities that are only partially known. Recovering these hidden utilities from data - the central goal of inverse game theory - is key for prediction, counterfactual analysis, and mechanism design. However, existing approaches based on inverse variational inequalities are highly sensitive to noisy and inconsistent equilibrium observations, thus limiting their applicability. In this paper, we resolve this issue by introducing a game-theoretic suboptimality loss that measures the aggregate utility gain players could obtain by unilaterally deviating from an observed strategy profile. First, we show that this loss is convex and admits an efficient decomposition into player-wise best-responses. Second, we show this loss is sandwiched between the predictability loss and the inverse variational inequality loss, making it a tractable surrogate for equilibrium prediction. Third, we develop a mirror descent algorithm to minimize it and demonstrate on a heterogeneous networked Cournot competition that our approach remains accurate under noisy observations and inconsistent equilibrium data while inverse variational inequality methods produce degenerate estimates.

cs.GT↗

How long is long enough? Finite-horizon approximation of energy storage scheduling problems

Energy storage scheduling problems, where a storage system is operated to maximize its profit in response to a price signal, are naturally formulated as infinite-horizon optimization problems, since storage systems operate continuously, without a foreseen end to their operation. Such problems can be solved to optimality with a rolling-horizon approach, provided that the planning horizon over which the problem is solved is long enough. Such a horizon is termed a forecast horizon. Despite its importance, the planning horizon is usually chosen arbitrarily for such applications. We introduce an easy-to-check condition that confirms whether a planning horizon is a forecast horizon, and which can be used to derive a bound on suboptimality when it is not the case. In practice, this condition enables practitioners to evaluate whether the planning horizon chosen is sufficient, thereby providing, for the first time, a practical means of evaluating and selecting planning horizons for energy storage scheduling problems. We also derive a lower bound on the minimum forecast horizon. Building on the theoretical results, we develop an algorithm to determine the minimum forecast horizon. It enables the identification, a posteriori, of the shortest planning horizon that guarantees optimal rolling-horizon decisions while avoiding unnecessary forecasting effort and computational cost. Numerical experiments illustrate the practical use of the proposed framework and investigate how storage system characteristics and electricity price patterns influence the minimum forecast horizon.

math.OC↗

Control-Oriented Scenario Tree Construction through Reinforcement Learning

Multistage stochastic model predictive control (MPC) handles uncertainty by optimizing over a scenario tree, a finite branching approximation of future outcomes constructed from sampled forecasts. To build such a tree, conventional methods focus on matching the underlying probability distribution---e.g., via Wasserstein-based scenario reduction---but improved distributional accuracy does not necessarily yield better control performance. We propose a control-oriented approach that learns scenario tree construction directly from its impact on downstream decisions. Fixing the tree topology, we formulate tree construction as a sequential assignment of sampled scenarios to leaves. This assignment is parameterized by an attention-based policy over the scenario set and trained using reinforcement learning, with closed-loop control profit as the objective. Training is stabilized by an asymmetric critic that leverages realized future trajectories. We evaluate the method on a risk-averse battery arbitrage problem. Across a range of forecast set sizes, the learned construction consistently achieves the highest profit, outperforming classical forward and backward reduction methods and certainty-equivalent (single-trajectory forecast) control. The learned policy also exhibits greater robustness on challenging instances, consistently demonstrating better tail-risk characteristics. Analysis of the resulting trees indicates that our method constructs compact, selectively branching structures that capture high-impact events while keeping most trajectories nearly deterministic. These findings highlight that the value of a scenario tree depends critically on the decisions it supports, and provide an effective framework to train scenario tree constructors merely based on the closed-loop control optimization signal.

cs.AI↗

QueryMarket: Cost-Aware Online Active Learning in Data Markets

Data acquisition is a major bottleneck for learning in real-time streams: analysts must decide on the fly which labels to purchase while respecting a rolling budget. However, existing online active learning rarely unifies pricing, information gain, and rolling budget constraints under concept drift. We introduce QueryMarket, a market-inspired framework that queries each incoming data point based on its estimated utility to the model and its price. Within this framework, we propose OVBAL (online variance-based active learning), which integrates data pricing with information-driven selection by estimating each sample's marginal utility via a D-optimality criterion with exponential forgetting and executing cost-aware purchases under rolling budget constraints. OVBAL yields a simple, fully online decision rule that adapts to nonstationary streams and heterogeneous label costs. Experiments on synthetic data and a real-world solar power generation forecasting task show that OVBAL is particularly effective under seller-centric pricing and yields a more favorable long-run error-cost trade-off in the real-world task under both pricing schemes.

cs.LG↗

Decision-calibrated prediction sets for robust power system operations

Robust optimization offers a tractable approach to balance operating costs and reliability in power systems dominated by weather-dependent renewable uncertainty, but its performance depends critically on the uncertainty set. Standard data-driven approaches often calibrate uncertainty sets to attain predictive coverage, which can produce unnecessarily large sets and costly operating decisions. In contrast, we introduce decision-calibrated prediction sets and embed them as uncertainty sets in robust optimization problems; these are conditional multivariate prediction sets where calibration is defined in terms of the reliability of downstream decisions, rather than in terms of the coverage. First, we learn these conditional prediction sets as sub-level sets of norm-based score functions represented by partially input-convex neural networks, capturing contextual information and multivariate dependence while preserving convexity and tractability in downstream robust formulations. Second, inspired by conformal risk control, we calibrate a score-threshold parameter that sets the volume of the uncertainty set, thereby controlling the expected violations of downstream operational constraints. We apply our approach to 15-minute-ahead reserve scheduling with network-constrained deliverability, which we formulate as a robust DC optimal power flow problem with affine recourse. Numerical experiments show that decision-calibrated sets attain prescribed constraint-satisfaction targets within about three percentage points, whereas standard coverage-based calibration systematically exceeds these targets by more than eleven percentage points, leading to larger sets and higher operating costs.

math.OC↗

S3TS: Stochastic Scenario-Structured Tree Search for Advanced Planning Under Uncertainty

Effective scheduling in the energy sector is essential to ensure the reliable operation of electrical grids and their connected assets by, for instance, optimizing the dispatch of generation units and storage systems. An effective planning strategy must (a) accommodate advanced and potentially non-linear system models -- exploiting the increasing data availability of modern grids, and (b) explicitly handle uncertainties arising, for instance, from the integration of renewable energy sources. While existing approaches can address either non-linearity (e.g., Monte Carlo Tree Search) or uncertainty (e.g., stochastic mathematical optimization), there is a lack of planning techniques capable of addressing both challenges simultaneously. To bridge this gap, we propose a Stochastic Scenario-Structured Tree Search (S3TS) algorithm that explicitly represents uncertainty through scenario trees while enabling the integration of advanced non-linear models. We evaluate S3TS on a simulated demand response signal publication problem, largely mimicking the imbalance settlement mechanism in Belgium. The results demonstrate near-optimal performance in linear, analytically tractable settings, with costs within 14% of the mathematically optimal solution conditioned to the scenario trees. In highly non-linear scenarios, S3TS significantly outperforms baseline methods, achieving cost reductions of up to 51% and 5.4% compared to a myopic algorithm and deterministic MCTS, respectively.

cs.AI↗

Stabilizing distribution-free probabilistic forecasts

Multi-step-ahead forecasts are often updated as new observations become available, since shorter forecast horizons typically improve forecast quality. However, such improvements come at the cost of forecast instability, i.e., variability in forecasts for the same target period. This instability can trigger costly changes to plans formulated based on the forecasts and may erode trust in the forecasting system. In this work, we integrate forecast stability alongside forecast quality into the training of distribution-free probabilistic time-series forecasting models, allowing us to control this trade-off. We propose a method for generating stabilized forecasted conditional quantile functions using regression splines parameterized by a neural network. This approach enables joint optimization of quality and stability, as it allows us to directly penalize dissimilarities arising from forecast updates. Furthermore, it allows assigning varying importance to stabilizing different parts of the forecast distributions (e.g., central parts vs. tails) to focus on the parts most relevant for the intended downstream use (e.g., the upper tail for inventory management). We empirically evaluate the proposed method on two datasets with different statistical properties and show that it can effectively reduce forecast instability without a substantial loss in forecast quality, and that it can target stabilization effort toward specific parts of the forecast distributions.

cs.LG↗

Counter-Dyna: Data-Efficient RL-Based HVAC Control using Counterfactual Building Models

Model-based reinforcement learning (MBRL) offers a promising approach for data-efficient energy management in buildings, combining the strengths of predictive modeling and reinforcement learning. While previous MBRL methods applied to HVAC control have reduced training data requirements, they still require several months of interaction with the building to learn a satisfactory control policy. A key reason is that existing surrogate models attempt to predict the entire state-space, including weather and electricity prices that are unaffected by control actions, or completely ignore these variables. Addressing these issues, we propose Counter-Dyna, a method that enhances the data-efficiency of Dyna, an MBRL method. We create data-efficient counterfactual surrogate models (CSM) by leveraging invariances in the state-space. Using a CSM in Dyna speeds up RL training measured in environment interaction data compared to previous results. In comparison with previous state-of-the-art that used 6-12 months of environment interactions, our method needs only 5 weeks. We evaluate our method in a large simulation study using the literature standard BOPTEST framework and proximal policy algorithm (PPO) as the RL algorithm. Our results show cost-saving potentials of 5.3% to 17.0% in a hypothetical deployment scenario. Our work is a significant step towards making real-world deployment of RL algorithms in HVAC control practically viable.

cs.LG↗

How to Purchase Labels? A Cost-Effective Approach Using Active Learning Markets

We introduce and analyse active learning markets as a way to purchase labels, in situations where analysts aim to acquire additional data to improve model fitting, or to better train models for predictive analytics applications. This comes in contrast to the many proposals that already exist to purchase features and examples. By originally formalising the market clearing as an optimisation problem, we integrate budget constraints and improvement thresholds into the label acquisition process. We focus on a single-buyer-multiple-seller setup and propose the use of two active learning strategies (variance based and query-by-committee based), paired with distinct pricing mechanisms. They are compared to benchmark baselines including random sampling and a greedy knapsack heuristic. The proposed strategies are validated on real-world datasets from two critical application domains: real estate pricing and energy forecasting. Results demonstrate the robustness of our approach, consistently achieving superior performance with fewer labels acquired compared to conventional methods. Our proposal comprises an easy-to-implement practical solution for optimising data acquisition in resource-constrained environments.

cs.LG↗

Synergies between AI Computing and Power Systems: Metrics, Scheduling, and Resilience

In this paper, we first clarify the concepts of green AI versus frugal AI, positioning frugality as efficiency by design and green AI as transparency and accountability. We then argue that these approaches, while complementary, are insufficient without a shared quantitative foundation that links AI computing to power system contexts. This motivates the development of standardized carbon metrics as a bridge between algorithmic decisions and their physical consequences. We next embed these signals into scheduling and planning frameworks, presenting two architectures: (i) an iterative signal-response loop for real-time operations, and (ii) an integrated optimization that learns and encodes flexible-load behavior for long-term planning. Finally, we show how the same coordination stack supports resilience, enabling signals to shift from emissions-first to stability-first during stress events, providing targeted relief and faster restoration.

eess.SY↗

Marginalize, Rather than Impute: Probabilistic Wind Power Forecasting with Incomplete Data

Machine learning methods are widely and successfully used for probabilistic wind power forecasting, yet the pervasive issue of missing values (e.g., due to sensor faults or communication outages) has received limited attention. The prevailing practice is impute-then-predict, but conditioning on point imputations biases parameter estimates and fails to propagate uncertainty from missing features. Our approach treats missing features and forecast targets uniformly: we learn a joint generative model of features and targets from incomplete data and, at operational deployment, condition on the observed features and marginalize the unobserved ones to produce forecasts. This imputation-free procedure avoids error introduced by imputation and preserves uncertainty aroused from missing features. In experiments, it improves forecast quality in terms of continuous ranked probability score relative to impute-then-predict baselines while incurring substantially lower computational cost than common alternatives.

cs.LG↗

Value-oriented forecast reconciliation for renewables in electricity markets

Forecast reconciliation is considered an effective method to achieve coherence (within a forecast hierarchy) and to improve forecast quality. However, the value of reconciled forecasts in downstream decision-making tasks has been mostly overlooked. In a multi-agent setup with heterogeneous loss functions, this oversight may lead to unfair outcomes, hence resulting in conflicts during the reconciliation process. To address this, we propose a value-oriented forecast reconciliation approach that focuses on the forecast value for all individual agents. Fairness is ensured through the use of a Nash bargaining framework. Specifically, we model this problem as a cooperative bargaining game, where each agent aims to optimize their own gain while contributing to the overall reconciliation process. We then present a primal-dual algorithm for parameter estimation based on empirical risk minimization. From an application perspective, we consider an aggregated wind energy trading problem, where profits are distributed using a weighted allocation rule. We demonstrate the effectiveness of our approach through several numerical experiments, showing that it consistently results in increased profits for all agents involved.

stat.ML↗