arXiv subjects

Marian Codreanu

Publications and source records attributed to Marian Codreanu.

At least 19 recordsLinked to original sources

arXiv2026

Age Dispersion and Higher-Order AoI in Status Update Systems

We introduce and characterize \emph{age dispersion} as a measure of temporal consistency in status update systems. Age dispersion is defined as the difference between the ages of the two most recently received updates, and its higher-order extension, the {$k$-th} order age dispersion, captures the difference between the ages of the most recent update and the $(k+1)$-th most recent one. We analyze age dispersion in an M/G/1/1 queueing system. Furthermore, we establish connections between the {$k$-th} order age dispersion and the {$k$-th} order age of information (AoI), where the latter quantifies the age of the {$k$-th} most recently received update.

cs.IT↗

arXiv2026

Squeezing the Most Out of Preemption for AoI Minimization: Single-source Case

In this work, we study a single-source single-server continuous-time status update system where the updates arrive according to a Poisson process and update service times are generally distributed. In our proposed setting, a preemption policy refers to one where a new update preempts the ongoing one with a probability depending on the age of the update in service. We first propose an analytical method to derive the average age of information (AoI) and average peak AoI (PAoI) for any such preemption policy. This analysis is then utilized to tune two particular preemption policies: (i) probabilistic preemption (PP), in which preemption takes place according to a fixed probability regardless of the update age, (ii) threshold-based preemption (TP), for which preemption is incurred when the update age exceeds a certain threshold, both using one-dimensional line search. The effectiveness of policy tuning for the PP and TP policies is validated using lognormal-distributed update service times.

cs.IT↗

arXiv2025

Multi-Source M/G/1/1 Queues with Probabilistic Preemption

We consider a multi-source status update system consisting of multiple independent sources, a single server, and a single sink. Each source generates packets according to a Poisson process, and packets are served according to a general service time distribution. The system has a capacity of one packet, i.e., no waiting buffer, and is modeled as a multi-source M/G/1/1 queueing system. We introduce a probabilistically preemptive packet management policy, under which an existing packet from the same source in the system is replaced by an arriving packet with a fixed probability. We derive the moment generating functions (MGFs) of the age of information (AoI) and peak AoI (PAoI) for each source under this policy. Numerical results demonstrate the effectiveness of the proposed packet management policy.

cs.IT↗

arXiv2025

Remote Tracking with State-Dependent Sensing in Pull-Based Systems: A POMDP Framework

We consider real-time remote tracking of a Markov source observed by multiple heterogeneous sensors with state-dependent sensing accuracy, motivated by distributed camera networks with overlapping coverage and spatial blind spots. Upon commands from a remote sink, sensors transmit their observations over error-prone channels. We aim to minimize the long-term average of a weighted sum of goal-aware distortion and transmission costs. The problem is formulated as a partially observable Markov decision process (POMDP) and cast into an equivalent belief-MDP. To address the intractability of the infinite and continuous belief space, we develop a truncation-based method that yields a finite-state MDP which can be solved via standard methods such as relative value iteration. We further use a discounted reformulation to derive a theoretical lower bound for the optimal average cost, which is tightened via the incremental pruning algorithm (IPA) and also induces a comparison policy. Numerical results demonstrate that the performance of the proposed policy improves with the truncation depth at the expense of computational effort, and also outperforms low-complexity baselines across a wide range of system parameters. The results also reveal a switching-type structure of the truncation-based policy over the belief simplex and quantify the impact of key system parameters, highlighting the importance of accounting for state-dependent~sensing.

eess.SP↗

arXiv2025

Goal-Oriented Remote Tracking Through Correlated Observations in Pull-based Communications

We address the real-time remote tracking problem in a status update system comprising two sensors, two independent information sources, and a remote monitor. The status updating follows a pull-based communication, where the monitor commands/pulls the sensors for status updates, i.e., the actual state of the sources. We consider that the observations are \textit{correlated}, meaning that each sensor's sent data could also include the state of the other source due to, e.g., inter-sensor communications or overlapping monitoring regions. The effectiveness of data communication is measured by a generic distortion, capturing the underlying application goal. We provide optimal command/pulling policies for the monitor that minimize the average weighted sum distortion and transmission cost. Since the monitor cannot fully observe the exact state of each source, we propose a partially observable Markov decision process (POMDP) and reformulate it as a belief MDP problem. We then effectively truncate the infinite belief space and transform it into a finite-state MDP problem, which is solved via relative value iteration. Simulation results show the effectiveness of the derived policy over age-based and deep-Q network baseline policies.

eess.SP↗

arXiv2025

AoI in M/G/1/1 Queues with Probabilistic Preemption

We consider a status update system consisting of one source, one server, and one sink. The source generates packets according to a Poisson process and the packets are served according to a generally distributed service time. We consider a system with a capacity of one packet, i.e., there is no waiting buffer in the system, and model it as an M/G/1/1 queueing system. We introduce a probabilistically preemptive packet management policy and calculate the moment generating functions (MGFs) of the age of information (AoI) and peak AoI (PAoI) under the policy. According to the probabilistically preemptive policy, when a packet arrives, the possible packet in the system is replaced by the arriving packet with a fixed probability. Numerical results show the effectiveness of the packet management policy.

cs.IT↗

arXiv2024

Real-time Tracking in a Status Update System with an Imperfect Feedback Channel

We consider a status update system consisting of a finite-state Markov source, an energy-harvesting-enabled transmitter, and a sink. The forward and feedback channels between the transmitter and the sink are error-prone. We study the problem of minimizing the long-term time average of a (generic) distortion function subject to an energy causality constraint. Since the feedback channel is error-prone, the transmitter has only partial knowledge about the transmission results and, consequently, about the estimate of the source state at the sink. Therefore, we model the problem as a partially observable Markov decision process (POMDP), which is then cast as a belief-MDP problem. The infinite belief space makes solving the belief-MDP difficult. Thus, by exploiting a specific property of the belief evolution, we truncate the state space and formulate a finite-state MDP problem, which is then solved using the relative value iteration algorithm (RVIA). Furthermore, we propose a low-complexity transmission policy in which the belief-MDP problem is transformed into a sequence of per-slot optimization problems. Simulation results show the effectiveness of the proposed policies and their superiority compared to a baseline policy. Moreover, we numerically show that the proposed policies have switching-type structures.

eess.SY↗

arXiv2023

Semantic-aware Sampling and Transmission in Energy Harvesting Systems: A POMDP Approach

We address the problem of real-time remote tracking of a partially observable Markov source in an energy harvesting system with an unreliable communication channel. We consider both sampling and transmission costs. Different from most prior studies that assume the source is fully observable, the sampling cost renders the source partially observable. The goal is to jointly optimize sampling and transmission policies for two semantic-aware metrics: i) a general distortion measure and ii) the age of incorrect information (AoII). We formulate a stochastic control problem. To solve the problem for each metric, we cast a partially observable Markov decision process (POMDP), which is transformed into a belief MDP. Then, for both AoII under the perfect channel setup and distortion, we express the belief as a function of the age of information (AoI). This expression enables us to effectively truncate the corresponding belief space and formulate a finite-state MDP problem, which is solved using the relative value iteration algorithm. For the AoII metric in the general setup, a deep reinforcement learning policy is proposed to solve the belief MDP problem. Simulation results show the effectiveness of the derived policies and, in particular, reveal a non-monotonic switching-type structure of the real-time optimal policy with respect to AoI.

eess.SP↗

arXiv2023

Optimal Semantic-aware Sampling and Transmission in Energy Harvesting Systems Through the AoII

We study a real-time tracking problem in an energy harvesting status update system with a Markov source and an imperfect channel, considering both sampling and transmission costs. The problem primary challenge stems from the non-observability of the source due to the sampling cost. By using the age of incorrect information (AoII) as a semantic-aware performance metric, our main goal is to find an optimal policy that minimizes the time average AoII subject to an energy-causality constraint. To this end, a stochastic optimization problem is formulated and solved by modeling it as a partially observable Markov decision process (POMDP). More specifically, to solve the main problem, we use the notion of a belief state and cast the problem as a belief MDP problem. Then, for the perfect channel setup, we effectively truncate the corresponding belief space and solve the MDP problem using the relative value iteration method. For the general setup, a deep reinforcement learning policy is proposed. The simulation results show the efficacy of the derived policies in comparison to an AoI-optimal policy and an opportunistic baseline policy.

eess.SP↗

arXiv2023

Status Updating under Partial Battery Knowledge in Energy Harvesting IoT Networks

We study status updating under inexact knowledge about the battery levels of the energy harvesting sensors in an IoT network, where users make on-demand requests to a cache-enabled edge node to send updates about various random processes monitored by the sensors. To serve the request(s), the edge node either commands the corresponding sensor to send an update or uses the aged data from the cache. We find a control policy that minimizes the average on-demand AoI subject to per-slot energy harvesting constraints under partial battery knowledge at the edge node. Namely, the edge node is informed about sensors' battery levels only via received status updates, leading to uncertainty about the battery levels for the decision-making. We model the problem as a POMDP which is then reformulated as an equivalent belief-MDP. The belief-MDP in its original form is difficult to solve due to the infinite belief space. However, by exploiting a specific pattern in the evolution of beliefs, we truncate the belief space and develop a dynamic programming algorithm to obtain an optimal policy. Moreover, we address a multi-sensor setup under a transmission limitation for which we develop an asymptotically optimal algorithm. Simulation results assess the performance of the proposed methods.

eess.SY↗

arXiv2022

On the Age-Optimality of Relax-then-Truncate Approach under Partial Battery Knowledge in Energy Harvesting IoT Networks

We consider an energy harvesting (EH) IoT network, where users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an EH sensor. The edge node serves users' requests by either commanding the corresponding sensor to send a fresh status update or retrieving the most recently received measurement from the cache. We aim to find a control policy at the edge node that minimizes the average on-demand AoI over all sensors subject to per-slot transmission and energy constraints under partial battery knowledge at the edge node. Namely, the limited radio resources (e.g., bandwidth) causes that only a limited number of sensors can send status updates at each time slot (i.e., per-slot transmission constraint) and the scarcity of energy for the EH sensors imposes an energy constraint. Besides, the edge node is informed of the sensors' battery levels only via received status update packets, leading to uncertainty about the battery levels for the decision-making.We develop a low-complexity algorithm -- termed relax-then-truncate -- and prove that it is asymptotically optimal as the number of sensors goes to infinity. Numerical results illustrate that the proposed method achieves significant gains over a request-aware greedy policy and show that it has near-optimal performance even for moderate numbers of sensors.

eess.SY↗

arXiv2022

Query-Age-Optimal Scheduling under Sampling and Transmission Constraints

This letter provides query-age-optimal joint sampling and transmission scheduling policies for a heterogeneous status update system, consisting of a stochastic arrival and a generate-at-will source, with an unreliable channel. Our main goal is to minimize the average query age of information (QAoI) subject to average sampling, average transmission, and per-slot transmission constraints. To this end, an optimization problem is formulated and solved by casting it into a linear program. We also provide a low-complexity near-optimal policy using the notion of weakly-coupled constrained Markov decision processes. The numerical results show up to 32% performance improvement by the proposed policies compared with a benchmark policy.

eess.SP↗

arXiv2022

Status Update Control and Analysis under Two-Way Delay

We study status updating under two-way delay in a system consisting of a sampler, a sink, and a controller residing at the sink. The controller controls the sampling process by sending request packets to the sampler. Upon receiving a request, the sampler generates a sample and transmits the status update packet to the sink. Transmissions of both request and status update packets encounter random delays. We develop optimal control policies to minimize the average age of information (AoI) using the tools of Markov decision processes in two scenarios. We begin with the system having at most one active request, i.e., a generated request for which the sink has not yet received a status update packet. Then, as the main distinctive feature of this paper, we initiate pipelining-type status updating by studying a system having at most two active requests. Furthermore, we conduct AoI analysis by deriving the average AoI expressions for the Zero-Wait-1, Zero-Wait-2, and Wait-1 policies. According to the Zero-Wait-1 policy, whenever a status update packet is delivered to the sink, a new request packet is inserted into the system. The Zero-Wait-2 policy operates similarly, except that the system holds two active requests. According to the Wait-1 policy, whenever a status update packet is delivered to the sink, a new request is sent after a waiting time which is a function of the current AoI. Numerical results illustrate the performance of each status updating policy under different system parameter values.

cs.IT↗

arXiv2022

Multi-Source AoI-Constrained Resource Minimization under HARQ: Heterogeneous Sampling Processes

We consider a multi-source hybrid automatic repeat request (HARQ) based system, where a transmitter sends status update packets of random arrival (i.e., uncontrollable sampling) and generate-atwill (i.e., controllable sampling) sources to a destination through an error-prone channel. We develop transmission scheduling policies to minimize the average number of transmissions subject to an average age of information (AoI) constraint. First, we consider known environment (i.e., known system statistics) and develop a near-optimal deterministic transmission policy and a low-complexity dynamic transmission (LC-DT) policy. The former policy is derived by casting the main problem into a constrained Markov decision process (CMDP) problem, which is then solved using the Lagrangian relaxation, relative value iteration algorithm, and bisection. The LC-DT policy is developed via the drift-plus-penalty (DPP) method by transforming the main problem into a sequence of per-slot problems. Finally, we consider unknown environment and devise a learning-based transmission policy by relaxing the CMDP problem into an MDP problem using the DPP method and then adopting the deep Q-learning algorithm. Numerical results show that the proposed policies achieve near-optimal performance and illustrate the benefits of HARQ in status updating.

eess.SY↗

arXiv2022

Status Updating with an Energy Harvesting Sensor under Partial Battery Knowledge

We consider status updating under inexact knowledge of the battery level of an energy harvesting (EH) sensor that sends status updates about a random process to users via a cache-enabled edge node. More precisely, the control decisions are performed by relying only on the battery level knowledge captured from the last received status update packet. Upon receiving on-demand requests for fresh information from the users, the edge node uses the available information to decide whether to command the sensor to send a status update or to retrieve the most recently received measurement from the cache. We seek for the best actions of the edge node to minimize the average AoI of the served measurements, i.e., average on-demand AoI. Accounting for the partial battery knowledge, we model the problem as a partially observable Markov decision process (POMDP), and, through characterizing its key structures, develop a dynamic programming algorithm to obtain an optimal policy. Simulation results illustrate the threshold-based structure of an optimal policy and show the gains obtained by the proposed optimal POMDP-based policy compared to a request-aware greedy (myopic) policy.

eess.SY↗

arXiv2022

Minimizing the AoI in Resource-Constrained Multi-Source Relaying Systems: Dynamic and Learning-based Scheduling

We consider a multi-source relaying system where independent sources randomly generate status update packets which are sent to the destination with the aid of a relay through unreliable links. We develop transmission scheduling policies to minimize the weighted sum average age of information (AoI) subject to transmission capacity and long-run average resource constraints. We formulate a stochastic control optimization problem and solve it using a constrained Markov decision process (CMDP) approach and a drift-plus-penalty method. The CMDP problem is solved by transforming it into an MDP problem using the Lagrangian relaxation method. We theoretically analyze the structure of optimal policies for the MDP problem and subsequently propose a structure-aware algorithm that returns a practical near-optimal policy. Using the drift-plus-penalty method, we devise a near-optimal low-complexity policy that performs the scheduling decisions dynamically. We also develop a model-free deep reinforcement learning policy for which the Lyapunov optimization theory and a dueling double deep Q-network are employed. The complexities of the proposed policies are analyzed. Simulation results are provided to assess the performance of our policies and validate the theoretical results. The results show up to 91% performance improvement compared to a baseline policy.

eess.SP↗

arXiv2022

On-Demand AoI Minimization in Resource-Constrained Cache-Enabled IoT Networks with Energy Harvesting Sensors

We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor. The edge node serves users' requests by deciding whether to command the corresponding sensor to send a fresh status update or retrieve the most recently received measurement from the cache. Our objective is to find the best actions of the edge node to minimize the average age of information (AoI) of the received measurements upon request, i.e., average on-demand AoI, subject to per-slot transmission and energy constraints. First, we derive a Markov decision process model and propose an iterative algorithm that obtains an optimal policy. Then, we develop an asymptotically optimal low-complexity algorithm -- termed relax-then-truncate -- and prove that it is optimal as the number of sensors goes to infinity. Simulation results illustrate that the proposed relax-then-truncate approach significantly reduces the average on-demand AoI compared to a request-aware greedy (myopic) policy and also depict that it performs close to the optimal solution even for moderate numbers of sensors.

eess.SP↗

arXiv2022

Moment Generating Function of Age of Information in Multi-Source M/G/1/1 Queueing Systems

We consider a multi-source status update system, where each source generates status update packets according to a Poisson process which are then served according to a generally distributed service time. For this multi-source M/G/1/1 queueing model, we introduce a source-aware preemptive packet management policy and derive the moment generating functions (MGFs) of the age of information (AoI) and peak AoI of each source. According to the policy, an arriving fresh packet preempts the possible packet of the same source in the system. Furthermore, we derive the MGFs of the AoI and peak AoI for the source-agnostic preemptive and non-preemptive policy, for which only the average AoI and peak AoI have been derived earlier. Finally, we use the MGFs to derive the average AoI and peak AoI in a two-source M/G/1/1 queueing model under each policy. Numerical results show the effect of the service time distribution parameters on the average AoI: for a given service rate, when the tail of the service time distribution is sufficiently heavy, the source-agnostic preemptive policy is the best policy, whereas for a sufficiently light tailed distribution, the non-preemptive policy is the best policy. The results also highlight the importance of higher moments of the AoI.

cs.IT↗