Real-Time Tracking in a Status Update System with an Imperfect Feedback Channel

We consider a status update system consisting of a finite-state Markov source, an energy-harvesting-enabled transmitter, and a sink. The forward and feedback channels are error-prone. We study the problem of minimizing the long-term time average of a (generic) distortion function subject to an energy limitation. Since the feedback channel is error-prone, the transmitter has only partial knowledge about the transmission results and, consequently, about the estimate of the source state at the sink. Therefore, we model the problem as a partially observable Markov decision process (POMDP), which is then cast as a belief-MDP problem. The infinite belief-state space makes solving the belief-MDP difficult. Thus, by exploiting a specific property of the belief evolution, we truncate the state space and formulate a finite-state MDP problem, which is then solved using the relative value iteration algorithm (RVIA). Furthermore, we propose an energy-agnostic low-complexity policy in which the belief-MDP problem is transformed into a sequence of per-slot optimization problems. Then, the energy-agnostic low-complexity policy is extended to an energy-aware low-complexity policy by adding a regularization term to the objective function of the per-slot problems. Simulation results show the structure and effectiveness of the proposed policies and their superiority compared to baseline policies.