What is a reward in RL?

In reinforcement learning (RL), a reward is a numerical signal that an agent receives from its environment after taking an action. It serves as feedback to guide the agent toward achieving its goal. The agent’s objective is to learn a policy—a strategy for choosing actions—that maximizes the total accumulated rewards over time. Rewards are fundamental because they define the problem the agent is trying to solve. For example, in a game, a reward might be +1 for winning, -1 for losing, and 0 for all other steps. Without a reward signal, the agent would have no direction to improve its behavior.

Rewards are typically defined by a reward function, which is part of the environment’s design. This function specifies how much reward the agent gets for each state-action pair or state transition. For instance, in a robotics task where a robot must navigate a maze, the reward function might give +10 for reaching the end, -5 for hitting a wall, and -0.1 for every step taken to encourage efficiency. The choice of rewards directly impacts what the agent learns. Poorly designed rewards can lead to unintended behaviors—like the agent prioritizing short-term gains over long-term success—or even exploiting loopholes in the reward system. Developers often start with simple reward structures and iteratively refine them based on observed agent behavior.

A key challenge in RL is balancing immediate rewards with future outcomes. This is addressed using a discount factor, which reduces the value of future rewards in the agent’s calculations. For example, a discount factor of 0.9 means a reward received two steps later is worth 0.81 times its original value. This encourages the agent to prioritize actions that yield higher rewards sooner. Rewards can also be sparse (e.g., only given at the end of a task) or dense (frequent feedback), with sparse rewards often making learning harder. In practice, developers might use techniques like reward shaping—adding intermediate rewards—to help the agent learn faster. For example, a self-driving car simulation might reward the agent for staying in the lane or maintaining a safe speed, not just reaching the destination.