What is an agent in RL?

An agent in reinforcement learning (RL) is an entity that learns to make decisions by interacting with an environment. Its goal is to maximize a cumulative reward signal over time through trial and error. The agent observes the environment’s state, takes actions based on its current strategy (called a policy), and receives feedback in the form of rewards or penalties. For example, in a game-playing scenario, the agent might be an AI that learns to move a character through a maze by trying different paths and adjusting its behavior based on rewards (e.g., points for reaching the goal).

The agent’s behavior is shaped by three core components: the policy, the value function, and optionally a model of the environment. The policy defines the agent’s strategy—like a rulebook that maps states to actions. A value function estimates the expected long-term reward of being in a state or taking an action, helping the agent prioritize better choices. A model, if used, allows the agent to predict how the environment will respond to its actions. For instance, a self-driving car agent might use a policy to decide when to accelerate, a value function to assess the safety of a lane change, and a model to predict traffic patterns based on historical data.

Agents can be categorized based on their approach. Model-free agents, like those using Q-learning, learn directly from interactions without building an environment model. Model-based agents, such as those using Monte Carlo Tree Search (used in AlphaGo), simulate future states to plan actions. Policy-based agents, like those trained with policy gradient methods, optimize their decision-making strategy by adjusting action probabilities. Developers choose these approaches based on problem complexity and available computational resources. For example, a simple grid-world navigation task might use a model-free Q-learning agent, while a complex robotics application could require a model-based approach for precise planning.