All Lessons

From Rewards to a Learnable Policy

Learn the basic reinforcement-learning loop and what the word policy means in PPO.

AI Narration Press play to listen
0  / 6 paragraphs
Click any paragraph to jump · Scroll freely without breaking narration

Reinforcement learning begins with an agent and an environment. The agent observes a situation, chooses an action, and receives a reward plus a new observation. Unlike supervised learning, no label says which action was correct.

An episode is one complete attempt at a task. A step contains the current observation, selected action, reward, next observation, and a signal saying whether the episode ended. Keeping these fields separate prevents many implementation errors.

A policy is the agent's decision rule. It maps an observation to probabilities over actions. In a maze, a policy might assign an eighty percent chance to moving right and twenty percent to moving left. Sampling those probabilities permits exploration.

The agent tries to maximize return: the sum of present and future rewards. A discount factor between zero and one controls how strongly later rewards matter. This lets an unrewarded action still be useful when it leads toward a later goal.

Rewards describe the task, so poorly chosen rewards teach poorly chosen behavior. Rewarding movement may teach endless wandering. Penalizing every step too heavily may teach the agent to quit. Always inspect what the reward encourages in practice.

Build a first rollout loop that resets the environment, samples actions, records transitions, and reports episode return and length. Also plot action frequencies. These checks reveal stuck policies, missing exploration, and reward exploits before the algorithm becomes complicated.