What is AlphaGo, and how did it use reinforcement learning?

AlphaGo is a computer program developed by DeepMind to play the board game Go. Unlike chess, Go has a vastly larger number of possible board configurations, making traditional brute-force search algorithms impractical. AlphaGo combined neural networks and reinforcement learning (RL) to tackle this challenge. It gained prominence in 2016 by defeating Lee Sedol, a top-ranked human player, marking a milestone in artificial intelligence. The system used a mix of supervised learning (training on human games) and RL (self-play) to develop strategies beyond human expertise. Its success demonstrated how machine learning could handle complex, intuition-driven tasks.

AlphaGo’s use of reinforcement learning centered on two neural networks: a policy network and a value network. The policy network predicted the probability of winning for each possible move, guiding the search for optimal actions. The value network estimated the long-term reward of a board position, reducing the need to simulate all possible future moves. These networks were trained through self-play: AlphaGo played millions of games against itself, adjusting its parameters to maximize the chance of winning. For example, if a move led to a loss, the policy network updated to reduce the likelihood of selecting that move in similar future scenarios. This iterative process allowed AlphaGo to discover novel strategies not present in human-played games.

A key technical detail was the integration of Monte Carlo Tree Search (MCTS) with the neural networks. MCTS explored possible move sequences by simulating games, but instead of evaluating every path exhaustively, it used the policy network to prioritize promising branches and the value network to estimate outcomes. For instance, in a critical match against Lee Sedol, AlphaGo’s 37th move in Game 2—a seemingly unconventional placement—was a result of MCTS guided by its networks. This approach balanced exploration (trying new moves) and exploitation (using known good strategies). By combining RL-based self-improvement with efficient search, AlphaGo achieved superhuman performance without relying solely on pre-existing human knowledge.