AlphaZero
By Google DeepMind
AlphaZero is a general-purpose reinforcement learning algorithm developed by Google DeepMind that masters chess, shogi, and Go starting from random play, using only the rules of each game and self-play, without any human game data or…
Definition
AlphaZero is a general-purpose reinforcement learning algorithm developed by Google DeepMind that masters chess, shogi, and Go starting from random play, using only the rules of each game and self-play, without any human game data or game-specific heuristics.
Overview
AlphaZero, introduced in 2017, generalized the approach behind AlphaGo Zero — DeepMind's version of AlphaGo that learned Go purely through self-play — into a single algorithm capable of learning multiple different board games. Given only the rules of a game, AlphaZero starts with random play and improves entirely through self-play reinforcement learning, using a deep neural network to evaluate positions and suggest moves, guided by Monte Carlo tree search. Unlike traditional game-playing engines such as Stockfish (chess) or earlier Go programs, which relied heavily on hand-crafted evaluation functions and domain-specific search optimizations refined over decades, AlphaZero learns its own understanding of strategy from scratch. Within hours of self-play training, it reportedly surpassed the strongest existing chess and shogi engines of the time and matched AlphaGo Zero's superhuman Go performance — using the same general algorithm across all three games, with no game-specific tuning beyond the rules themselves. AlphaZero's chess play was particularly notable to the chess community for its unconventional, sometimes sacrificial style, favoring long-term positional and mobility advantages over some patterns valued by traditional engines. This sparked substantial discussion among professional players and engine developers about what "optimal" chess strategy actually looks like when discovered by a system unconstrained by human opening theory. AlphaZero's demonstration that one general algorithm could master multiple complex games from scratch was an important step toward more broadly capable reinforcement learning systems, and its core techniques directly informed MuZero, a later DeepMind system that extended the approach to work even when the rules of the environment are not known in advance.
Key Concepts
- General algorithm that learns chess, shogi, and Go using the same approach
- Learns entirely through self-play reinforcement learning, given only the game rules
- No reliance on human game data or hand-crafted evaluation heuristics
- Combines a deep neural network with Monte Carlo tree search
- Surpassed the strongest existing chess and shogi engines within hours of training
- Directly generalized from the AlphaGo Zero self-play approach