When a fair game of chess becomes a matter of statistics in the courtroom

How can you tell when a player has cheated?

The chess world was rattled in late 2022 when Magnus Carlsen, the current world champion, accused Hans Niemann, a 19-year-old US chess grandmaster, of cheating using a chess-playing artificial intelligence (AI) system. Niemann had defeated Carlsen, prompting Carlsen’s accusation; Niemann asserted that he had played Carlsen fairly even though he later admitted to having cheated twice in online chess games, at the ages of 12 and 16.

A month later, a 72-page investigation report drafted by Chess.com claimed that Niemann had “likely cheated” more than a hundred times while playing online chess. But the report also said, “There is no direct evidence that proves Hans cheated at the September 4, 2022, game with Magnus.”

Cheating in chess has become a major problem, especially in the online era. Among the more-than 500,000 accounts that Chess.com has terminated for cheating, more than 500 belonged to titled players (titling is a mark of skill). By the beginning of 2024, the site expects to close more than a million accounts.

How can you tell when a player has cheated? First, researchers build a statistical model using the database of millions of finished chess matches. Then they estimate the probability that a human player’s move will coincide with a move made by a chess engine using the fitted model.

This is somewhat like a DNA crime-scene analysis for every chess player in the world. Chess engines like Leela Chess Zero and Stockfish aren’t only better players than their human counterparts (on average) but also play differently. Stockfish has an Elo rating of more than 3,500, compared to Carlsen’s 2014 Elo score of 2,882, the highest that a human has ever achieved. Additionally, engines’ playing styles may well be from another planet because they’re developed differently than humans develop their styles. So the likelihood of cheating is said to increase when the correlation between a player’s moves and those of chess engines increases.

By feeding records of Niemann’s games into chess engines, some experts discovered that Niemann had played a lengthy series of AI-recommended moves in tournament games and that his tactics were frequently similar to those of a computer. But some experts contended that the onboard movements in actual games of many players could resemble those of an AI, since human players’ training, preparation, and practices are now affected by these engines as well.

The Carlsen-Niemann dispute may finally be decided in court: Niemann has sued Carlsen, Chess.com and chess prodigy Hikaru Nakamura, who also accused Niemann of cheating in online games, for $100 million for defamation. It will hardly be the first instance of statistics being crucial to legal proceedings. There are numerous instances in the USA, the UK and other countries where statistical theories – primarily those related to calculating probabilities – have been applied in both good and bad ways.