fenshot / Accuracy
How accurate is fenshot?
We measure it on chess images where the right answer is known: boards drawn by chess.com and Lichess themselves, book diagrams typeset the way printed books are, and real screenshots posted on Reddit. Below are the results, the method, and every place where fenshot fails. The test, the data and the results are open source, so anyone can re-run them.
Results by source
"Right" means every one of the 64 squares matches. A warning means fenshot told the user the read may be wrong and offered the editor to fix it. The red column, wrong with no warning, is the one that matters most.
| Source | Images | Right | Right, but warned | Wrong, warned | Wrong, no warning | No board found |
|---|---|---|---|---|---|---|
| chess.com analysis page, desktop and phone | 60 | 58 | 0 | 2 | 0 | 0 |
| Lichess analysis page, desktop and phone | 60 | 60 | 0 | 0 | 0 | 0 |
| Lichess page with other piece sets and boards | 60 | 41 | 6 | 12 | 1 | 0 |
| chess.com board images, 14 piece sets and 12 boards | 240 | 188 | 14 | 35 | 3 | 0 |
| Book diagrams typeset with LaTeX, hatched squares | 120 | 55 | 33 | 32 | 0 | 0 |
| Reddit screenshots, r/chess, r/chessbeginners, r/TournamentChess | 488 | 390 | 29 | 45 | 19 | 5 |
| Piece artwork it was trained on | 363 | 330 | 20 | 9 | 4 | 0 |
| Piece artwork it has never seen | 177 | 72 | 33 | 72 | 0 | 0 |
What changed since the first run
10 November 2026. The first run showed that fenshot could not find the board in printed-style diagrams: the hatching inside the dark squares hid the board's grid from it. On marbled boards the read could also slide by one square. fenshot now takes a second look at a softened copy of the image whenever the first look is not clearly right. The piece reader itself is unchanged.
| Read right | First run | Now |
|---|---|---|
| Book diagrams | 1 / 120 | 88 / 120 |
| chess.com and Lichess in other piece sets and boards | 211 / 300 | 249 / 300 |
| chess.com and Lichess pages in their default look | 118 / 120 | 118 / 120 |
| Reddit screenshots | 405 / 488 | 419 / 488 |
Where fenshot struggles
- Board direction in endgames. fenshot reads the squares, then works out which side is at the bottom from where the pawns stand. In endgames that goes wrong in about 4 to 7 positions out of 100: every square is right, but the board is turned around, and there is no warning. Every wrong read without a warning on the drawn boards is of this kind.
- 3D piece sets. chess.com's 3D wood and 3D plastic pieces are almost never read right. They were left out of training on purpose, and fenshot warns instead of guessing.
- Small book diagrams. When the squares of a printed diagram are about 30 pixels wide or less, fenshot often gets one to three squares wrong, and warns.
- Unusual piece sets. Lichess's horsey and monarchy pieces are misread; monarchy was never in training.
- Busy real screenshots. Misses on Reddit include phone screenshots with overlays, very small boards, and images with more than one board.
When a read is uncertain, fenshot shows a warning and opens the editor next to the original image, so a wrong square takes seconds to fix. The common case, a chess.com or Lichess screenshot in the default look, is read right nearly every time.
How we measured
- Positions. Real positions from the Lichess puzzle database: equal numbers of opening, middlegame and endgame positions, chosen by a fixed rule, not by hand.
- Images we did not draw. Every site board is drawn by the site itself from the position we give it: chess.com's board image service, screenshots of chess.com's and Lichess's analysis pages at desktop and phone sizes, and Lichess with its own piece set and board settings changed. Each chess.com page was checked to show the position we asked for before it was kept.
- Book diagrams. 120 diagrams typeset with LaTeX in the chess fonts printed books use, with hatched dark squares: full pages with text, tight crops, a scanned-page look, small diagrams, and Black's side. They cover hatched diagrams only; books that print solid grey squares are not in the set yet.
- Pages without a board. Encyclopedia articles, news sites, documentation and chess site pages with no board, including checkerboards, graph paper and tiles, to test that fenshot does not invent a position.
- Reddit. Chess images posted on Reddit, where the Chessvision.ai Reddit bot also replied with its reading. Where fenshot and the bot agree, the read counts as right; a random sample of these was compared to the image by eye and no mismatch was found. Every disagreement counts against fenshot, even when fenshot was the one that was right, so the Reddit figure is a floor. The set only includes images the bot answered, which may leave out the hardest ones.
- Same code as the product. The test runs the exact recognizer and model that the website and the extension use, including the confidence check that decides when to warn.
- Honest about training. The model was trained on piece and board artwork from chess.com and Lichess, drawn on synthetic positions. None of the test positions, pages or screenshots were used in training, and the table separates artwork it has seen from artwork it has not.
Re-run it yourself
The benchmark lives in the open source repository under tools/benchmark: the list of positions, every image's source and fingerprint, the Reddit posts with both readings, and the results above, image by image. Rebuild the images with npx tsx tools/benchmark/build.ts and score them with npx tsx tools/benchmark/run.ts.