Reasoning eval · Traces
The evidence behind the numbers
One row per model per puzzle. Open any of them to see the position, what the model wrote about it, and the engine’s verdict on every claim in that reasoning — the headline rates are these, summed.
600 traces
| Puzzle | Model | Rating | Move | Claims true | Breakdown | Opens with |
|---|---|---|---|---|---|---|
| asq4q | Claude Sonnet 5 | 1100 | Missed | — | 0 · 0 · 0 | |
| asq4q | GPT-5.2 | 1100 | Solved | 82% | 9 · 2 · 2 | White has a direct attack on the exposed black king on a5. |
| asq4q | Qwen3.8 A95B | 1100 | Missed | 25% | 2 · 6 · 3 | Let me analyze the position. |
| JaWSF | Claude Sonnet 5 | 1106 | Missed | 67% | 2 · 1 · 2 | Black's a2 pawn is one step from promotion, which is the most urgent threat. |
| JaWSF | GPT-5.2 | 1106 | Missed | 50% | 1 · 1 · 1 | White has no sound direct check: 1.Rh7+ fails to 1...Kxh7 because h7 is not protected. |
| JaWSF | Qwen3.8 A95B | 1106 | Missed | 25% | 3 · 9 · 0 | Looking at the position: White has rooks on f7 and c1, king on g2, pawns on f2 and g3. |
| 2UfqJ | Claude Sonnet 5 | 1111 | Missed | 67% | 2 · 1 · 1 | White threatens Nxg7+ forking king and rook h8, winning material. |
| 2UfqJ | GPT-5.2 | 1111 | Missed | 50% | 3 · 3 · 0 | Black has no immediate winning check: …Qxf5 loses the queen to Qxf5, …Qh4+ is not check because f2 blocks the diagonal to e1, and other queen checks are either blocked by the e5… |
| 2UfqJ | Qwen3.8 A95B | 1111 | Solved | 60% | 3 · 2 · 0 | Looking at the position, I notice that Black's pawn on h5 can capture White's queen on g4 (hxg4). |
| uwO40 | Claude Sonnet 5 | 1113 | Missed | 67% | 2 · 1 · 0 | Queens are mutually attacking each other (Wd6 vs Bc7). |
| uwO40 | GPT-5.2 | 1113 | Solved | 50% | 2 · 2 · 1 | Black can exploit a deflection tactic: the white rook on d1 is the only piece that can recapture on d6 after ...Qxd6. |
| uwO40 | Qwen3.8 A95B | 1113 | Solved | 57% | 4 · 3 · 1 | Looking at the position, White's queen on d6 is defended by the rook on d1. |
| ZT4Ab | Claude Sonnet 5 | 1114 | Missed | 0% | 0 · 4 · 2 | The black knight on e4 is unsupported apart from tactical tricks; simply attacking it with f2-f3 forces it to move, gaining tempo and space while keeping the strong Ne5 and cent… |
| ZT4Ab | GPT-5.2 | 1114 | Missed | 100% | 3 · 0 · 3 | White can win material by pushing the c-pawn: 11.c5 attacks the black queen on d6. |
| ZT4Ab | Qwen3.8 A95B | 1114 | Missed | 67% | 6 · 3 · 0 | Looking at the position, White has a strong knight on e5 and Black has a knight on e4 that's centrally placed. |
| DxWfR | Claude Sonnet 5 | 1115 | Solved | 80% | 4 · 1 · 0 | The king is in check from the rook on e1 and possibly the queen; capturing the checking rook with the rook on f1 removes that threat and the queen does not give check along that… |
| DxWfR | GPT-5.2 | 1115 | Solved | 33% | 2 · 4 · 1 | White is in check from the black rook on e1 along the 1st rank (e1–c1). |
| DxWfR | Qwen3.8 A95B | 1115 | Missed | 13% | 1 · 7 · 3 | Looking at the position, the white king on c1 is in check from the black rook on e1 (along the first rank, with d1 empty between them). |
| wAgLF | Claude Sonnet 5 | 1127 | Missed | 86% | 6 · 1 · 2 | The knight on e4 is attacked by both the f3 pawn and the bishop on d3, so it must move. |
| wAgLF | GPT-5.2 | 1127 | Solved | 100% | 5 · 0 · 2 | Black can immediately exploit White’s uncastled king: 10...Qh4+ gives a strong diagonal check (h4–e1). |
| wAgLF | Qwen3.8 A95B | 1127 | Solved | 86% | 6 · 1 · 2 | Black has a powerful knight on e4 and White's king is still stuck on e1. |
| Tf9Cg | Claude Sonnet 5 | 1132 | Missed | — | 0 · 0 · 0 | |
| Tf9Cg | GPT-5.2 | 1132 | Solved | 44% | 4 · 5 · 1 | White has a direct mating attack on the exposed black king. |
| Tf9Cg | Qwen3.8 A95B | 1132 | Missed | 42% | 8 · 11 · 1 | Looking at this position, White has a queen on h2, rooks on d1 and h1, bishop on g5, and knight on e2. |
| xjzsD | Claude Sonnet 5 | 1134 | Solved | 0% | 0 · 2 · 2 | The knight on d6 can jump to c4 with check, forking the white king on d2 and attacking pawns on a3/b2 while being safe from immediate capture, winning material. |
| xjzsD | GPT-5.2 | 1134 | Missed | 20% | 1 · 4 · 1 | Black’s e5-knight can jump to f3 with check, forking the white king on d2 and the rook on e1. |
| xjzsD | Qwen3.8 A95B | 1134 | Missed | 33% | 3 · 6 · 1 | The position features Black's advanced pawn on d3, directly blocked by White's king on d2. |
| z2EKn | Claude Sonnet 5 | 1137 | Missed | 25% | 1 · 3 · 3 | The black queen on d5 is currently unchallenged and centralizing dangerously. |
| z2EKn | GPT-5.2 | 1137 | Solved | 75% | 3 · 1 · 2 | White has a forcing check with 24.Qf6+, and because the check is from an adjacent file it cannot be blocked and the queen cannot be captured (Kxf6 is illegal since f6 is protect… |
| z2EKn | Qwen3.8 A95B | 1137 | Solved | 50% | 2 · 2 · 0 | Looking at the position, White has Queen on a1, Knight on g4, and Bishop on h4 targeting the Black king on f7. |
| E302L | Claude Sonnet 5 | 1138 | Missed | 75% | 3 · 1 · 1 | Black's queen on c2 attacks the bishop on d2. |
| E302L | GPT-5.2 | 1138 | Solved | 67% | 4 · 2 · 0 | White has a forcing mating attack: 24.Qf7+ gives a diagonal check on g8 that Black cannot capture or block, so Black’s only legal reply is 24...Kh8. |
| E302L | Qwen3.8 A95B | 1138 | Solved | 100% | 5 · 0 · 3 | Looking at the position, White's queen on f3 and rook on f1 are aligned on the f-file pointing toward Black's king on g8. |
| MaJk5 | Claude Sonnet 5 | 1138 | Solved | 40% | 2 · 3 · 1 | Black's knight can deliver a devastating check: 1...Ne2+ forces the white king to h1 or h2, since nothing can block or capture the knight. |
| MaJk5 | GPT-5.2 | 1138 | Solved | 67% | 6 · 3 · 0 | Black has a forcing tactic based on the advanced g2-pawn. |
| MaJk5 | Qwen3.8 A95B | 1138 | Solved | 50% | 2 · 2 · 1 | Black's pawn on g2 is one step from promotion but blocked by White's king on g1. |
| rPWjo | Claude Sonnet 5 | 1139 | Missed | 0% | 0 · 2 · 1 | Black's rook can deliver a check on the same rank as White's king, forcing the king to move and gaining tempo while keeping the rook and knight coordinated against White's advan… |
| rPWjo | GPT-5.2 | 1139 | Solved | 0% | 0 · 4 · 2 | Black can win material immediately with a knight fork. |
| rPWjo | Qwen3.8 A95B | 1139 | Solved | 60% | 3 · 2 · 0 | Looking at the position: Black has Kg7, Rb3, Nc3, pawn f6. |
| vUrqT | Claude Sonnet 5 | 1140 | Solved | 100% | 3 · 0 · 2 | Black's bishop is blocking its own rook on e4 from attacking the queen on e2. |
| vUrqT | GPT-5.2 | 1140 | Solved | 100% | 5 · 0 · 0 | Black’s bishop on e3 is currently blocking the e4-rook from attacking White’s queen on e2. |
| vUrqT | Qwen3.8 A95B | 1140 | Missed | 17% | 2 · 10 · 3 | Let me analyze the position carefully. |
| iS9Y2 | Claude Sonnet 5 | 1142 | Missed | 33% | 1 · 2 · 1 | Black king is trapped on h7 with pawns on g7. |
| iS9Y2 | GPT-5.2 | 1142 | Solved | 91% | 10 · 1 · 0 | White can force mate by using the g-pawn to drive the black king onto a square where the rook delivers an unanswerable check. |
| iS9Y2 | Qwen3.8 A95B | 1142 | Missed | 20% | 1 · 4 · 1 | The rook on d8 can't capture on d3 because the d7 pawn blocks the path. |
| hTdN0 | Claude Sonnet 5 | 1143 | Missed | 50% | 2 · 2 · 1 | Qxg2+ captures the rook and gives check along the second rank, since g2 and d2 share rank 2. |
| hTdN0 | GPT-5.2 | 1143 | Solved | 50% | 2 · 2 · 2 | Black has a strong forcing check. |
| hTdN0 | Qwen3.8 A95B | 1143 | Missed | 14% | 1 · 6 · 3 | Looking at the position, Black has a powerful attacking setup with Qh3, Re3, and Bg3 all targeting the white king on d2. |
| hPTg4 | Claude Sonnet 5 | 1144 | Missed | 100% | 2 · 0 · 0 | The knight on a6 is attacked by the rook on a4. |
| hPTg4 | GPT-5.2 | 1144 | Solved | 60% | 3 · 2 · 0 | White can exploit the fact that Black’s bishop on d6 is absolutely pinned to the king on d7 by the rook on d2. |
| hPTg4 | Qwen3.8 A95B | 1144 | Missed | 80% | 4 · 1 · 0 | Looking at the position, White has a rook on d2, knight on a6, and king on g2. |
| 13ZpW | Claude Sonnet 5 | 1147 | Missed | 57% | 4 · 3 · 0 | Black's knight on e5 attacks the c4 pawn, which is only defended by the White knight on e3. |
| 13ZpW | GPT-5.2 | 1147 | Solved | 100% | 4 · 0 · 0 | Black has an immediate forced mate. |
| 13ZpW | Qwen3.8 A95B | 1147 | Solved | 100% | 4 · 0 · 4 | Black has a powerful attacking setup with Queen h4, Rook f2, Knight e5, and Bishop g7 all aimed at White's king on g1. |
| JbyDX | Claude Sonnet 5 | 1149 | Missed | 40% | 2 · 3 · 3 | Black's rook on f5 currently blocks the g6-e4 diagonal, so playing Rf5-d5 accomplishes two things at once: it attacks the undefended pawn on d4 (safely, since d5 is protected by… |
| JbyDX | GPT-5.2 | 1149 | Solved | 80% | 4 · 1 · 0 | Black has a tactical shot against White’s exposed king and queen: 32...Rxg5+ is a forcing check that also removes the pawn on g5 and, crucially, clears the g6–e4 diagonal (g6–f5… |
| JbyDX | Qwen3.8 A95B | 1149 | Missed | 38% | 3 · 5 · 0 | Looking at the position again with the legal moves, the White king on g3 is exposed. |
| plbGC | Claude Sonnet 5 | 1154 | Solved | 100% | 3 · 0 · 3 | Black's bishop on e8 blocks the a8-rook from ever reaching g8, which is the key to this combination. |
| plbGC | GPT-5.2 | 1154 | Solved | 83% | 5 · 1 · 0 | White has a direct forcing attack on the boxed-in black king. |
| plbGC | Qwen3.8 A95B | 1154 | Missed | 0% | 0 · 7 · 2 | Looking at the position, White has Queen on g4, Rooks on h2 and g1, with the Black king on h8. |
| fKSnX | Claude Sonnet 5 | 1161 | Solved | 80% | 4 · 1 · 4 | Black's rooks are doubled on the g-file against White's king on f1. |
| fKSnX | GPT-5.2 | 1161 | Solved | 25% | 2 · 6 · 1 | Black has a forcing check with 27...Rg1+ (from g3 to g1). |
| fKSnX | Qwen3.8 A95B | 1161 | Solved | 29% | 2 · 5 · 2 | Looking at the position, Black has two rooks on the g-file (g8 and g3) aimed at White's king on f1. |
| 3gNwP | Claude Sonnet 5 | 1164 | Missed | 75% | 6 · 2 · 2 | The queen on f3 is attacked by the bishop on g4, and simply capturing (Qxg4 Nxg4) loses the queen for a minor piece since the knight on f6 defends g4. |
| 3gNwP | GPT-5.2 | 1164 | Solved | 86% | 6 · 1 · 0 | White can exploit the fact that Black’s bishop on g4 is attacking the white queen: 8.Nxf6+ is a forcing check that prevents Black from simply taking on f3, and it wins material. |
| 3gNwP | Qwen3.8 A95B | 1164 | Solved | 100% | 7 · 0 · 0 | Looking at the position, White has a powerful knight on d5. |
| BpBjT | Claude Sonnet 5 | 1166 | Missed | 50% | 2 · 2 · 1 | Qxb7 wins a pawn while forking the rook on a8 and the knight on c6; neither defended piece can be saved simultaneously, e.g. |
| BpBjT | GPT-5.2 | 1166 | Solved | 100% | 3 · 0 · 0 | White has an immediate tactical win on f7: the queen from b3 can go to f7 with check (the diagonal b3–c4–d5–e6–f7 is clear). |
| BpBjT | Qwen3.8 A95B | 1166 | Missed | 20% | 1 · 4 · 1 | Looking at the position, White has a knight on g5 that attacks f7. |
| A0jOX | Claude Sonnet 5 | 1166 | Missed | 80% | 4 · 1 · 0 | The black queen on e5 eyes e1 through an open file, but the rook is defended by the bishop on f2. |
| A0jOX | GPT-5.2 | 1166 | Solved | 100% | 3 · 0 · 1 | Black has a decisive forcing line: 23...Qxe1+ gives check along the 1st rank. |
| A0jOX | Qwen3.8 A95B | 1166 | Missed | 0% | 0 · 6 · 2 | Looking at the position, Black's queen on e5 is very active. |
| lp8Vq | Claude Sonnet 5 | 1169 | Missed | 67% | 2 · 1 · 3 | Black's queen on c4 attacks the c3 pawn, which is currently undefended, and after ...Qxc3 the queen would also fork the rooks on a1 and e3. |
| lp8Vq | GPT-5.2 | 1169 | Solved | 100% | 6 · 0 · 0 | White has a direct mating attack on the exposed black king. |
| lp8Vq | Qwen3.8 A95B | 1169 | Solved | 100% | 5 · 0 · 3 | Looking at the position, White has a powerful attack setup with Qh3, Be5, and Re3 targeting the black king on h7. |
| hCYxi | Claude Sonnet 5 | 1170 | Missed | 17% | 1 · 5 · 4 | The key tactical point is that White's queen on a2 lies on the a2–g8 diagonal, which is currently blocked only by the pawn on f7. |
| hCYxi | GPT-5.2 | 1170 | Solved | 43% | 3 · 4 · 0 | White has a forced mate starting with an immediate check: 33.Qxf7+ attacks the black king on g8. |
| hCYxi | Qwen3.8 A95B | 1170 | Solved | 50% | 3 · 3 · 0 | Looking at the position, White has Qa2, Rf4, Re3, Kh3 vs Black's Kg8, Re8, Qc6, Rc3, with pawns. |
| s7Hec | Claude Sonnet 5 | 1173 | Solved | 100% | 2 · 0 · 0 | Black's king can capture the undefended white a5-pawn, since it's not adjacent to the white king and eliminates White's outside passed pawn, which is the key long-term threat. |
| s7Hec | GPT-5.2 | 1173 | Solved | 75% | 3 · 1 · 2 | Black’s king on b4 can immediately capture the undefended white pawn on a5. |
| s7Hec | Qwen3.8 A95B | 1173 | Missed | 0% | 0 · 4 · 1 | Black is in zugzwang - every king move allows White's king to penetrate and win. |
| 4OFsi | Claude Sonnet 5 | 1178 | Missed | 0% | 0 · 4 · 0 | Qe3+ hits the king along the e3-g5 diagonal, forcing the king to move since nothing can block or capture. |
| 4OFsi | GPT-5.2 | 1178 | Solved | 100% | 6 · 0 · 0 | Black can force a quick mate because the white king on g5 has almost no safe squares and cannot capture a well-placed checking queen. |
| 4OFsi | Qwen3.8 A95B | 1178 | Missed | 0% | 0 · 3 · 0 | Let me analyze this position. |
| zb6P7 | Claude Sonnet 5 | 1181 | Solved | 0% | 0 · 3 · 0 | Rc7-c6+ delivers check along the 6th rank while also attacking the rook on d6, winning material after the king moves. |
| zb6P7 | GPT-5.2 | 1181 | Solved | 57% | 4 · 3 · 2 | White’s passed pawn on d7 is ready to promote, but 39.d8=Q? fails immediately to 39...Rxd8 because Black’s rook on d6 controls the d-file. |
| zb6P7 | Qwen3.8 A95B | 1181 | Solved | 30% | 3 · 7 · 0 | Let me analyze the position. |
| zEHev | Claude Sonnet 5 | 1181 | Missed | 100% | 2 · 0 · 0 | The black rook on a2 can capture the undefended a3 pawn, gaining material safely since no white piece defends a3. |
| zEHev | GPT-5.2 | 1181 | Solved | 80% | 4 · 1 · 3 | Black can deliver an immediate back-rank mate: the rook from a2 goes to a1 giving check along the 1st rank (a1–d1). |
| zEHev | Qwen3.8 A95B | 1181 | Solved | 67% | 4 · 2 · 3 | The position has Black's king on d3, rook on a2, and pawn on f4 against White's king on d1, rook on e1, bishop on d4, and pawns on a3, b5, c3. |
| 5O7J6 | Claude Sonnet 5 | 1182 | Solved | 67% | 4 · 2 · 0 | The knight on d7 is only defended by the queen on c7. |
| 5O7J6 | GPT-5.2 | 1182 | Solved | 50% | 2 · 2 · 0 | White can win decisive material with a forcing check. |
| 5O7J6 | Qwen3.8 A95B | 1182 | Missed | 10% | 1 · 9 · 2 | Looking at the position, White's queen on b5 can capture the knight on d7 with check. |
| InxAC | Claude Sonnet 5 | 1189 | Missed | — | 0 · 0 · 0 | |
| InxAC | GPT-5.2 | 1189 | Solved | 80% | 4 · 1 · 4 | White has no direct checks on the g-file because the g7-pawn blocks the rook, and the bishop is boxed in. |
| InxAC | Qwen3.8 A95B | 1189 | Missed | 11% | 1 · 8 · 1 | The position features White's rook on g2 and queen on e2 against Black's king on g8. |
| 5MPMl | Claude Sonnet 5 | 1191 | Solved | 60% | 3 · 2 · 3 | Black's knight on f5 attacks the queen on e3, so White must act with tempo. |
| 5MPMl | GPT-5.2 | 1191 | Solved | 67% | 4 · 2 · 2 | White has a tactical fork: 20.Nxf6+ captures the knight on f6 with check (the knight attacks g8), forcing Black to respond to the check. |
| 5MPMl | Qwen3.8 A95B | 1191 | Solved | 100% | 4 · 0 · 2 | I need to reconsider. |
| pPOJg | Claude Sonnet 5 | 1200 | Missed | 0% | 0 · 1 · 3 | The b6 pawn can advance to b7, forking the rook on c8 (attacking it) and threatening to promote on b8. |