Tagging chess puzzles without calling the engine

We built a puzzle generator for a chess-training product. It creates puzzles from a player's own games, so every user gets a private stream of positions drawn from their real mistakes and missed shots. The trouble: the puzzles arrived untagged. A user could not ask for "just my fork puzzles," and we could not tell them what a puzzle was even teaching.

The obvious way to tag a puzzle is to ask a chess engine. We didn't. We tag every puzzle straight from the data we already store — its position and its solution line — with no engine call at all. Because tagging costs nothing, we can also re-tag the entire back catalog whenever the rules improve.

The "puzzles from your games" feature was working. It found sharp moments in a user's games and turned them into puzzles. But each puzzle landed in the database with a position and a solution and nothing else describing it.

That left two gaps. Users could not filter. Someone grinding pins had no way to pull their pin puzzles out of the pile. And the product could not describe its own output — no per-theme counts, no sense of which tactics a user saw most or least.

The information was technically recoverable. A chess engine can look at a position and tell you a great deal about it. But engine calls are the single most expensive thing in this pipeline. The scan that generates puzzles already leans on the engine hard, and every extra call adds latency and load. Paying that cost again just to attach a label was the wrong trade.

The reflex is to reach for the engine. Feed it the position, ask what tactical motifs are present, store the answer. It would work, and it would be accurate.

It would also be slow and expensive at exactly the wrong moment. Classifying inside the scan means an engine call per puzzle, on top of the engine calls the scan already makes to find puzzles in the first place. That inflates the cost of the one operation we most want to keep cheap.

It gets worse when you look backward. We already had a large history of generated puzzles, all untagged. Engine-based tagging means re-running the engine across the entire back catalog — effectively re-scanning history to attach labels we could have derived another way. The cost scales with the archive, not with new work.

We already store everything a human needs to name the tactic: the starting position and the full solution line. A coach doesn't run an engine to say "that's a fork." They look at the move and the resulting position and recognize the pattern. So we encoded that recognition as rules.

The ThemeClassifier takes a stored puzzle's position and solution and tags it with tactical themes:

• fork
• pin
• skewer
• hanging piece
• back rank
• discovered attack
• overloaded defender
• mate
• promotion
• wins material

Each theme is a rule over the position and the moves — piece geometry, what gets captured, where the king sits, whether a pawn reaches the last rank. No engine sits in this loop.

That single decision changes the economics. Because classification reads only stored data, it is free inside the scan — the generator attaches themes as it writes the puzzle, with no added engine cost. And because it never touches the engine, it can re-tag historical puzzles without re-scanning the games they came from. Improve a rule, re-run the classifier over the archive, and every old puzzle picks up better tags. Tagging stopped being a cost center and became something we could run as often as we liked.

The flow is a straight line: a stored puzzle feeds the classifier, the classifier emits tags, and the tags land in two new columns that the API and a startup job read from.

stored puzzle
  ├─ position
  └─ full solution line
        │
        ▼
  ThemeClassifier  (rule per theme — no engine)
        │
        ▼
  theme tags  ──►  Themes column
                   ThemesVersion column
        │
        ├──►  theme-summary endpoint (per-theme totals, worst-first)
        └──►  theme filter on the puzzle list

Stored position and solution in, tags out. The classifier's only inputs are what the puzzle already carries. Each theme is its own rule evaluated against the position and the move sequence. A puzzle can match several themes at once — a move can fork and win material in the same breath — so the output is a set, not a single label.

Themes and ThemesVersion columns. We added two columns. Themes holds the tags. ThemesVersion records which version of the classifier produced them. The version stamp is what makes safe re-tagging possible: we can tell at a glance which puzzles were tagged by old rules and need another pass.

Worst-first summary endpoint and a filter. A theme-summary endpoint returns per-theme totals, ordered worst-first, so the weakest-covered themes surface at the top. A theme filter on the puzzle list lets a user pull exactly the puzzles for one tactic. Together they turn raw tags into something a user and the product can act on.

Version-aware startup backfill. A backfill runs at startup and compares each puzzle's ThemesVersion against the current classifier version. Anything stale — including everything tagged before the feature existed — gets reclassified. Because the classifier needs no engine, this backfill is cheap enough to run as a normal part of coming online rather than a special migration.

Two things bit us.

The first was accuracy, and it taught us where a tactic lives. Our first classifier read only the first move of each solution. That felt reasonable — the first move is the point of the puzzle. But it left far too many puzzles unnamed, because much of a tactic's identity plays out past move one. A discovered attack, a skewer that wins the piece behind, a mate that lands three moves deep — you cannot see any of these from the opening move alone. When we changed the classifier to read the whole solution line, the unnamed rate dropped sharply. The full sequence is where the tactic actually resolves.

The second was operational. On the first deploy, the startup backfill kicked off and quietly starved a live scan — two heavy jobs competing for the same resources, with the backfill winning and the user-facing scan suffering. The fix was to throttle the backfill so it yields to live work. The lesson: "cheap" per item is not the same as "safe to run flat out." A backfill over the whole archive is still a big job, and it has to be a polite one.

We measured against 362 real puzzles and iterated twice.

The headline metric is coverage: how many puzzles the classifier could name at all. Unnamed (unclassified) puzzles fell from 49% to 27%. Nearly half the archive had been untaggable under the first-move-only rule; after reading the whole solution line, roughly three in four puzzles carry at least one theme.

To be honest about what this number is and isn't: it measures coverage — the share of puzzles that got any theme — on the 362-puzzle sample, not per-tag precision against an expert-labeled ground truth. The 27% still-unnamed slice is real. Some of those puzzles resist simple rules, and some tactics we don't yet classify. The gain came almost entirely from one change: reading the full solution instead of its first move.

The cost side is the quieter win. Every one of those tags came with zero engine calls, inside the scan and across the backfill.

1. Classify from what you already store. If the label is derivable from data on hand, deriving it beats re-deriving it from an expensive source. The position and solution were enough to name most tactics — no engine needed.
2. Read the whole signal, not the first token. First-move-only classification looked reasonable and was quietly wrong. A tactic's identity lives across its full line. Cutting the unnamed rate from 49% to 27% came from reading more of what we already had.
3. Version your labels. A ThemesVersion stamp turns re-tagging from a risky migration into a routine, idempotent pass. You always know what's stale.
4. Cheap per item is not free in aggregate. A no-engine backfill still starved a live scan until we throttled it. Batch jobs over a full archive must yield to user-facing work.
5. Free re-tagging changes what you'll build. Once tagging cost nothing and could safely re-run over history, improving the rules stopped being a budget decision. That's the real payoff of keeping the engine out of the loop.