CardVision second look after Gemini: use Gemini's name+number to pick the right printing (alt art / promo / reprint) #911

New Issue

2026-10-01T04:48:29Z

gambit-admin commented

2026-10-01 04:48:29 +00:00

Problem

When CardVision can't match a raw card, the scan falls back to Gemini. Gemini reads the card well (name, number, set), but what it reads often maps to the wrong printing. Alt arts, stamped promos and reprints can share a name and number with the base card, so a correct read still lands on the base card (or a picker).

Proposal: Gemini reads, then CardVision takes a second, focused look

CardVision gets no match (as today).
Gemini transcribes name / number / set (as today; the prompt stays transcription-only).
New: CardVision re-runs on the same photo, restricted to the catalog rows that match Gemini's name + number. That's typically 2–8 prints (base, alt art, stamped promo, reprint in other sets) instead of ~92k. Because the search space is tiny, it can accept a weaker visual match than its global gate allows without much risk.
If one candidate clearly wins, that's the answer. Otherwise fall back to today's behavior (Gemini's read + the printing picker).

CardVision already does a small version of this inside an accepted match: the number-read re-election in cardvision.py (number_pick / number_swapped, plus verifying unscored group members whose number matches). This extends the idea to the miss case, with the hint coming from Gemini instead of its own OCR.

Scope

Fixes: alt arts, stamped promos, and same-number reprints across sets. Their reference images differ, so a focused visual comparison can separate them.

Does NOT fix: holo vs reverse holo vs cosmos holo. Same artwork, only the foil differs; reference images are stock product photos without foil, and these are usually printings inside one product, not separate rows. These still need the printing picker (or a future foil classifier, out of scope).

Prerequisite: alt-art reference images aren't in the index yet

Checked 2026-09-30: the live index on the Tower (data/ids.json, built 2026-09-17, 92,099 rows) has 0 variant rows. The ~5.5k alt-art/parallel images were never added, so a second look can't pick an alt art it has never seen. Unblocking needs, in order:

gambit PR #517 (phone vectors path) → PR #519 (variant artwork rows; /scan/cardvision-resolve honours variant_ids) merged + deployed
Phone stops doing Number(candidate.id) (lib/scan/cardvisionScan.ts:249, still on main), which turns the new product:variant ids into NaN. Fixed in #519; ships via OTA/build.
cardvision PR #6 (manifest variant meta), merged 2026-09-23
Delta index build from the PROD variant export (5,462 rows already exported). Run with CARDVISION_IMAGE=cardvision:gpu, or roll_fleet.sh reverts the foil fix.

Stamped promos and cross-set reprints that are already their own products should benefit even before this lands.

Build plan

CardVision service: an optional restriction on /identify (e.g. hint_name, hint_number, or an explicit candidate_ids list). It ORB-verifies only those rows and uses a relaxed acceptance bar scaled to the candidate count, plus a clear-margin rule between the top two. Response flags hinted: true.
Backend (preferred over phone orchestration, so the phone doesn't re-upload the photo over cellular): after the cloud scan's Gemini read, look up catalog rows by name + number (all prints) and call CardVision with that restriction. A confident result overrides the printing; otherwise behavior is unchanged. Optional, fail-open, and time-boxed (~1s budget), so a CardVision outage never slows or breaks the Gemini path.
Telemetry: log fallback → Gemini read → second-look outcome (hit / no-clear-winner / skipped / error) and the chosen id vs Gemini's original pick. This tells us if it's paying off.
Also log CardVision's first-pass top-3 candidates on fallback (currently discarded) to measure how often the right card was already in its shortlist.

Cost / risk

Latency: one extra CardVision call (~0.3–0.6s) only on scans that already fell back to Gemini (multi-second today).
Wrong-print risk: mitigated by requiring a clear margin; ties fall through to the picker as today.
No DB migration. Additive optional fields → old app builds unaffected.

Filed from a Claude Code session

## Problem When CardVision can't match a raw card, the scan falls back to Gemini. Gemini reads the card well (name, number, set), but what it reads often maps to the **wrong printing**. Alt arts, stamped promos and reprints can share a name and number with the base card, so a correct read still lands on the base card (or a picker). ## Proposal: Gemini reads, then CardVision takes a second, focused look 1. CardVision gets no match (as today). 2. Gemini transcribes name / number / set (as today; the prompt stays transcription-only). 3. **New:** CardVision re-runs on the same photo, restricted to the catalog rows that match Gemini's name + number. That's typically 2–8 prints (base, alt art, stamped promo, reprint in other sets) instead of ~92k. Because the search space is tiny, it can accept a weaker visual match than its global gate allows without much risk. 4. If one candidate clearly wins, that's the answer. Otherwise fall back to today's behavior (Gemini's read + the printing picker). CardVision already does a small version of this inside an *accepted* match: the number-read re-election in `cardvision.py` (number_pick / number_swapped, plus verifying unscored group members whose number matches). This extends the idea to the miss case, with the hint coming from Gemini instead of its own OCR. ## Scope **Fixes:** alt arts, stamped promos, and same-number reprints across sets. Their reference images differ, so a focused visual comparison can separate them. **Does NOT fix:** holo vs reverse holo vs cosmos holo. Same artwork, only the foil differs; reference images are stock product photos without foil, and these are usually printings inside one product, not separate rows. These still need the printing picker (or a future foil classifier, out of scope). ## Prerequisite: alt-art reference images aren't in the index yet Checked 2026-09-30: the live index on the Tower (`data/ids.json`, built 2026-09-17, 92,099 rows) has **0 variant rows**. The ~5.5k alt-art/parallel images were never added, so a second look can't pick an alt art it has never seen. Unblocking needs, in order: - [ ] gambit PR #517 (phone vectors path) → PR #519 (variant artwork rows; `/scan/cardvision-resolve` honours `variant_ids`) merged + deployed - [ ] Phone stops doing `Number(candidate.id)` (`lib/scan/cardvisionScan.ts:249`, still on main), which turns the new `product:variant` ids into NaN. Fixed in #519; ships via OTA/build. - [x] cardvision PR #6 (manifest variant meta), merged 2026-09-23 - [ ] Delta index build from the PROD variant export (5,462 rows already exported). Run with `CARDVISION_IMAGE=cardvision:gpu`, or `roll_fleet.sh` reverts the foil fix. Stamped promos and cross-set reprints that are already their own products should benefit even before this lands. ## Build plan 1. **CardVision service:** an optional restriction on `/identify` (e.g. `hint_name`, `hint_number`, or an explicit `candidate_ids` list). It ORB-verifies only those rows and uses a relaxed acceptance bar scaled to the candidate count, plus a clear-margin rule between the top two. Response flags `hinted: true`. 2. **Backend (preferred over phone orchestration, so the phone doesn't re-upload the photo over cellular):** after the cloud scan's Gemini read, look up catalog rows by name + number (all prints) and call CardVision with that restriction. A confident result overrides the printing; otherwise behavior is unchanged. Optional, fail-open, and time-boxed (~1s budget), so a CardVision outage never slows or breaks the Gemini path. 3. **Telemetry:** log fallback → Gemini read → second-look outcome (hit / no-clear-winner / skipped / error) and the chosen id vs Gemini's original pick. This tells us if it's paying off. 4. Also log CardVision's first-pass top-3 candidates on fallback (currently discarded) to measure how often the right card was already in its shortlist. ## Cost / risk - Latency: one extra CardVision call (~0.3–0.6s) only on scans that already fell back to Gemini (multi-second today). - Wrong-print risk: mitigated by requiring a clear margin; ties fall through to the picker as today. - No DB migration. Additive optional fields → old app builds unaffected. --- Filed from a Claude Code session

gambit-admin added the feature enhancement labels 2026-10-01 04:48:29 +00:00

gambit-admin commented

2026-10-01 05:45:52 +00:00

Note: the feedback-request bot announced PR #685 as this issue — false match (the PR description mentioned #911). #685 is a PREREQUISITE only: it lets CardVision index alt arts / art-distinct prints as their own rows (backend live 2026-10-01, OTA going out now; CardVision index rebuild still to do). The second-look-after-Gemini feature in this issue is NOT built yet; nothing to test under #911.

gambit-admin closed this issue

2026-10-01 05:46:02 +00:00

gambit-admin commented

2026-10-01 05:46:02 +00:00

Closed — fixed in PR #685; @kris says no test is needed.

via gambit-test

Closed — fixed in PR #685; @kris says no test is needed. _via gambit-test_

gambit-admin commented

2026-10-01 07:46:16 +00:00

Prerequisite done 2026-10-01: CardVision index now has 2,486 art-distinct variant rows (alt arts, manga/special/full arts, Jolly Roger foil, stamps) from the PROD export; 129 rows that out-scored their own base image (same-art starterdeck prints, near-identical stamps) were dropped. Live on all Tower workers + Mac fallback; prod resolve verified (OP09-014 Limejuice alt art → Altart print). The second-look feature in this issue is still not built.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: gambit/gambit#911