feat: VLM classifier for blank scene-break candidates (multimodal verification)

[Arkoniak]

Motivation

Gap-based scene-break detection (detect_scene_breaks.py) reliably finds visual ornaments (small
centred Picture blocks between paragraphs) using geometry alone. But a second class of scene-break —
a plain blank gap between paragraphs, with no ornament at all — is fundamentally ambiguous:

Gap looks like…	Could be…
Two paragraphs with extra whitespace	Real scene break (blank-line convention)
Text block → indented block → text	Quoted letter / telegram / epigraph set apart by layout
Two adjacent Surya/Unlimited blocks	Segmentation artefact (models split one paragraph into two)

Concrete example from mindblast (issue #22 work):

• p139 y=[0.812, 0.853] gap=0.041 — only Unlimited sees it; Surya has a continuous block there.
  Visually: no scene break at all. Pure segmentation artefact.
• p140 y=[0.648, 0.672] and y=[0.820, 0.842] — both models agree, high confidence.
  Visually: the hero is reading a letter; the blank space is typographic indentation to
  set the letter apart, not a scene-break. There is no way to know this from gap geometry alone.

A human editor would glance at the page image and read the surrounding sentences — the text
"He read the letter carefully:" before the gap makes the intent obvious. Geometry cannot see that.

Idea

Add an optional VLM verification pass on top of detect_scene_breaks.py for blank-gap candidates.

Inputs (per candidate gap)

1. Image crop — a tight crop of the page scan covering the gap zone plus ~2–3 lines of text above
   and below (so the model sees the visual whitespace and any decoration in it).
2. Text context — the voted_tokens from ~100–200 words before and after the gap (already
   available in the consensus pages JSON).

Both are passed together in a single multimodal prompt, letting the model use them jointly — the way
a human editor would.

Model

Qwen3-VL-8B (already in the pipeline, runs via llama-server + GGUF). It handles image + text
in one inference call and is accurate enough for a binary classification task this simple.

Prompt sketch

You are reviewing a scanned book page. A detected gap (white space) appears between two text blocks.

[IMAGE CROP of the gap region]

Text BEFORE the gap:
<voted_tokens before>

Text AFTER the gap:
<voted_tokens after>

Question: Is this gap a deliberate **scene break** (a section divider the author placed between
two distinct scenes or episodes), or is it something else — such as indentation for a quoted
letter, a telegram, an epigraph, or normal paragraph spacing?

Answer with one word: scene_break or not_scene_break.

Role in pipeline

detect_scene_breaks.py
  ├── ornament candidates   → geometry already decides (no VLM needed)
  ├── typographic candidates → regex already decides (* * * etc.)
  └── blank candidates      → feed to VLM verifier → scene_break / not_scene_break

The VLM answer is a verifying signal, not source of truth. Suggested confidence mapping:

• geometry says blank + VLM says scene_break → high confidence blank break
• geometry says blank + VLM says not_scene_break → downgrade to low / drop
• VLM disagreement or error → keep original confidence, flag for manual review

Why this is feasible

• The task is binary and simple compared to full OCR.
• The model does not need to transcribe anything — it needs to classify a region.
• Crop + 200 tokens is a tiny context; inference should be fast (seconds per candidate, not per page).
• Blank-gap candidates are rare (a handful per book), so the extra VLM cost is negligible.

Open questions / not-yet-decided

• Which page coordinate system to use for cropping (absolute pixel from the source PDF/image)?
• Minimum/maximum crop size to give enough context without noise.
• Whether to run Qwen3-VL or try GLM-OCR 0.9B first (faster, but text-focused — probably worse here).
• Whether to integrate into detect_scene_breaks.py or as a separate verify_scene_breaks.py.
• Prompt engineering: single-word answer vs. chain-of-thought + answer (CoT may help accuracy).

Prerequisites

• Scene-break detection by recurrence -> <hr> #22 (scene-break detection) — must be merged first; this extends it.
• Access to original page scan images (currently not exposed to the pipeline beyond PDF input).

[Arkoniak]

Live example validating the multimodal approach — mindblast p140

While working on issue #22, I tested this exact scenario manually (Claude as the VLM, looking at the
actual page scan + surrounding text). The result perfectly illustrates why both modalities are needed.

Detected gaps on page 140

p140  y=[0.648, 0.672]  gap=0.024  blank  high  S+U
p140  y=[0.820, 0.842]  gap=0.022  blank  high  S+U

Both gaps are high-confidence (Surya + Unlimited agree). Geometry alone cannot distinguish these
from real scene breaks.

What the page actually shows

The two gaps are:

• y≈0.648 — whitespace between "Joss was looking at the note from Lucretia. It said:"
  and the body of the letter (ALL CAPS, visually indented from the margins)
• y≈0.820 — whitespace between the end of the letter ("SOONEST. L.") and the return to
  narrative (""Wonderful," he said to himself…")

Verdict: not scene breaks. The scene is continuous — Joss is simply reading a memo. The
gaps are typographic indentation to set the quoted document apart from the surrounding dialogue.

Why each modality alone fails

Signal	What it sees	Conclusion
Geometry only	Two gaps with empty zones — identical to ornament-less scene breaks	❌ false positive
Image crop only	White space above/below an indented block	❓ ambiguous
Text only	"It said:" before gap, ALL CAPS content, normal prose after	✅ clearly a quote
Image + text together	ALL CAPS + indented block + "It said:" intro	✅ immediately obvious

The text alone is actually sufficient here ("It said:" is an unambiguous quotation introducer),
but the visual confirms it: the ALL CAPS register and left-margin indentation are typographic
signals that the content is a quoted document, not a new scene.

This is exactly the kind of reasoning a multimodal model should be able to reproduce with a
prompt like: "Is the white space a scene break, or typographic separation for a quoted passage?
Consider both the visual layout and the surrounding text."