Fit the model's token budget¶
Media is the expensive part of an annotation run, and the size of the image you send is the only
lever that scales with the whole dataset. Two decisions belong here: how big the preview is, and
where the cache breakpoint sits. Both are per-model, both go in config/, and neither belongs in
code.
The rule for the first: never send an original. Send an annotools preview at the size the model
bills well at, and when detail is missing, crop a region at that same size rather than raising the
resolution of the whole frame.
What an image costs¶
Verified against vendor documentation on 2026-08-27; the pages behind each number are listed in the
skill's references/sources.md.
Re-check anything you are about to base a budget on — vendors move tiers without notice.
| Model family | Rule | Cheapest useful size |
|---|---|---|
| Gemini 2.5 / 3.x | ≤ 384×384 → 258 tokens; larger images are tiled: crop unit ≈ floor(min(w, h)/1.5), tiles = ceil(w/unit) × ceil(h/unit), 258 tokens per tile (768×768 → 4 tiles, 768×432 → 6). media_resolution low/default caps tokens per image/frame. |
≤ 384 px both sides (258 tokens); otherwise accept 4–6 tiles at 768 |
| Claude 4.7+ (high-res tier); older Claude (standard) | 28×28 patches: ⌈w/28⌉ × ⌈h/28⌉ tokens — cost grows with area. Images above the tier limit (2576 px edge / 4784 tokens high-res; 1568 px / 1568 tokens standard) are downscaled, then padded to a multiple of 28 on the right/bottom. 768×768 → 784 tokens; 768×432 → 448; 1568×1568 would be 3136 and is therefore resized on the standard tier. | Smallest size that still shows the target: 768 long side ≈ 450–800 tokens; 384 ≈ 100–200. 1568/2576 px are resize thresholds, not cheap sizes |
| GPT-4o / GPT-4.1 (tile models) | detail=low → 85 base tokens only. high: fit in 2048², then shortest side to 768, count 512-px tiles × 170 + 85. 768×768 → 4 tiles = 765; ≤ 512×512 → 1 tile = 255. |
≤ 512 px both sides for one tile; detail=low only when layout does not matter |
| GPT-5.1 (tile model) | Same procedure with 70 base + 140 per tile: 768×768 → 630; ≤ 512×512 → 210. | as above |
| GPT-5.2 / 5.4 / 5.6 families, GPT-4.1-mini/nano, gpt-5-mini/nano, o4-mini (patch models) | 32-px patches: ceil(w/32) × ceil(h/32) × model multiplier (1.2 for 5.⅖.⅘.6 and gpt-5-mini; 1.62 for 4.1-mini; other multipliers per the source page). 768×768 → 576 patches × 1.2 ≈ 692. | Cost is linear in area: shrink to the smallest legible size; no tile boundary to exploit |
| Qwen2.5-VL | 14-px patches merged 2×2 → one token per 28×28 px; min_pixels/max_pixels bound the area (typical 256·28² – 1280·28²), sizes rounded to multiples of 28. 768×768 → 784 tokens. |
Set max_pixels to the budget; 768 long side ≈ 450–800 tokens |
| Qwen3-VL | 16-px patches merged 2×2 → one token per 32×32 px; sizes rounded to multiples of 32; pixel budgets in units of 32² (video fps default 2; the total_pixels default differs between the README and vision_process.py — set it explicitly). 768×768 → 576 tokens. |
Same rule with 32-px units; 768 long side ≈ 330–600 tokens |
The families split into three billing shapes, and the table above says where each boundary sits.
Gemini has a cliff at a small size: below it an image is one unit, above it several. The GPT-4o /
4.1 / 5.1 tile models have a coarser one at 512 px, where an image is a single tile. Claude, the GPT
patch models, and Qwen bill by area with no boundary at all, so for them a smaller image saves a
little and loses detail. This is why the annotools default of 384×384 — chosen for Gemini's cliff —
is the wrong default for most projects, and why the size belongs in config/ rather than in code.
| Model family | max_width × max_height |
Why |
|---|---|---|
| Gemini 2.5 / 3.x | 384 × 384 (the default) | one 258-token unit; 768 costs 4–6 units |
| Claude (standard or high-res tier) | 768 × 768 (up to 1092 × 1092 stays under 1568 tokens) | 28-px patches, area billing; well inside both tier limits so coordinates need no rescale |
| GPT-5.2 / 5.4 / 5.6 (patch models) | 768 × 768 (up to 1024 × 1024 ≈ 1229 tokens) | 32-px patches × 1.2, area billing |
| GPT-4o / 4.1 / 5.1 (tile models) | 512 × 512 for one tile, else 768 × 768 (4 tiles) | 512-px tiles; detail=high |
| Qwen2.5-VL / Qwen3-VL | 768 × 768 | 28- or 32-px units, area billing; set max_pixels/total_pixels to match |
Set it once at registration (ANNOTOOLS_MAX_WIDTH / ANNOTOOLS_MAX_HEIGHT, or
annotools --max-width W --max-height H); every tool also takes max_width/max_height per call.
For a pipeline, the size lives in config/ and is passed to the tool constructor —
see step 5.
Layout for the cache¶
The static part of the prompt is the class definitions, the output schema, the tool definitions, and the few-shot examples; the variable part is one item. Everything static goes first, then the cache breakpoint, then item metadata, the image, and the question. One item id or file name placed above the breakpoint sets cache reads to zero for the entire run — the prefix has to be byte-identical.
Each provider also has a minimum cacheable prefix — a few hundred to a few thousand tokens, depending on the model — and below it nothing caches at all. The skill's "Prompt caching minimums" table lists the current figure per model, dated.
What this looks like when it works: the three-image trial of
object-detection-claude
billed 24 uncached input tokens against 60,379 cache reads and 15,616 cache-creation tokens, with
1,222 output tokens, for 0.220 USD — 0.073 per image at a 768 px preview. Almost everything the run
reads is the static prefix, re-read on every turn at the cache rate; the per-item text is the 24.
Do not divide one by the other: the Claude SDK's token fields come from each item's final result
message, while cost_usd covers every turn of the item.
Measure before the full run¶
Estimates are for choosing; usage is for believing. Run three items, read the cached and input
token counts, compare with the estimate, and only then submit the rest. The target is measured
tokens per item within 20 % of the estimate and non-zero cache reads from the second request on.
That is the same three items as the trial, so it costs nothing extra.
For video, sparser beats smaller: 32 frames at 768 px is already 15–25 k tokens, so lower the frame rate before lowering the resolution. The skill's video and audio table has the per-provider rates.
Next: the localization loop, where the preview size meets coordinates.
Source: skills/mllm-multimodal-input