How to Run Vision-LLM Classification at Scale Without Burning Your Budget

Every time you call a vision LLM API to classify an image, you're paying a lot more per call than you would for plain text — image tokens are heavier, and reported multipliers land in the low-single-digits-to-10× range depending on the model and image size. (That range is a reported industry pattern, not a JuiceFactory guarantee — measure it against your own images and prompts.) It adds up fast. Run 1,000 classification calls at €0.005 each, you're at €5. Iterate that 10 times while tweaking prompts? Now you're €50 deep on a feature that doesn't even ship yet.

We solved this at matkalkylen (a Swedish nutrition calculator that classifies food photos) by inverting the workflow: pay once for vision, iterate for free on everything else. This guide walks the pattern—and shows you the actual code that made it work.

Start building: Get a free JuiceFactory API key — no credit card required.

The Problem: Vision Tokens Are Expensive, Iteration Kills Your Budget

Vision LLM calls cost more because images get tokenized, and that tokenization is computationally heavier than text. You're not just paying for inference; you're paying for pixel processing.

Here's the naive approach:

1. User uploads food photo → call Qwen3-VL to classify
2. Classify returns match → call Qwen3-VL again to extract portions
3. Tweak the classification prompt → re-run all 1000 test images
4. Tweak the portion prompt → re-run all 1000 test images again
5. Realize you need a confidence threshold fix → re-run everything

Cost: 2 calls/image × 1000 images × 5 iterations = 10,000 API calls

Each call is €0.005 minimum (on Qwen3-VL at JuiceFactory). (That per-image figure is from a past production run — confirm current Qwen3-VL pricing on JuiceFactory before you budget against it.) Total: €50 wasted on refinement.

The smarter move? Pre-compute once, cache aggressively, iterate on everything downstream for free.

The Pattern: Pre-cache Vision Once, Iterate Templates Cheap

Here's the architecture:

┌─────────────────────────────────────────┐
│ Phase 1: Pre-compute (one-time, €5)     │
│ - Load 1000 benchmark images             │
│ - Call Qwen3-VL for each:                │
│   · Classify against corpus              │
│   · Extract component portions           │
│ - Save JSON cache files                  │
└─────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────┐
│ Phase 2: Iterate (free)                 │
│ - Load cached VL predictions             │
│ - Tweak component defaults/limits        │
│ - Tweak bias logic, thresholds           │
│ - Test new system prompts (cached data)  │
│ - Zero API calls                         │
└─────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────┐
│ Phase 3: Deploy (amortized)             │
│ - User uploads new photo                 │
│ - Hit cache FIRST (if similar)           │
│ - Fall back to live API (one call)       │
│ - €0.005 cost spread across usage        │
└─────────────────────────────────────────┘

The key: Phase 2 costs nothing. You iterate as much as you want on templates, logic, confidence rules—all offline against cached predictions.

Real Code: How matkalkylen Does It

Our production build script (autoresearch/build_cache.py) does exactly this:

Step 1: Classify Each Image

CLASSIFY_SYSTEM = """Du är en svensk näringsexpert som identifierar maträtter på bilder.
Du får en lista med dish_id och deras namn. Välj det dish_id som BÄST matchar bilden.
...
Returnera ENBART JSON:
{
  "dish_id": "exact_id_from_list" eller "unknown",
  "confidence": "high" | "medium" | "low",
  "description": "...",
  "reference_object_used": boolean
}"""

def call_vl(image_data_url, system_prompt, user_text, max_tokens=800):
    r = requests.post(
        "https://api.juicefactory.ai/v1/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"},
        json={
            "model": "qwen3-vl",
            "messages": [
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": [
                    {"type": "text", "text": user_text},
                    {"type": "image_url", "image_url": {"url": image_data_url}},
                ]},
            ],
            "max_tokens": 400,
            "temperature": 0.2,
            "response_format": {"type": "json_object"},
        },
        timeout=90,
    )
    return parse_json_strict(r.json()["choices"][0]["message"]["content"])

Simple: two payloads, two vision calls per image, temperature tuned low (0.2) for repeatability. The endpoint is OpenAI-compatible (https://api.juicefactory.ai/v1), so any OpenAI SDK or requests call drops straight in.

Step 2: Extract Component Portions (If Matched)

def build_question_prompt(template):
    components = "\n".join(
        f"  - {c['namn']} (mäts i {c['enhet']}, default {c['default']}, intervall {c['min']}-{c['max']})"
        for c in template.get("components", [])
    )
    return f"""Du tittar på en bild av {template['name']}.
Bedöm bara mängderna av varje synlig komponent. Returnera ENBART JSON:
{{
  "answers": {{
    "ingrediens_namn_1": <antal eller "liten" / "normal" / "stor">,
    "ingrediens_namn_2": <antal>
  }},
  "portion_g": <total uppskattad vikt>,
  "uncertainty": "kort om vad som är osäkert"
}}
...

Second vision call extracts the actual measurements. Only fires if classification matched.

Step 3: Save to Cache

def process_one(entry, corpus, corpus_prompt, templates_by_id):
    dish_id = entry["dish_id"]
    cache_path = CACHE_DIR / f"{dish_id}.json"
    
    if cache_path.exists():
        return "cached"  # Skip if already done
    
    image_url = image_to_data_url(image_path)
    
    cls, err = call_vl(image_url, CLASSIFY_SYSTEM, classify_user, 400)
    out = {"dish_id_truth": dish_id, "classify": cls, "questions": None}
    
    # Only ask questions if we got a match
    if cls.get("dish_id") != "unknown":
        template = templates_by_id[cls["dish_id"]]
        q_result, err = call_vl(image_url, q_prompt, "Räkna och svara.", 800)
        out["questions"] = q_result
    
    cache_path.write_text(json.dumps(out, ensure_ascii=False))
    return "ok"

# Parallel execution (4 workers default)
with ThreadPoolExecutor(max_workers=4) as pool:
    for fut in as_completed(futures):
        result = fut.result()

All results go to data/cache/{dish_id}.json. Once cached, iteration is free. Because the API is stateless, nothing on the server depends on request order—each call is independent, which is exactly what makes parallel pre-caching safe.

The Cost Math

Pre-cache phase:

• 1,000 benchmark images
• 2 VL calls per image (classify + portions)
• €0.005 per call (Qwen3-VL @ JuiceFactory)
• Total: 2,000 calls × €0.005 = €10 cost

But the script says ~€5 total. Why? Because not all images match a known dish (some classify as "unknown"), skipping the second call. Real cost: ~€0.005 per image, amortized.

Iteration phase:

• 0 API calls (all data cached locally)
• Tweak component defaults? Free.
• Adjust confidence thresholds? Free.
• Test new system prompts on cached data? €0

Break-even: Spread that €5 one-time cost across your live queries: at 1,000 requests it's €0.005 each, at 100,000 it's €0.00005, at 1,000,000 it's €0.000005 — effectively free. The more you serve, the closer the upfront cost rounds to zero.

Why This Works: Amortization + Determinism

Three principles converge:

1. Amortization Over Volume

One expensive compute task, spread over many users/queries. A €5 benchmark cost disappears into rounding error at scale. As long as you're running 1000+ classifications per month, you've won.

2. Determinism on Stable Prompts

System prompts are static. The corpus of dishes doesn't change mid-iteration. Vision predictions on the same image + same prompt = deterministic (with low temperature). This means cache is reliable; you won't get surprised by drift.

3. Downstream Logic Is Cheap

All the smart stuff—confidence filtering, component remapping, bias correction—happens on structured JSON. No vision calls. This is where iteration speed counts, and it's free.

Implementation Checklist

Phase 1: Build the Pre-cache

• Collect 500–2000 representative benchmark images
• Write a vision classification script (similar to the code above)
• Parallelize with ThreadPoolExecutor (4–8 workers)
• Save each result as JSON with ground-truth labels
• Log success rate; aim for 90%+ match accuracy

Phase 2: Iterate Offline

• Load cached JSON into your app/testing harness
• Build logic against cache: confidence rules, defaults, component mapping
• Measure accuracy without touching the API
• Tweak prompts, templates, enum values—all free
• When confident, commit updated templates to repo

Phase 3: Cache Invalidation (critical)

• If you change a system prompt → delete affected cache files and rebuild
• If you tweak component defaults/min/max → no rebuild needed
• If you add a new confidence rule → no rebuild needed
• Document which changes require re-cache (a config comment or runbook)

Phase 4: Production Caching (optional—for user requests)

• Implement local image-hash → prediction cache on your server
• Cache hits on similar food photos save €0.005 each
• Set a cache TTL (e.g., 30 days) or invalidate on model updates

Where JuiceFactory Fits

Qwen3-VL on JuiceFactory gives you:

• Transparent pricing: a published per-image rate with no surprise overage, so you can forecast cost up front instead of reverse-engineering an invoice.
• Predictable latency: 2–5 sec per VL call (good enough for async batch jobs).
• Vision quality: Qwen3-VL is competitive for food classification; it handles reference objects (coins, rulers) natively.
• GDPR compliance: EU-hosted by default. No image pixels sent to the US. Critical for health data.

For the matkalkylen team, this meant: no vendor lock-in, no T&Cs that forbid caching, and billing that's easy to forecast. For a wider view of how EU-hosted LLM providers price vision and text inference, compare the options before you commit.

Real Numbers: A Worked Example

Scenario: Nutrition app classifying user-uploaded food photos.

By month 6, the original €5 investment is spread across ~3,000 inferences — about €0.0017 of upfront cost per query, on top of the €0.005 live call. That's sustainable at scale, and it keeps shrinking. (Treat the per-image numbers as illustrative — verify the current Qwen3-VL rate before you plug them into your own model.)

Compare to the naive approach: €50/month (re-running tests 10x). This saves €540/year on vision refinement alone.

Where This Breaks Down (Honest Limits)

• High cardinality corpus (100k+ dishes): second VL call gets slow. Consider bucketing or tiering.
• Model updates: If Qwen3-VL improves in six months, your cached predictions become stale. Plan for periodic re-caching (maybe quarterly).
• Diverse image domains: If your users upload photos that differ wildly from your benchmark set (e.g., professional plating vs. home meals), cache hit rate drops. Mitigate with a larger, more varied benchmark.
• Deployment latency: Pre-cache has a one-time 20–30 min pipeline cost. Not suitable for one-off classification tasks.

This pattern works best when: classification is recurring, your corpus is stable, and you iterate on logic more than vision models.

Next Steps

1. If you're starting: grab the matkalkylen source code as a template. Adapt build_cache.py for your corpus.

2. If you're already calling vision APIs: audit your last 100 calls. How many were identical queries or slight prompt variations? If >20%, pre-caching will save money.

3. If you're worried about cost: use JuiceFactory's transparent pricing to model your Phase 1 investment, then confirm the current per-call rate so you can calculate exactly when pre-cache pays for itself.

4. For production safety: implement cache versioning (e.g., cache v1, v2 per model release) so you can A/B test new models against old predictions.

Further Reading

For deeper cost optimization on LLM APIs, check out:

• Prompt Compression and Cache Tuning: Cut Your LLM API Costs by 60%
• LLM Cost Optimization: 5 Levers to Cut API Spend 70-85%
• How to Reduce LLM API Costs by 60%

And for vision-specific patterns:

• 10 Strategies to Reduce LLM Costs
• LLM Cost Optimization: Complete Guide to Reducing AI Expenses by 80%

tl;dr: Vision calls are expensive. Pre-compute them once on a benchmark set (€5, one-time). Iterate on everything downstream for free (templates, logic, prompts—cached). Deploy knowing your amortized cost per inference is micro-cents. At scale, you've saved €500+/year and moved faster than the naive "call API every time" flow.

Now go build something cool. 🚀

Start building: Get a free JuiceFactory API key — no credit card required.

Related guides

Related guides: EU LLM API Comparison · Multi-Modal AI Assistant · Multi-Modal Fraud Detection · Structured Data Extraction