v0.0.6: 4x Cheaper by Knowing Which Model to Use

The Latency Problem

After v0.0.5 shipped media support, Mio could see images, hear voice messages, and display emoji. Feature-complete for a companion chat. But there was a problem I'd been ignoring: response latency.

Gemini 3 Pro Preview — the model powering all of Mio's chat — took 8-10 seconds for a first token when thinking was enabled. In a companion conversation, that's an eternity. You send "hey, how was your day?" and then you sit there watching a typing indicator. It kills the conversational illusion.

I knew the latency was bad. I'd been living with it during development because I was focused on features. But now that the features were done, the latency was the most obvious remaining problem.

Research First, Then Code

Instead of guessing, I did a proper comparison: Gemini 3.1 Pro vs Gemini 3 Flash, specifically for emotional companion AI.

The numbers told a clear story:

The cost difference was 4x. The latency difference was the one that mattered: 1-2 second vs 8-10 seconds.

But what about quality? For companion AI, emotional nuance is the product. If Flash couldn't hold a character, the cost savings would be meaningless.

The community consensus was surprisingly clear: Flash was a "side-grade to 2.5 Pro, not a downgrade." The quality gap was about 1-2% on extreme emotional nuance — expert writers could spot the difference, most users couldn't. Flash was actually better at narrative initiative and character commitment.

One critical finding: thinking mode actively degrades creative writing quality. Multiple developers reported that Gemini's reasoning mode makes emotional and creative output worse, not better. The recommendation was thinkingLevel: minimal for Flash — which, conveniently, also gave the 1-2 second TTFT.

A counterintuitive reversal: turning on thinking mode makes creative and emotional writing worse, while minimal thinking is better.

The Routing Architecture

Not all tasks are equal. Daily chat needs speed. Personality extraction needs depth. The solution was obvious: route by task type.

Route requests by task type: 90% of everyday chat goes to the cheap fast Flash, 10% of high-value tasks go to the deeper Pro.

Chat (90% of API calls): Gemini 3 Flash with thinkingLevel: minimal

• 1-2s TTFT
• 214 tokens/second throughput
• ~4x cheaper than Pro per million tokens
• Good enough emotional nuance for conversational flow

Premium tasks (10% of API calls): Gemini 3.1 Pro with thinkingLevel: low

• Personality extraction from onboarding conversations
• Memory summarization and emotional pattern analysis
• Tasks where you pay once and cache the result
• Deeper emotional nuance justified by low frequency

This split meant ~4x cost reduction on 90% of traffic, while actually improving quality on the 10% that matters most — because we upgraded premium tasks from 3 Pro to 3.1 Pro.

Vision That Actually Sees

v0.0.5 added image uploads. The AI could "see" images, but the descriptions were generic. Send a screenshot of a game and you'd get "I see a colorful game screen." Not useful for a companion who's supposed to care about what you're doing.

v0.0.6 rewrote the vision prompts to be specific: identify games by name, recognize brands and logos, describe locations with context. The thinking level for vision went from MINIMAL to LOW — a small latency cost for significantly better understanding.

The difference: "I see a game" became "Looks like you're playing Genshin Impact — how's the new region?" That's the kind of response that makes a companion feel present.

Chinese Transcription Fix

Voice messages were coming through with English transcription artifacts. A user sends a voice clip in Mandarin, and the transcription would hallucinate English words or miss colloquial expressions.

Two fixes:

1. OpenAI transcriber: Added explicit language: 'zh' parameter. Without it, the model was auto-detecting language per-segment and occasionally getting confused by code-switching or background noise.
2. Gemini fallback prompt: Enhanced for colloquial Chinese expressions and proper nouns. When OpenAI's transcription wasn't confident enough, the Gemini fallback now handles slang, internet-speak, and Chinese-specific proper nouns correctly.

Small change, big impact. Voice is the most intimate input modality — getting the transcription wrong breaks immersion harder than anything else.

DRY Refactor

A code quality issue had been nagging since v0.0.5: the POST /chat and POST /chat/stream handlers shared nearly identical logic for media resolution and context preparation. About 80 lines of duplicated code.

Extracted two shared helpers:

• resolveMedia() — fetches pending uploads by mediaId, validates, returns processed media
• prepareChatContext() — builds the conversation context with history, persona, and media

Both handlers now call the same functions. ~80 lines eliminated. The audit from v0.0.5 had flagged this as a MEDIUM issue — it was the right time to fix it.

Platform Risk

One finding from the research deserves its own section: Google explicitly does not want Gemini used for emotional companion AI. Their team lead said it publicly — Gemini is positioned as "a super tool, not an emotional companion."

Three concrete risks:

1. An external safety filter operates independently and can delete responses mid-sentence, even with BLOCK_NONE
2. Google has banned developers using the API for roleplay-adjacent use cases
3. Future model versions may further restrict emotional/creative output

This doesn't change the v0.0.6 decision — Flash is still the best option today. But it does change the architecture requirement: Mio needs to be model-agnostic enough to swap providers without rewriting the application. The model routing layer we just built makes that easier, not harder.

The Numbers

v0.0.6 was six commits over a few hours:

The most impactful release yet, measured by user experience per line of code changed.

Data-Driven, Not Vibes-Driven

Part 0 of this series was a forensic analysis of why the previous companion framework had an unsustainable early burn rate. The lesson was: understand your costs before they understand you.

v0.0.6 applies the same principle to model selection. Not "Pro sounds better than Flash" — but actual latency benchmarks, cost calculations, quality comparisons, and community experience. The research said Flash with minimal thinking was the right call. The deployment confirmed it.

Mio now responds in 1-2 seconds instead of 8-10. That single change makes the conversation feel natural instead of stilted.