Skip to content
sinc-LLMBook a 30-minute call

Engineering reference

How Clarification Loops Affect Your LLM Bill

The author's prompt-cost analysis, including exchange-rate definitions, repeated interactions, and a revised prompting method.

Methods, definitions, and examples for the topic discussed here. Check applicability against the task and operating conditions.

Mario Alexandre  ·  March 26, 2026  ·  llm-costs token-savings prompt-engineering

You have probably looked at your Anthropic or OpenAI bill and asked: why is this so expensive? You have probably tried the obvious things. You switched to smaller models. You cached responses. You shortened your system prompts. Maybe you even tried writing better prompts for a week before giving up.

None of that fixed it. The reason is simple: the actual problem is not what you think it is.

The problem is your exchange rate. Most people have never heard of this as a metric.

Exchange rate, in the way I use it here, is the number of AI responses you get per prompt you send. In a perfect world, every prompt gets one good answer. That is an exchange rate of 1.0.

In the real world, the model often does not have enough information to answer correctly on the first try. So it asks a clarifying question. Or it answers the wrong question and you have to correct it. Or it does half the task and you prompt again for the rest. Each extra exchange multiplies your cost.

To assess the bill, count responses and completed tasks over a defined period. Inspect conversation history and retries in the actual billing records; a response-to-prompt ratio alone is not a token-cost multiplier.

Before (unstructured)
4.2x
responses per prompt
After (sinc scatter)
1.6x
responses per prompt

The Model Isn't the Problem

People constantly blame the model. They say things like: Claude is too wordy. GPT-4 asks too many questions. Gemini does not follow instructions.

The model is doing exactly what it should do. When it lacks information, it asks. That is correct behavior. The real problem is that your prompts are not giving it enough to work with.

A structured prompt fixes this. It tells the model who it should be, what the context is, what data matters, what limits apply, what format to use, and what the task actually is. With all of that in one prompt, the model gets everything it needs up front. You get your answer in one shot.

sinc-LLM — treating prompts as sampled signals
x(t) = Σ x(nT) · sinc((t - nT) / T)

Here is why the cost grows so fast. Each exchange in a conversation carries the full history of every exchange before it. So if you need 10 exchanges to finish one task, the last few exchanges are paying input tokens for all 10 turns of history.

In my measurement: 12.9 million output tokens in 7 days. 7.14 billion total tokens across the whole system. Output tokens cost more per token. But the huge volume of input tokens from compounding context is where the surprise bill comes from.

Compare responses per prompt, token use, scatter overhead, and output quality checks for the same workload.

Why Smaller Models Alone Don't Fix This

Many people try to cut costs by switching to cheaper models. That helps a little. But it misses the main problem. If your exchange rate is 4.2, switching from Claude Sonnet to Claude Haiku lowers your per-token cost. It does not lower your exchange count. You still get 4.2 responses per prompt. They are just cheaper per token.

The real fix is lowering your exchange rate. To do that, give the model complete information on the very first prompt, every time. That takes a structural change in how you build prompts. Switching price tiers does not solve it.

I built an auto-scatter hook that receives prompts before execution, calls Claude Haiku to separate six fields, and injects the structured result as context. Include both the scatter and main-model calls in a cost comparison.

The code is open source. Leave a comment and I will drop the link. Setup takes about 15 minutes. If your bill is anywhere near what mine was, it pays for itself in the first hour of use.

// Production AI Engineering

Build AI systems that hold up in production.

sinc-LLM designs, audits, and stabilises production AI infrastructure: from vendor evaluation and cost accountability to incident controls and MCP architecture.

See what we do →
Scope and context

sinc-LLM About Blog Services

DOI: 10.5281/zenodo.19152668 · © 2026 sinc-LLM · Mario Alexandre