Skip to content
sinc-LLMBook a 30-minute call

Engineering reference

Reviewing LLM Costs With Structured Prompts

A first-person account of reviewing LLM spending and changing prompt structure, with implementation lessons.

The services, methods, and examples described here do not establish an expected performance result for your workload.

Published March 27, 2026 · By Mario Alexandre
Baseline
Before (monthly)
Comparison
After (monthly)
Prompt-cost review

I reviewed an OpenAI workflow by applying the sinc-LLM six-field format and examining prompt content, retries, and output length. Related: sinc-LLM's.

This is the story of how I learned that most LLM costs come from bad prompts, not from the price of the model.

I looked at our API usage and found something I did not expect. A typical task was not one request and one reply. It was one request, then a reply that was wrong. Then a fix request. Then another reply. Then another fix. The average task took 3.2 API calls to finish.

A repeated clarification can resend conversation context. Count the input and output tokens, responses, and retries when comparing the complete task cost.

Illustrative clarification loop: a request for a document summary omits length and format, so the user asks for a shorter version and then bullet points. When later calls include earlier messages, count that history as part of input usage. The growth depends on the actual messages sent.

The Realization: Prompts Are Signals

I found sinc-LLM while reading about prompt engineering. The main idea was clear to me right away: a prompt is a signal. A vague prompt causes aliasing. The model fills in the gaps with its own guesses, and those guesses rarely match what you want.

The sinc-LLM framework uses the sampling theorem to show this:

x(t) = Σ x(nT) · sinc((t - nT) / T)

If you fill all 6 bands (PERSONA, CONTEXT, DATA, CONSTRAINTS, FORMAT, TASK), the model can rebuild your intent with no gaps. If you only fill 2 or 3 bands, the model guesses the rest. The output comes out wrong, and you have to send fix requests.

I saw that our fix requests were just supplying the missing bands after the fact. "Make it shorter" was the CONSTRAINTS band. "Format as bullet points" was the FORMAT band. We were paying to send the full conversation again just to add information we could have put in the first prompt.

I changed our 20 most-used prompt templates to the sinc-LLM 6-band format. Here is what the "document summary" prompt looked like before and after:

Before (raw prompt, average 3.2 calls):

"Summarize this document: [document text]"

After (sinc-LLM structured, average 1.0 calls):

{
  "formula": "x(t) = Σ x(nT) · sinc((t - nT) / T)",
  "T": "specification-axis",
  "fragments": [
    {
      "n": 0, "t": "PERSONA",
      "x": "Technical writer producing executive summaries. Concise, factual, no opinions."
    },
    {
      "n": 1, "t": "CONTEXT",
      "x": "Internal document summary for the engineering team's weekly digest. Readers are senior engineers who need to quickly assess relevance."
    },
    {
      "n": 2, "t": "DATA",
      "x": "[document text inserted here]"
    },
    {
      "n": 3, "t": "CONSTRAINTS",
      "x": "Exactly 3 sentences. First sentence: what the document is about. Second sentence: the key finding or recommendation. Third sentence: action items or next steps. Do not include background information. Do not include the author's qualifications. Do not use the phrase 'this document.' No bullet points. Under 80 words total."
    },
    {
      "n": 4, "t": "FORMAT",
      "x": "Plain text paragraph. No markdown headers. No bullet points. Three sentences separated by periods."
    },
    {
      "n": 5, "t": "TASK",
      "x": "Write the 3-sentence executive summary of the provided document following all constraints."
    }
  ]
}

The Results: Month by Month

Baseline: record model settings, raw prompts, API calls per task, tokens, and the workload.

February: $320. The top 10 templates were changed to 6-band format. Average 1.4 calls per task. Still 15,000 tasks. The tasks still using old templates were the ones with high call counts.

For the comparison period, record which templates changed, task volume, accepted outputs and all calls requiring human review. Use the resulting usage and billing records to calculate cost; this paragraph establishes no monthly total or retry rate.

Why Structured Prompts Save Money

Review four possible contributors to the cost of a completed task:

1. Clarification calls: record why a response needed another attempt. Test whether stating the missing constraints upfront changes the total calls required for an accepted result.

2. Output length: state the requested format and length, then check compliance. Count the actual output tokens and charges rather than assuming a fixed saving from a shorter brief.

3. Model selection: test a smaller model on the same acceptance criteria as the current model. Include retries and review effort before moving a workload.

4. Prompt caching: identify repeated prompt sections and inspect the provider’s actual cache records and billed charges. Compare complete costs; repeated text alone does not establish a cache hit or a saving.

I finished this project with a clear belief I did not have before: cutting LLM costs is mainly a prompt problem, not an infrastructure problem. Caching, batching, and model choice all matter. But they are small gains compared to writing good prompts.

The 6-band structure from sinc-LLM is not just a style rule. It forces you to think about what you really want before you send the request. That thinking up front is what kills the expensive fix loops later.

If the bill is higher than expected, count calls per accepted task and inspect the reason for each retry. Review prompt requirements alongside model selection, caching, tool use and human review; a response ratio alone cannot diagnose the cause.

// Production AI Engineering

Build AI systems that hold up in production.

sinc-LLM designs, audits, and stabilises production AI infrastructure: from vendor evaluation and cost accountability to incident controls and MCP architecture.

See what we do →
Additional context

sinc-LLM About Blog Services

© 2026 sinc-LLM. All rights reserved. DOI: 10.5281/zenodo.15086622