The Token Bloat Problem
AI systems that use large language models grow slowly out of control. The context window fills up with old conversation history. System prompts get longer each time someone adds a fix for a new edge case. Retry loops make the token cost even higher. A system that starts at 5,000 tokens per request can reach 80,000 tokens within months.
My sinc-LLM paper discusses token bloat and the author's proposed signal-to-noise approach to prompt review.
Signal-to-Noise Ratio for Prompts
The author uses SNR as a ratio of useful prompt content to total content. Compare that input measure with separate output checks:
Scroll sideways to see all table columns.
| Prompt approach | Input tokens | Defined input score | Complete task cost |
|---|---|---|---|
| Sliding-window context | Not measured | Not measured | Not measured |
| Band decomposition | Not measured | Not measured | Not measured |
| Sleep-time consolidation | Not measured | Not measured | Not measured |
An SNR of 0.003 means only 0.3% of tokens carry useful information. The rest is noise: repeated history, duplicate context, and filler text.
Three Optimization Techniques
1. Band Decomposition
Organize instructions into PERSONA, CONTEXT, DATA, CONSTRAINTS, FORMAT and TASK. Review repetition and irrelevant material while preserving required context. Count tokens and evaluate outputs before deciding what to remove.
2. Topic-Shift Detection
In a conversation, detect when the topic changes (threshold: 0.15 cosine distance) and remove history from the old topic. Most conversation history is about a different topic than the current request.
3. Deduplication
Find messages in context that say the same thing (threshold: 0.6 similarity) and keep only the newest one. Long conversations pile up many versions of the same information.
Implementation Architecture
I built the sinc-LLM pipeline to process prompts in three stages:
Raw Prompt (80,000 tokens) | v [Band Decomposition] -- extract 6 specification bands | v Structured Prompt (3,500 tokens, SNR 0.78) | v [Sleep-Time Consolidation] -- async dedup + topic pruning | v Optimized Prompt (2,500 tokens, SNR 0.92)
The added delay on the main path is only +8ms. You will not notice it in production. The sleep-time consolidation runs in the background via setTimeout(fn, 0) and does not slow down requests.
Getting Started
Here is how to start optimizing your prompts today:
- Measure your current token usage and SNR (specification tokens divided by total tokens).
- Apply band decomposition to your 5 highest-token prompts.
- Add the sinc-LLM framework to automate the optimization.
- Try the free online transformer for quick experiments.
The framework is open source. The full methodology is in my paper at DOI: 10.5281/zenodo.19152668.
Review a prompt using six specification fields
Try sinc-LLM FreeReal sinc-LLM Prompt Example
This is the exact JSON format that sinc-LLM uses. Paste any raw prompt at sincllm.com to generate one automatically.
{
"formula": "x(t) = Σ x(nT) · sinc((t - nT) / T)",
"T": "specification-axis",
"fragments": [
{
"n": 0,
"t": "PERSONA",
"x": "You are a LLM optimization engineer. You provide precise, evidence-based analysis with exact numbers and no hedging."
},
{
"n": 1,
"t": "CONTEXT",
"x": "This analysis is part of a production system where accuracy determines revenue. The sinc-LLM framework identifies 6 specification bands with six named fields."
},
{
"n": 2,
"t": "DATA",
"x": "Example allocation assumptions: CONSTRAINTS=42.7%, FORMAT=26.3%, PERSONA=7.0%, CONTEXT=6.3%, DATA=3.8%, TASK=2.8%. SNR formula: 0.588 + 0.267 * G(Z1) * H(Z2) * R(Z3) * G(Z4). These allocations are illustrative assumptions, not observed quality weights."
},
{
"n": 3,
"t": "CONSTRAINTS",
"x": "State facts directly. Never hedge with 'I think' or 'probably'. Use exact numbers for every claim. Do not suggest generic solutions. Every recommendation must be specific and verifiable. Include at least 3 MUST/NEVER rules specific to this task."
},
{
"n": 4,
"t": "FORMAT",
"x": "Lead with the definitive answer. Use structured headers. Tables for comparisons. Numbered lists for sequences. Code blocks for implementations. No trailing summaries."
},
{
"n": 5,
"t": "TASK",
"x": "Optimize a 3,000-token prompt to under 500 tokens while maintaining SNR above 0.85"
}
]
}// Production AI Engineering
Build AI systems that hold up in production.
sinc-LLM designs, audits, and stabilises production AI infrastructure: from vendor evaluation and cost accountability to incident controls and MCP architecture.
See what we do →
sinc-LLM