Reviewing Model Inputs
Alongside model-output measures such as accuracy, latency, and token counts, this article discusses reviewing the specification supplied as input.
The author adapts Signal-to-Noise Ratio to compare useful specification content with ambiguity and redundancy in a prompt.
The SNR definition is a proposed prompt-review measure. An output-quality comparison needs a declared workload, scoring method, model settings, and observed results.
Defining SNR for AI Prompts
This formula comes from classical signal processing:
In this ratio definition, 1.0 represents all signal and no noise. Treat any operational SNR threshold as a design assumption to evaluate, not a quality guarantee. Related: sinc-prompt specification.
SNR is not about prompt length. A 200-token prompt can score 0.92. A 2,000-token prompt can score 0.01. What matters is the ratio of useful tokens to total tokens, not how many words you write.
What Counts as Signal
For this prompt-review method, classify content according to the six specification fields:
- PERSONA example: “You are a senior data engineer” states the requested role.
- CONTEXT example: “We are migrating from PostgreSQL 12 to 15 on AWS RDS” states the environment.
- DATA example: table count, row volume, and daily writes describe the supplied workload.
- CONSTRAINT example: maximum downtime, data-loss limits, and read-replica requirements state rules.
- FORMAT example: a numbered migration plan and table state the requested structure.
- TASK tokens: "Design the migration sequence" uses 4 signal tokens.
This is an illustrative classification of the example’s specification content. Token counts depend on the selected tokenizer and the classification rule.
What Counts as Noise
A token is noise if it adds no useful information or makes things less clear:
- Filler words: "I was wondering if you could maybe help me with..." adds 10 noise tokens. None of them tell the model what you need.
- Redundant politeness: "Please, if it is not too much trouble..." adds 8 noise tokens.
- Vague qualifiers: "Give me a good strategy" uses "good" as noise. It carries no real information. Good how? Cheap? Fast? Thorough? The model has to guess.
- Implicit context: "You know, the usual approach" adds 5 noise tokens. The model does not know your "usual." It guesses from training data.
- Unnecessary hedging: "Maybe you could try to..." adds 5 noise tokens. These words actually increase uncertainty because they signal that the task itself is uncertain.
- Restating the obvious: "As an AI language model, you can..." adds 7 noise tokens. The model already knows what it is.
How to Calculate Your Prompt SNR
The sinc-LLM validator does this calculation for you. To do it by hand:
- Count all tokens in your prompt (any tokenizer works, such as tiktoken or cl100k).
- For each token, ask: "Does this token tell the model something specific about PERSONA, CONTEXT, DATA, CONSTRAINTS, FORMAT, or TASK?"
- If yes, it is a signal token. If no, it is a noise token.
- SNR = signal count / total count.
For a manual approximation, identify names, supplied data, explicit instructions, formats, and boundaries. Apply the same classification rule to both prompt versions.
SNR Ranges: Illustrative Categories
Scroll sideways to see all table columns.
| SNR Range | Classification | Typical Source | Output checks required |
|---|---|---|---|
| 0.00 - 0.05 | Illustrative low range | Casual conversational prompts | Assess task completion and factual correctness separately. |
| 0.05 - 0.20 | Illustrative range | Slightly structured natural language | Assess task completion and factual correctness separately. |
| 0.20 - 0.50 | Illustrative middle range | Prompts with some constraints | Assess task completion and factual correctness separately. |
| 0.50 - 0.70 | Illustrative range | Structured prompts with most bands | Assess task completion and factual correctness separately. |
| 0.70 - 0.90 | Illustrative high range | Full sinc format with all 6 bands | Assess task completion and factual correctness separately. |
| 0.90 - 1.00 | Illustrative upper range | Optimized sinc with constraint saturation | Assess task completion and factual correctness separately. |
The ranges above are illustrative categories for this measure. They do not describe a verified distribution of users or deployments, or predict output quality. Related: $200 billion blame game.
A Worked Prompt-Structure Example
Short prompt example:
"Hey, I need help with my database. It is running slow and I do not know what to do. Can you give me some suggestions for making it faster? We use PostgreSQL and it has been getting worse over the past few months. Any ideas would be great, thanks!"
Total tokens: about 55. Signal tokens: about 3 (PostgreSQL, slow, database). Noise tokens: about 52. SNR = 3/55 = 0.054.
Six-field prompt example:
PERSONA: PostgreSQL DBA with 10+ years production experience CONTEXT: PostgreSQL 14 on AWS RDS db.r6g.xlarge. 300GB data. 847 tables. Degradation started 3 months ago after adding 3 new reporting queries. DATA: Slowest query: 47 seconds (was 2 seconds). pg_stat_statements shows sequential scans on orders table (180M rows). Connection count: 85 average, 340 peak. CPU: 78% average. IOPS: 12,000 (provisioned: 15,000). CONSTRAINTS: Cannot add read replicas (budget). Cannot upgrade instance size. Must maintain <5 second response time for top 10 queries. Changes must be reversible. No downtime. FORMAT: Ranked list of optimizations. Each item: problem description, exact SQL fix, expected improvement percentage, risk level, reversibility. TASK: Diagnose the 3 highest-impact performance bottlenecks and provide the exact fixes.
Total tokens: about 165. Signal tokens: about 129. SNR = 129/165 = 0.78.
The two prompts specify the database task at different levels of detail. The arithmetic illustrates the author’s classification method; it does not establish an observed output improvement.
Use SNR as one proposed input-review measure alongside task-specific output checks.
// Production AI Engineering
Build AI systems that hold up in production.
sinc-LLM designs, audits, and stabilises production AI infrastructure: from vendor evaluation and cost accountability to incident controls and MCP architecture.
See what we do →
sinc-LLM