Compress the un-cached context Research Copilot re-sends every call
Prune the 27984 un-cached tokens Research Copilot re-sends on every call — retrieved context, conversation history, repeated blocks — NOT the cached prefix
Verify before implementing — this recommendation rests on evidence the engine does not have:
- No before/after quality evaluation — the compressible fraction is a heuristic, not a measured safe-to-cut amount.
- No independent quality evaluation — the quality impact is unmeasured.
Implement only if these checks pass. The full self-critique is in the reasoning trail below.
Payback divides by an implementation cost of ≈€320 under the agent-assisted-v1-2026-08 cost model: 3h of human review at €100/h plus a €20 agent-run allowance — an AI agent implements, your team reviews and verifies. Implemented by hand instead, the same move is estimated at 3 engineer-days (≈€2,400). Every figure is a prior, not a measurement of your team — and the model is versioned so a changed assumption can never move a figure silently.
Context completeness
Confidence 71% is how sure the engine is on the evidence it has. This is how much of the evidence that could change the conclusion the engine could establish for this application — a different question. The declare-context page counts what you have answered workspace-wide, which can legitimately read differently: an “I don't know” is an answer there and establishes nothing here.
- Telemetry
- Provider bill— calibrated from the monthly spend declared for this workspace, not a reconciled invoice
- Per-app criticality
- Prior experiments
- Codebase
- Quality evaluation
We cannot detect Codebase, Quality evaluation — you can tell us, and confidence is re-derived on the fuller evidence.
Observation → Evidence → Inference → Recommendation
— every fact, judgment, self-critique and failure condition behind this decision
Observations (facts)
- [avg_input_tokens] Research Copilot sends 30774 input tokens on average vs 900 output. (N=24,960)
Evidence (objective-relevant)
- Research Copilot averages 27984 UNCACHED input tokens vs 900 output (ratio 31.1) (2790 already cached — not compressible) — paying input rates on likely-static context. (strength 0.8)
Inference (judgment)
Prompt bloat is inflating input-token cost; compression is Pareto-positive under a cost objective but risks a small quality drop under a strict-quality one.
Recommendation (move)
Prune the 27984 un-cached tokens Research Copilot re-sends on every call — retrieved context, conversation history, repeated blocks — NOT the cached prefix
Critical Review (self-critique)
Assumptions
- Assumes ~15% of the prompt is redundant boilerplate that can be pruned without changing answers.
Alternative explanations
- The large prompt may be load-bearing context the model actually uses; trimming it would degrade quality, not just cost.
Missing evidence
- No before/after quality evaluation — the compressible fraction is a heuristic, not a measured safe-to-cut amount.
- No independent quality evaluation — the quality impact is unmeasured.
This is wrong if…
- Wrong if the trimmed context was genuinely used by the model for correctness.
- This figure counts ONLY the un-cached surface. Shortening the cached prefix instead reclaims far less — those tokens already bill at the cache-read rate — and forces a re-warm at the write premium, which can cost more than the trim saves.
- Wrong if this workload's real quality bar is higher than assumed and the change degrades answers.
Download the canonical decision record (JSON) — the deterministic decision block (byte-pinned by CI), the run context (engine + catalog versions, measured-window boundaries, exact pricing rows, fixed FX with effective date, formula inputs, the overlap group, and the ledger version), and the LLM-phrased narrative as a visibly separate section.
Business case & technical detail
— the phrased explanation — the decision above is computed without it
Business case
Research Copilot re-sends a large un-cached context on every call and pays full input rate for it; pruning ~15% of that surface cuts 12.8% of this application’s spend. Estimated €3,240/yr at the current run-rate, payback 37 days, confidence 71%.
Technical detail
Dedupe retrieved context, cap conversation history, and remove repeated blocks from the PER-CALL portion of the prompt — measure quality before/after. Leave the cached prefix byte-identical: changing it re-writes the cache at the write premium, and the tokens it holds already bill at the cache-read rate — so the trade is the wrong way round. Affects ~31,200 req/mo; expected cost -12.8%, latency -5%, quality unmeasured (medium risk).
Benchmark context & Optimization Memory
— your value against the cohort, and what prior outcomes taught the engine
Benchmark context
Average input tokens per request (lower is leaner prompts). (illustrative)
Optimization Memory
Learned caution: only 1/3 prior implementations succeeded (N=51,000); confidence penalized.
Why NOT the alternatives?
— each rejected route, with the constraint that rejected it
- Aggressively truncate contextcost-28%latency-10%qualityunmeasured (high risk)Bigger cost cut, but an aggressive prune carries a materially higher unmeasured quality risk on this workload — the quality effect of removing context has never been measured here.
- LLM summarization pre-passcost+5%latency+20%quality+0%Adds an extra model call — raises cost and latency, defeating the purpose.
Implementation packet
advisory_onlyrisk: prompt-shapeReviewed by Bayeto engineering on 2026-07-26 · rule version v1 · 94% of the €56,350/yr open on this workspace is carried by a packet an engineer can execute (5 of 8 open moves); 3 of 14 specs meet that standard. Missing mandatory for this risk class: guardrails, stopIf, rollback, qualityValidation.
Review attests that these steps are sound and reversible as written. It does not guarantee an outcome: whether the change saves money is measured by the validation loop afterwards, never promised by the reviewer.
Not reviewed to executable standard
Guardrails — watch these, Stop if, Rollback, Quality validation are all unwritten for this move, for one reason: Not reviewed to executable standard. Bayeto does not observe prompt content, so it cannot say which tokens carry task-relevant information — an agent acting on this without a human deciding what to cut would be removing text it has never been shown to be safe to remove.
What you would need to decide
Bayeto withholds steps here because these are your calls to make, not ours. Answer them and this move can be specified.
Which parts of this prompt are load-bearing for the task, and which are habit?
Bayeto never sees prompt content. It can price the tokens and cannot read them, so it has no basis whatever for saying which ones carry task-relevant information.
How will you tell that a shorter prompt still produces an adequate answer?
Adequacy is a judgement about your task. Bayeto sees neither the prompts nor the answers, so it cannot construct the comparison, let alone score it.
- Preconditions — 1 specified instruction (mandatory for this risk class)
- Steps — 1 specified instruction (mandatory for this risk class)
- Guardrails — watch these — none specified (mandatory for this risk class)withheld — for the reason stated above
- Stop if — none specified (mandatory for this risk class)withheld — for the reason stated above
- Quality validation — none specified (mandatory for this risk class)withheld — for the reason stated above
- Rollback — none specified (mandatory for this risk class)withheld — for the reason stated above
Instruction text is withheld here: the public demo shows complete packets only for the moves reviewed to the executable standard — 3 of 14 specs — as its worked examples; the rest summarize what exists and what was deliberately not specified.
Simulate — Before You Build
The cost delta derives from this workspace’s telemetry. The latency and quality deltas are heuristic priors carried by the rule: no quality evaluation is connected, and no rule measures the latency effect of its own move — the figures are per-rule constants, so nothing here observes either outcome.
- · Applies to ~31,200 requests/month at current mix.
- · Scenario realization assumption (sim-v1, not a statistical expected value): 0.7 + 0.3 × confidence = 0.914. The 0.7 floor encodes implementation shortfall — even a certain move rarely captures 100% at rollout; rationale published at bayeto.ai/methodology.
- · Per-request cost/latency/quality deltas hold at current traffic shape.