The ledger supersedes the "Earlier conversation excerpts" message, as on
main; sending both duplicated dropped-turn content. Removal is consistent
whenever the ledger is included, so the cached prefix stays stable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The summary ledger is regenerated every turn. At the head of history it
invalidated the cached retained turns on every request once trimming
started. Insert it immediately before the current turn's HumanMessage so
system, tools and retained history stay byte-identical; excerpts of dropped
turns stay at the head since they only change when the cut moves.
Reserve a fixed ledger ceiling (SUMMARY_RESERVE_TOKENS) rather than its
current size, so a shrinking ledger cannot pull the cut back and change the
retained prefix; fall back to the actual size for tight budgets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llama-server only reuses a byte-identical prompt prefix. Several things
changed the prefix every turn, forcing most of the prompt to be re-prefilled:
- Memories: rank still chooses which memories fit the budget, but the chosen
ones are emitted in stored order, so the block is identical across queries.
- History trimming: the conversation-evidence summary is only included when
turns were actually dropped (it duplicated retained turns otherwise). On
overflow, old turns are dropped at stable cut points down to ~65% of the
budget, so the retained prefix holds for several turns instead of shifting
by one turn every request.
- Tools: TOOL_ROUTING_MODE=static (default) binds the full role-appropriate
tool set in registration order with no discovery tool; admin gating is
unchanged. TOOL_ROUTING_MODE=dynamic keeps the keyword router.
- Observability: per-call model_calls in the stream_request log now include
cached_prompt_tokens when the provider reports cache reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ryan Chen