44 lines
2.4 KiB
Markdown
44 lines
2.4 KiB
Markdown
# Conversation context management
|
|
|
|
The LangGraph checkpoint retains original messages and artifacts. Each model
|
|
call constructs a bounded view; compaction never rewrites the checkpoint.
|
|
|
|
The prompt budget includes system instructions, saved memories, tool schemas
|
|
(including admin tools), message content and tool arguments. The default target
|
|
is 16,000 estimated tokens. A 4,096-token generation limit and 2,048-token safety
|
|
reserve also constrain the budget against the configured 32,768-token window.
|
|
|
|
Estimates use UTF-8 bytes divided by three, message overhead, and a fixed image
|
|
allowance. These are estimates rather than model-specific tokenizer counts.
|
|
Compare `prompt_budget.estimated_prompt_tokens` with actual `stream_request`
|
|
model-call usage and adjust the target/window for the deployed backend.
|
|
|
|
The current user request is always retained. Oversized tool results become
|
|
marked excerpts preserving their beginning and end; all tool IDs and arguments
|
|
remain. If necessary, intermediate assistant text is also shortened. If the
|
|
mandatory request and exchange structure cannot fit, the request fails with an
|
|
explicit error rather than silently dropping the question.
|
|
|
|
Recent history is selected as complete turns. Older user messages, assistant
|
|
answers and tool results are capped; earlier images become placeholders.
|
|
Incomplete tool exchanges from cancelled calls and orphan results are removed
|
|
from the prompt. A small budget is reserved for bounded question/answer excerpts
|
|
from older turns. These excerpts are not an LLM-generated rolling summary and do
|
|
not guarantee retention of every historical decision. No extra model call is
|
|
added to the request path. Saved memories use a separate 1,000-token estimated
|
|
budget, preferring recent entries without deleting omitted memories.
|
|
|
|
Obsidian reads accept `start_line` (one-based content line, excluding frontmatter)
|
|
and `max_lines`, allowing focused follow-up reads. Full artifact content remains
|
|
available to the interface and checkpoint.
|
|
|
|
Configuration:
|
|
|
|
- `CONTEXT_WINDOW_TOKENS`: smallest supported context window across backends.
|
|
- `PROMPT_BUDGET_TOKENS`: prompt target, including fixed instructions and tools.
|
|
- `RESPONSE_RESERVE_TOKENS`: maximum generated tokens and reserved answer space.
|
|
|
|
`context_budget` logs history estimates, message counts, dropped turns and excerpt
|
|
inclusion. `prompt_budget` logs fixed overhead and the complete prompt estimate.
|
|
Neither log includes message content.
|