Budget conversation context and preserve complete tool exchanges #69

2026-10-12T19:46:09-04:00

ryan commented

2026-10-12 19:46:09 -04:00

The fixed 12-message history window can drop the current user question during a long tool turn, while full tool results and uncapped memories keep increasing prompt size. Replace it with an estimated token budget that includes system instructions and tool schemas, reserves generation space, and preserves the current question and complete tool exchanges.

Compact oversized results, select recent history by whole turns, retain bounded excerpts from older turns, and bound saved memories without changing checkpointed content. Remove incomplete exchanges left by cancelled turns. Add optional line-range reads for Obsidian and context-budget logs.

Defaults: 16,000 estimated prompt tokens, a 32,768-token context window, 4,096 generated tokens, and a 2,048-token safety reserve. Counts use UTF-8 estimates rather than a backend tokenizer; historical excerpts are not a generated rolling summary. Configuration and limitations are documented in docs/context-management.md.

Validation: 57 tests pass across tests/unit/test_context_trim.py, tests/unit/test_stream_stall.py, and tests/unit/test_tool_artifacts.py. Ruff checks and formatting hooks pass.

The fixed 12-message history window can drop the current user question during a long tool turn, while full tool results and uncapped memories keep increasing prompt size. Replace it with an estimated token budget that includes system instructions and tool schemas, reserves generation space, and preserves the current question and complete tool exchanges. Compact oversized results, select recent history by whole turns, retain bounded excerpts from older turns, and bound saved memories without changing checkpointed content. Remove incomplete exchanges left by cancelled turns. Add optional line-range reads for Obsidian and context-budget logs. Defaults: 16,000 estimated prompt tokens, a 32,768-token context window, 4,096 generated tokens, and a 2,048-token safety reserve. Counts use UTF-8 estimates rather than a backend tokenizer; historical excerpts are not a generated rolling summary. Configuration and limitations are documented in docs/context-management.md. Validation: 57 tests pass across tests/unit/test_context_trim.py, tests/unit/test_stream_stall.py, and tests/unit/test_tool_artifacts.py. Ruff checks and formatting hooks pass.

ryan added 1 commit 2026-10-12 19:46:09 -04:00

Budget conversation context while preserving current turns and tool exchanges 1c9405cfdf

ryan merged commit 5d504e0314 into main

2026-10-12 19:46:36 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ryan/simbarag#69