Shared runtime
- One agent entry point (runtime.invoke_agent_for_user / prepare_agent_turn) for
web, email, iMessage, WhatsApp, scheduled and cron, so identity, permissions,
memories, summaries and metrics match across channels.
- Serialize turns per checkpoint thread; /query validates input and returns 409
while a durable run owns the conversation.
Reconnectable chat (durable runs)
- POST /runs creates an idempotent, owner-scoped run; GET .../events replays an
ordered SSE log by cursor, so generation survives refresh and device switches.
Stop cancels; disconnect only detaches. /stream-query is a compatibility adapter.
- Coalesce streamed deltas (150 ms) and wake same-process subscribers instead of
polling; keep buffered text when a run is stopped.
- Drain runs on shutdown; serve via Hypercorn with a shutdown timeout covering
the drain (and no longer in debug mode). CHAT_RUN_* settings are forwarded.
Scheduled messages and cron
- Atomic delivery claims, attempt history and explicit retries; ambiguous sends
become "uncertain" and are never resent automatically.
- Cron triggers need a run_id/Idempotency-Key; legacy jobs need an explicit owner.
- Cancelling never rewrites a started delivery; series stop via tombstones.
Legacy sent rows are not forked into new series. Malformed bodies return 400.
History, integrations and retrieval
- Message-content search, pinned conversations, JSON/Markdown export and a
memory editor.
- Integration status panel and YNAB data-freshness provenance.
- Date-filtered document/note search with stable citations and a source viewer.
Migration 20 is generated by aerich.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llama-server only reuses a byte-identical prompt prefix. Several things
changed the prefix every turn, forcing most of the prompt to be re-prefilled:
- Memories: rank still chooses which memories fit the budget, but the chosen
ones are emitted in stored order, so the block is identical across queries.
- History trimming: the conversation-evidence summary is only included when
turns were actually dropped (it duplicated retained turns otherwise). On
overflow, old turns are dropped at stable cut points down to ~65% of the
budget, so the retained prefix holds for several turns instead of shifting
by one turn every request.
- Tools: TOOL_ROUTING_MODE=static (default) binds the full role-appropriate
tool set in registration order with no discovery tool; admin gating is
unchanged. TOOL_ROUTING_MODE=dynamic keeps the keyword router.
- Observability: per-call model_calls in the stream_request log now include
cached_prompt_tokens when the provider reports cache reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 30s time-to-first-token guard measured from the turn's first model call
to the first answer token, so thinking, tool execution and the second call's
prefill all counted against it. Long reasoning on a long conversation was
cancelled mid-generation (llama-server: stop: cancel task at n_gen=1287).
- stream_query: the timeout now fires only when an in-flight model call goes
silent for TTFT_TIMEOUT_SECONDS. Any chunk (thinking or empty) resets the
clock, each model call starts a fresh one, and time spent in tools between
calls doesn't count.
- Trim replayed history (blueprints/conversation/context.py): tool results
and user messages from earlier turns are capped at 4000 chars (attachments
included) and their images replaced by a placeholder. The current turn is
untouched. The window no longer starts with orphaned ToolMessages.
- Log per-call prefill time, duration and prompt/output tokens
(stream_usage=True) plus max_prompt_tokens in the stream_request line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ryan Chen