Every SSE request emits a JSON summary line on completion, cancellation,
or error:
{"event":"stream_request","conv":"...","user":"ryan",
"total_ms":45210,"thinking_ms":38100,"ttft_ms":38200,
"tokens":112,"tools":[{"tool":"web_search","ms":820}],
"model":"llama","cancelled":false}
Fields:
- total_ms end-to-end wall time
- thinking_ms time from first LLM call to first content token (reasoning)
- ttft_ms time to first content token from request start
- tokens approximate content token count (stream deltas)
- tools each tool invoked with wall-clock duration
- model LLM name from LangChain metadata (catches fallback to OpenAI)
- cancelled true when client disconnected mid-stream
Tool timers use the LangChain run_id to correctly pair start/end even
when the same tool is called multiple times in one turn.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Aborting the fetch on component unmount sent a disconnect to the server,
which cancelled the in-flight generation. Now the stream runs to
completion in the background (isMountedRef guards prevent stale state
updates), the response is persisted, and it's there when the user
returns to the conversation. stopGeneration() still aborts immediately
on explicit user cancel.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
qwen3.8-27b is a reasoning model: it can think for hundreds of tokens
before any visible content streams. The SSE stream only forwarded
content deltas, so the UI showed dead air for the whole think and long
answers looked hung (users cancel ~1 min in).
The stream now emits a throttled {"type": "thinking"} heartbeat while
the current model turn has produced no content yet, and the frontend
renders a pulsing "Thinking..." pill until content or a tool call
arrives. The heartbeat also keeps bytes flowing on the SSE connection
during long thinks, so idle-timeout disconnects in the proxy chain
can't kill the stream.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two fixes for the agent losing track of conversation context:
- History replay compared msg.speaker (a Speaker enum) to the string
"user", which is always false, so every prior message — including the
user's — was sent to the LLM with role "assistant". Compare against
Speaker.USER instead. Affected web, iMessage, WhatsApp, and email.
- Tool outputs (web searches, RAG chunks) were discarded after each
turn, so follow-up questions about fetched content had nothing to
reference. Persist them as speaker="tool" messages (capped at 4000
chars) and replay them in history as assistant context. The
conversation GET endpoint serves only their header line so the UI
tool pill stays compact.
Also bumps the replayed history window from 10 to 20 messages since
tool messages now share it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Image messages previously ran two sequential LLM generations: a full
non-streamed vision call to describe the image, then a second agent call
with the description pasted in as text. The agent's llama-server model is
already vision-capable, so attach the image to the user message as an
OpenAI-style image_url content part and let the single agent call see it.
Roughly halves image-turn latency and streams from the start.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
analyze_user_image now runs input bytes through Pillow before base64-
encoding, converting HEIC/HEIF and any other non-JPEG format to JPEG.
pillow_heif is already registered at module load so HEIC opens cleanly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When SendBlue delivers a message with a media_url, download the image
and pass it through analyze_user_image so the agent receives a text
description alongside any caption. Image-only messages (no caption) are
now accepted; previously they were rejected as missing content.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The OpenAI client defaults to a 600s read timeout, so a wedged
llama-server blocked requests for 10 minutes and never raised, which
meant with_fallbacks() to OpenAI never triggered. Set a per-chunk read
timeout (120s, configurable via LLAMA_TIMEOUT_SECONDS) plus one retry
on the ChatOpenAI and LLMClient clients. Context is preserved because
conversation history is rebuilt from Postgres on every request.
Split the raggr service into raggr-blue (8080) and raggr-green (8081)
behind compose profiles, add a /api/health endpoint and container
healthcheck, and add scripts/deploy.sh to build the inactive color,
wait for it to become healthy, then gracefully stop the old one.
Makefile targets resolve the active color automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bump Conversation.updated_at on every message (not just the first) so
actively-used conversations rise to the top of the list endpoint's
existing -updated_at sort. Refresh the sidebar after every send so the
just-used conversation reorders live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first message in a new conversation navigated to /c/:id, which remounted
ChatScreen (two separate routes) and aborted the in-flight /stream-query fetch.
Server-side that surfaced as asyncio.CancelledError, which isn't an Exception,
so the except branch missed it and skipped the add_message_to_conversation
save — the fully-streamed answer was never persisted.
Backend: handle CancelledError and persist via a detached, shielded task so a
disconnect at any point (navigation, tab close, network blip) still commits the
answer. Frontend: serve "/" and "/c/:id" from a single catch-all ChatScreen
route so first-message navigation no longer remounts and aborts the stream;
derive the conversation id via useMatch instead of useParams.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The /stream-query endpoint previously emitted only tool-lifecycle events plus
one final complete message, so the answer still appeared all at once. Handle
on_chat_model_stream events to emit incremental `content` deltas, accumulate
them for persistence fallback, and consume them on the frontend to build a live
assistant bubble (reconciled with the authoritative final text). Hide the
redundant loading indicator once tokens start arriving.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rework mobile navigation into a proper left slide-in drawer with a dimmed
backdrop, sharing a new SidebarContent component with the desktop sidebar so
mobile users can reach Admin/Scheduler/Sign out. The sidebar now loads only the
10 most recent conversations; a dedicated /conversations page browses the full
list with server-side search and "Load more" pagination.
- Backend: limit/offset/search query params on GET /api/conversation/
- SidebarContent shared by desktop sidebar and mobile drawer
- useConversationBrowser hook powers the dedicated page
- Remove duplicated inline conversation lists from ChatScreen
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strip the card, border, shadow, and accent bar from AnswerBubble so
Simba's replies read like Claude/ChatGPT — plain full-width markdown on
the page. User messages stay bubbled for turn contrast. Widen the reading
column to max-w-3xl with roomier spacing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Give each conversation a deep-linkable URL (/c/:conversationId) so threads
survive refresh, support browser back/forward, and are shareable within a
session. The app was already an SPA but had no router.
- Add react-router-dom; wrap the app in BrowserRouter and define routes
("/" for the empty/new state, "/c/:conversationId" for a thread)
- Make the URL the source of truth for selection: selectedConversation is
derived from the :conversationId param; create/delete navigate instead of
mutating local selection state
- Load a thread's messages in a URL-keyed effect that clears immediately for
an instant switch, with a stale-response guard
- Preserve the optimistic message + in-flight stream when the first message
lazily creates a conversation and navigates into it (skip-ref keyed on the
new id, guarded against the null/home-route collision)
No backend changes: app.py already serves index.html for unmatched paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Frontend:
- Add stop-generation button (wire up the unused AbortController)
- Surface stream/network errors in a dismissible banner instead of console
- Add rename and delete for conversations via a per-thread menu
- Make simba mode a single controlled source; fix miscast keydown handler
- Auto-grow the message textarea; remove dead ConversationMenu component
Backend:
- Scope get/query/stream-query/rename/delete conversation routes to the
authenticated owner via a shared _get_owned_conversation helper (fixes IDOR)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enable remark-gfm in AnswerBubble so pipe-table syntax parses instead
of showing as raw text, and add table styling to markdown-content.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Upgrades pypetkitapi to 1.28.0, which adds the Eversweet Ultra (W7H
fountain with AI camera). The poller now extracts per-pet drinking
events (drink_over/pet_detect records) into a new drinking_events
table, plus Ultra-specific alerts for waste tank full and filter
expiry. Uncertain state fields (cwt/wt/pump) are logged for threshold
tuning against real data.
Exposes the data via a new /api/pets/drinking-events endpoint, a
Fountain Activity dashboard section, and a simbakit_drinking_activity
LangChain tool so the agent can answer hydration questions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires up the existing MealieService to the conversational agent with tools
for querying today's meals, weekly meal plans, full recipes, shopping lists,
and creating new meal plan entries. Adds env vars to docker-compose and
updates the system prompt so the agent knows when to use each tool.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds simbakit/ as a subfolder service that polls PetKit smart devices
(litter box, fountain, feeder) and exposes pet weight, litter activity,
and device health data via three new LangChain tools.
Also fixes stale alerts bug where cleared conditions (e.g. waste bin
full) persisted because the query used max alert time instead of the
device's last_seen_at timestamp.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Markdown-aware chunking (split on headers before size-based splitting)
- Prepend note filename to each chunk for self-contained context
- Source-filtered retrieval (obsidian/paperless queries stay isolated)
- MMR search with k=8, fetch_k=24 for better recall and diversity
- Add source metadata to Paperless docs and folder metadata to Obsidian docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When use_agent is enabled, the scheduler runs the message content as a
prompt through the LangChain agent and sends Simba's response instead of
the raw content. Frontend adds an Ask Simba toggle with visual indicator.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend scheduled messages with a recurrence field. After sending a
recurring message, the scheduler automatically creates the next pending
occurrence. Frontend adds repeat toggle (Once/Daily/Weekly/Monthly) and
displays recurrence in the messages table.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Strip markdown formatting (bold, italic, headers, code, links, lists) from
LLM responses before sending via iMessage. Add scheduled messages feature
with CRUD API, background scheduler loop, and admin frontend panel.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Revert get_conversation_for_user to use Conversation.get() with
MultipleObjectsReturned fallback. Add channel field to Conversation
model and get_conversation_for_channel helper so each messaging
channel gets its own isolated conversation per user.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Validates sb-signing-secret header against SENDBLUE_WEBHOOK_SECRET env var.
Can be disabled with SENDBLUE_SIGNATURE_VALIDATION=false for development.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ryan Chen