Commit Graph

379 Commits

Author SHA1 Message Date
Ryan Chen c24a44618f Demote friend ranking 405s to debug — private profiles are expected
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 22:13:02 -04:00
Ryan Chen c183a13db2 Add migration for beli_category column
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 22:09:46 -04:00
Ryan Chen c682d2b320 Add beli_category (RES/DES/BAR/BAK) to restaurant records
Store Beli's category code on each ranking so the agent can filter by type
(restaurant/bar/cafe/bakery). Category label shown in tool output.

Requires migration for beli_category column + updated unique constraint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 22:09:09 -04:00
Ryan Chen aa61ec54b9 Add migration for Beli friend ranking columns
Adds friend_beli_user_id (default '') and friend_name to beli_restaurants,
updates unique constraint to include friend_beli_user_id.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:55:10 -04:00
Ryan Chen b79229a5b0 Add random friend ranking refresh to Beli integration
Each nightly profile refresh now also picks FRIEND_SAMPLE_SIZE (5) random
followed users and fetches their rankings, building up a picture of friends'
lists over time. Rows are tagged with friend_beli_user_id/friend_name so the
agent tool can surface attribution ("via Alex").

Requires migration to add friend_beli_user_id/friend_name columns and update
unique_together on beli_restaurants.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:52:14 -04:00
Ryan Chen 7d1a89475a Fix Beli API client against actual spec
- Category codes: RES/DES/BAR/BAK (was full English words)
- score field: use 'value' from Score model (was 'score', which doesn't exist)
- rank_index: pass array position (no rank field in response)
- fetch_bookmarks: no query params (user param caused 405)
- notes: correctly None (stored separately, not in ranking results)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:23:52 -04:00
Ryan Chen 69b8697389 Fix Beli category codes: use RES/DES/BAR/BAK not full words
The API uses 3-letter codes, not full English words. Remove debug logging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:21:14 -04:00
Ryan Chen 0367dac3cd Log full JWT claims on auth and refresh to find correct user UUID
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:15:04 -04:00
Ryan Chen 828fc7d16a Log full Beli ranking response for debugging
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:12:25 -04:00
Ryan Chen fcadec8bf6 Log raw response keys from Beli ranking endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:09:56 -04:00
Ryan Chen 2674a49574 Add debug logging to Beli refresh to inspect API response shape
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:06:13 -04:00
Ryan Chen 4c55168d52 Make Beli bookmark fetch non-fatal, pass user_id param
Bookmarks returning 400 was aborting the whole refresh and discarding
successfully-fetched ranking data. Now bookmarks log a warning and
continue. Also pass user= to the bookmark endpoint in case it's required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:02:13 -04:00
Ryan Chen 863ebe8e20 Fix EncryptedTextField crashing on plaintext values during model creation
Tortoise calls to_python_value() when setting field values at create() time,
not only when loading from DB. Add _is_fernet_token() to distinguish encrypted
DB values (≥73 decoded bytes, version 0x80) from plaintext being set — the
latter passes through untouched while wrong-key errors on real tokens still raise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 19:54:20 -04:00
Ryan Chen b6b866c9fc manage_beli: use uv run and username instead of email
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 19:43:15 -04:00
Ryan Chen f709528def Add manage_beli.py management script
CLI to add/list/refresh/remove Beli profiles without a frontend.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 19:42:05 -04:00
Ryan Chen d23273590f Add Beli restaurant recommendation integration
Multi-tenant system to fetch and store restaurant lists from Beli accounts.
Each user can connect up to 3 Beli profiles; a nightly Cronicle job (1–5 AM,
randomised) refreshes the 3 stalest profiles using the reverse-engineered Beli
API (mirrored at git.torrtle.co/ryan/beli-api). Rankings (been) and bookmarks
(want-to-try) are stored in Postgres and queryable via a new LangChain agent
tool. Beli passwords and refresh tokens are stored Fernet-encrypted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 19:36:16 -04:00
Ryan Chen 53bbea42aa Use fresh UUID thread_id per agent run in scheduler and cron
Reusing the message/job ID as thread_id causes corrupted checkpoint state
when a previous run failed mid-tool-call. A fresh UUID each time guarantees
a clean conversation context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:43:57 -04:00
Ryan Chen d806416f93 Round up scheduled_at to next minute for Cronicle one-shot timing
If scheduled_at has sub-minute seconds (e.g. retry = now+30s), the
Cronicle event would fire at the start of that minute before the message
is due and find nothing. Round up so it always fires after the due time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:42:31 -04:00
Ryan Chen c6419f8931 Increase Cronicle trigger timeout to 5 min to allow LLM agent to complete
The shell script's Node.js socket timeout was 10s, too short for agent
runs with use_agent=true. Bump to 300s (5min) and set Cronicle job
timeout to 360s to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:38:58 -04:00
Ryan Chen 790a7059ce Per-message Cronicle events with recurrence-aware timing
Instead of an every-minute poller, each scheduled message gets its own
Cronicle event created at schedule time:
- One-time: year/month/day/hour/minute one-shot
- Daily/weekly/monthly: recurring timing matching the recurrence setting

Retry (PUT with new scheduled_at on failed messages) also creates a new
Cronicle event and resets status to pending. Recurring messages reuse
their existing Cronicle event across occurrences.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:32:22 -04:00
Ryan Chen 171410cf26 Fix agent invocation missing thread_id in scheduler and cron trigger
The checkpointer requires a thread_id in configurable. Use stable IDs
derived from the message/job ID so each has its own checkpoint thread.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:26:20 -04:00
Ryan Chen 504753eef9 Use Node.js script for Cronicle trigger to avoid shell dependency
Switches from curl (shell not available in Cronicle container) to a Node.js
inline script using the built-in http module. Tries raggr-green first, falls
back to raggr-blue automatically — no manual update needed on color swap.

Delete the existing broken Cronicle event before redeploying.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:21:09 -04:00
Ryan Chen 15a03dba91 Fix Cronicle event: use HTTP plugin (no shell needed) and every-minute timing
- Switch shellplug → urlplug to avoid spawn errors in shell-less containers
- Explicitly set timing to all 60 minutes (Cronicle {} defaults to hourly :00)
- Restore RAGGR_INTERNAL_URL env var for the HTTP plugin target URL

Delete the existing broken Cronicle event before redeploying so it gets recreated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:19:04 -04:00
Ryan Chen c9c75b6be3 Fix Cronicle event creation: fetch real category ID instead of hardcoded c_general
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:13:15 -04:00
Ryan Chen 0e80e8b9df Try both raggr-green and raggr-blue in Cronicle scheduled messages script
Shell script falls back to the other color if the primary is down,
so the trigger works regardless of which container is currently active.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:10:44 -04:00
Ryan Chen 6e72d5f95a Auto-register scheduled messages event in Cronicle on startup
On startup raggr calls the Cronicle API (CRONICLE_URL + CRONICLE_API_KEY)
and creates a 'Scheduled Messages Trigger' shell event if it doesn't exist,
replacing the need to configure it manually in the Cronicle UI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 08:09:27 -04:00
Ryan Chen 35b0e74a41 Migrate scheduled messages from internal poll loop to Cronicle trigger
Replaces the asyncio background loop (15s polling) with a POST
/api/scheduled-messages/trigger endpoint authenticated by X-Cron-Secret,
matching the existing cron jobs blueprint pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 07:55:31 -04:00
Ryan Chen 2542fa2342 Fix: capture main_agent before SSE generator to avoid lost app context
Quart exits the request/app context before iterating the SSE response
body, so current_app inside event_generator() raised "Not within an app
context". Capture the reference before the generator is defined. Also
add return after the exception error yield to prevent falling through to
a second "No response generated" event on the same stream.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 07:43:52 -04:00
Ryan Chen e36bf35b2a Add ynab_category_allocations tool and fix category spending cap
Add get_category_allocations() to YNABService that fetches all categories
grouped by category group (using the categories API for group structure),
showing assigned and available amounts for every non-hidden category.

Add the corresponding ynab_category_allocations agent tool so the LLM
can answer questions about per-category budgets even when a category has
no spending activity this month.

Also remove the [:10] cap in ynab_category_spending so all spending
categories are returned — previously categories ranked below 10th by
activity (e.g. Coffee slush fund) were silently dropped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 23:19:17 -04:00
Ryan Chen 19aa2b17d7 Fix _build_prompt receiving state dict instead of messages list
LangGraph passes the full agent state dict {"messages": [...]} to the
prompt callable, not just the messages list. Iterating a dict yields
its keys, so "messages" (the string) was getting appended to the LLM
input — causing the agent to respond as if the literal word "messages"
was the user's query.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 23:10:22 -04:00
Ryan Chen 43e0de24d6 Restore dynamic system prompt via RunnableLambda prompt parameter
Uses prompt=_build_prompt (a RunnableLambda) so LangGraph passes the
runtime config through, allowing per-request system_prompt injection.
Restores user memory loading for web and iMessage plain-text enforcement.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 23:04:43 -04:00
Ryan Chen 2a81ea8f76 Fix: migrate all remaining main_agent imports to current_app
email, scheduled_messages scheduler, and cron blueprints still imported
main_agent directly from agents. All now use current_app.main_agent so
they read the initialized value set by build_agent() in the lifespan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:59:13 -04:00
Ryan Chen 810d32dcc3 Fix: store agent on app object, access via current_app
Global module variable was unreliable across import boundaries. Storing
the compiled agent on app.main_agent and reading it via current_app is
the idiomatic Quart pattern and guaranteed to reflect the initialized value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:55:39 -04:00
Ryan Chen 6621a99149 Fix: access main_agent via module reference to avoid None at import time
from-import captures the None value set at module load; importing the module
and accessing .main_agent at call time gets the value set by build_agent().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:51:18 -04:00
Ryan Chen 801a90746b Fix: replace deprecated state_modifier with prompt in create_react_agent
state_modifier was removed in this version of LangGraph; use the prompt
parameter with a static system prompt string instead. Dynamic memory
injection via configurable is deferred — memories are still saved by the
save_user_memory tool, just not pre-loaded into the system prompt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:43:39 -04:00
Ryan Chen 202baa17e2 Fix: use langgraph.prebuilt.create_react_agent for state_modifier support
langchain.agents.create_agent does not accept state_modifier or checkpointer;
those are LangGraph-only params available on create_react_agent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:37:11 -04:00
Ryan Chen f1ab1accaf Add LangGraph SQLite checkpointing and cron_jobs migration
Replaces per-request history reconstruction with a persistent AsyncSqliteSaver
checkpointer keyed by conversation ID, so agent state (tool calls, intermediate
steps) carries forward between turns without manual message list building.
Also includes the missing migration for the CronJob model added in bb4619a.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:34:04 -04:00
Ryan Chen bb4619aefe Add Cronicle-driven cron jobs with local migration workflow
Adds a new cron blueprint with UUID-keyed jobs that Cronicle triggers via
HTTP POST with a shared secret header. Also adds scripts/migrate-new.sh so
migrations can be generated locally against a Docker postgres when no prod
container is running.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 19:10:15 -04:00
Ryan Chen 82f16bfd6a Add Google Calendar and Gmail integration via Python SDK
Replaces the gws CLI subprocess approach with google-api-python-client.
Adds get_calendar_events, create_calendar_event, gmail_search_emails, and
gmail_get_email as LangChain tools; all admin-gated and auto-enabled when
token.json is present. Includes uv-runnable scripts/google_auth.py for
one-time OAuth2 setup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 08:43:18 -04:00
Ryan Chen 7912c83ca3 Instruct LLM to avoid markdown formatting in iMessage responses
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-15 23:24:43 -04:00
Ryan Chen bc19f52052 Fix iMessage webhook timeout: ack SendBlue immediately, process in background
Move LLM invocation into a background asyncio task so the webhook returns 200
before SendBlue's timeout triggers retries. Validation, conversation setup, and
user message persistence still happen synchronously before the ack.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-15 23:21:20 -04:00
ryan 0122e018ac Add migration: message_handle on conversation_messages 2026-09-15 23:15:30 -04:00
Ryan Chen 7474055c97 Add iMessage threading, typing indicators, and fix tool-use response extraction
- Native reply threading: inbound replies continue their conversation via
  SendBlue message_handle lookup; standalone messages create a new conversation
- Our responses are sent with reply_to so they thread visually in iMessage
- Typing indicator fires before agent invocation and stops in a finally block
- Add extract_agent_text() to handle list-type AIMessage content and strip
  <think> tokens from Qwen3 — applied to iMessage, WhatsApp, and web chat
- ConversationMessage gains a nullable message_handle field (migration needed)
- /threads command lists recent iMessage conversations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-15 23:10:50 -04:00
Ryan Chen 0478ee7568 Add Prometheus metrics endpoint for chat response observability
Tracks end-to-end response time and request counts (by stream mode and
result) via prometheus_client. Exposes GET /metrics behind a bearer
token (METRICS_TOKEN env var) so Prometheus can scrape it without
exposing raw data publicly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-15 22:57:12 -04:00
Ryan Chen 46a4b874bd Fix iMessage agent: pass user config and load memories
The iMessage webhook called main_agent.ainvoke without a config, so
save_user_memory always raised KeyError('user_id') when invoked. The
system prompt instructs the agent to save memories proactively, causing
repeated tool failures and ultimately returning the error fallback to
the user on most messages.

Fix:
- Pass agent_config with user_id/is_admin to ainvoke, matching the web
  endpoint.
- Extract _build_system_prompt_with_memories out of the conversation
  blueprint __init__ into memory.py (public name) so both the web
  endpoint and iMessage share the same implementation, giving iMessage
  users access to their saved memories.
- Remove the leftover debug print() in add_message_to_conversation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-15 22:57:04 -04:00
Ryan Chen 9777eaf9e1 Fix ThinkingBubble position: render above streaming answer not below
The ThinkingBubble was rendering after the messages list, so it appeared
below the partial answer while streaming. Inject it inside the messages
map just before the last streaming simba message so it sits above the
answer text. Keep the post-list fallback for when thinking is active
but no content has arrived yet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-12 16:43:48 -04:00
Ryan Chen bb2103e81e Stream qwen3 thinking tokens with collapsible UI panel
Route <think>…</think> content from the llama-server stream into
thinking_delta SSE events instead of discarding it. A small look-ahead
buffer in _route_content() handles tags split across chunk boundaries.
Strip thinking tags from the persisted final_message so the DB stores
only the actual response.

Frontend: ThinkingBubble component accumulates the live reasoning text
in a scrollable panel that auto-collapses to "Thought for Xs" once the
first content token arrives. Falls back to the existing heartbeat
behaviour for models that don't emit <think> tags.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-12 16:26:25 -04:00
Ryan Chen 2a1f125fa5 Temporary debug: log chunk.additional_kwargs during think phase
Log the first non-empty additional_kwargs seen during the pre-content
thinking window to confirm whether llama-server surfaces reasoning_content
for qwen3. Will be removed after verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-12 16:14:35 -04:00
Ryan Chen 8fd6fa9ece Add TTFT watchdog to detect and surface LLM stalls
When llama-server is queue-backed it accepts the connection but never
produces tokens; after ~60s Caddy drops the connection and the frontend
clears loading state with no response and no error shown to the user.

Backend: replace the async-for event loop with an explicit __anext__()
iterator wrapped in asyncio.wait_for(). A 30s timeout (TTFT_TIMEOUT_SECONDS)
applies only in the window between on_chat_model_start and the first content
token — tool calls are excluded. On timeout, emits a stalled error event and
logs it rather than waiting for the proxy to kill the connection.

Frontend: separates stall errors from generic errors; shows a "Simba didn't
respond in time. Retry" pill that re-sends the original query.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-12 16:10:44 -04:00
ryan 3875ed4e4f Merge pull request 'Add structured timing logs to stream-query for slow-request diagnosis' (#51) from stream-observability into main
Reviewed-on: #51
2026-09-12 15:48:32 -04:00