23 Commits

Author SHA1 Message Date
ryan 9034a5bcec Merge pull request 'Create scheduled_messages for databases that never had it' (#80) from fix/scheduled-messages-migration into main
CI / backend (push) Successful in 42s
CI / frontend (push) Failing after 7s
2026-10-13 07:52:55 -04:00
Ryan Chen 1cd387c59f Create scheduled_messages for databases that never had it
No committed migration creates the table: migrations 6-10 were generated
outside the repository, so fresh databases fail once anything alters it.
The new migration is CREATE TABLE IF NOT EXISTS, a no-op where the table
already exists. Aerich cannot generate it because its saved state already
includes the table; the DDL is Tortoise's schema output for the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 23:17:06 -04:00
Ryan Chen 5809ad3adb Add durable chat runs, shared agent runtime, reliable jobs, history tools and citations
CI / backend (pull_request) Successful in 42s
CI / frontend (pull_request) Failing after 7s
CI / backend (push) Successful in 41s
CI / frontend (push) Failing after 7s
Shared runtime
- One agent entry point (runtime.invoke_agent_for_user / prepare_agent_turn) for
  web, email, iMessage, WhatsApp, scheduled and cron, so identity, permissions,
  memories, summaries and metrics match across channels.
- Serialize turns per checkpoint thread; /query validates input and returns 409
  while a durable run owns the conversation.

Reconnectable chat (durable runs)
- POST /runs creates an idempotent, owner-scoped run; GET .../events replays an
  ordered SSE log by cursor, so generation survives refresh and device switches.
  Stop cancels; disconnect only detaches. /stream-query is a compatibility adapter.
- Coalesce streamed deltas (150 ms) and wake same-process subscribers instead of
  polling; keep buffered text when a run is stopped.
- Drain runs on shutdown; serve via Hypercorn with a shutdown timeout covering
  the drain (and no longer in debug mode). CHAT_RUN_* settings are forwarded.

Scheduled messages and cron
- Atomic delivery claims, attempt history and explicit retries; ambiguous sends
  become "uncertain" and are never resent automatically.
- Cron triggers need a run_id/Idempotency-Key; legacy jobs need an explicit owner.
- Cancelling never rewrites a started delivery; series stop via tombstones.
  Legacy sent rows are not forked into new series. Malformed bodies return 400.

History, integrations and retrieval
- Message-content search, pinned conversations, JSON/Markdown export and a
  memory editor.
- Integration status panel and YNAB data-freshness provenance.
- Date-filtered document/note search with stable citations and a source viewer.

Migration 20 is generated by aerich.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 23:11:04 -04:00
Ryan Chen 746aaa4844 Index conversation, message, memory and artifact foreign keys
Postgres had only primary keys on these tables, so loading a
conversation, listing conversations, checking for a first message on
every send, and fetching memories/artifacts were sequential scans.
Migration generated with aerich; index-only apart from the no-op JSONB
re-casts aerich emits on every run (as in migrations 16 and 18), which
were checked not to rewrite tables. Upgrade and downgrade verified on a
throwaway pgvector database.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 22:57:35 -04:00
Ryan Chen 0fa30edb82 Merge origin/main into feat/ynab-mirror
Resolve config/db.py (artifacts + ynab models) and regenerate the YNAB
migration with aerich as 18 on top of main's 17_add_artifacts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:07:26 -04:00
Ryan Chen 10f3f1bcab Merge origin/main into feat/ynab-mirror
Resolve the Makefile .PHONY conflict (desktop + ynab targets) and
regenerate the YNAB migration with aerich as 17 on top of main's
16_add_oidc_states, replacing the colliding 16_add_ynab_mirror.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:06:47 -04:00
Ryan Chen 1cdc380870 Add artifacts table migration
Generated with aerich against a fresh Postgres after applying 1-16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:56:26 -04:00
Ryan Chen bc3e3d2797 Accept access tokens on protected routes; persist OIDC state
- Add transitional @jwt_access_required (accepts access tokens, and for now
  also refresh tokens) and use it on every protected route and inside
  admin_required. /api/user/refresh stays refresh-only.
- Move pending OIDC state/PKCE verifiers from an in-process dict to an
  oidc_states table (10 min TTL, single-use) so logins survive a blue/green
  flip. Drop the unused redirect_after_login.
- Unit tests for the decorator and the state store.

Deploy this commit on its own, before the access-token frontend: the old
backend answers access tokens with 422, so the new frontend only works once
this backend is live on both colors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:44:34 -04:00
Ryan Chen 8f02318542 Mirror YNAB budget into Postgres with delta sync and change log
YNAB has no event feed, so keep a local copy in ynab_* tables synced via
server_knowledge delta requests: full download on first run, deltas every
YNAB_SYNC_INTERVAL seconds and on demand before each tool call, full
resync daily. Each sync diffs against the mirror and appends real edits
(transactions, assignments, renames) to ynab_changes, the edit history
YNAB doesn't keep.

Agent YNAB tools now answer from the mirror: budget summary (spending
computed from transactions, since YNAB's month activity nets in credit
card payment categories), account balances, transaction search with
splits expanded, spending reports by category/payee/account/month,
category budget, and recent changes. Adds an admin-only read-only
ynab_sql tool guarded by a sqlglot allowlist (scope-resolved tables, no
dangerous functions), a READ ONLY transaction, a timeout, and a row cap.

Replaces utils/ynab_service.py and drops the unused ynab SDK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:32:38 -04:00
Ryan Chen c183a13db2 Add migration for beli_category column
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 22:09:46 -04:00
Ryan Chen aa61ec54b9 Add migration for Beli friend ranking columns
Adds friend_beli_user_id (default '') and friend_name to beli_restaurants,
updates unique constraint to include friend_beli_user_id.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 20:55:10 -04:00
Ryan Chen d23273590f Add Beli restaurant recommendation integration
Multi-tenant system to fetch and store restaurant lists from Beli accounts.
Each user can connect up to 3 Beli profiles; a nightly Cronicle job (1–5 AM,
randomised) refreshes the 3 stalest profiles using the reverse-engineered Beli
API (mirrored at git.torrtle.co/ryan/beli-api). Rankings (been) and bookmarks
(want-to-try) are stored in Postgres and queryable via a new LangChain agent
tool. Beli passwords and refresh tokens are stored Fernet-encrypted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 19:36:16 -04:00
Ryan Chen f1ab1accaf Add LangGraph SQLite checkpointing and cron_jobs migration
Replaces per-request history reconstruction with a persistent AsyncSqliteSaver
checkpointer keyed by conversation ID, so agent state (tool calls, intermediate
steps) carries forward between turns without manual message list building.
Also includes the missing migration for the CronJob model added in bb4619a.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:34:04 -04:00
ryan 0122e018ac Add migration: message_handle on conversation_messages 2026-09-15 23:15:30 -04:00
Ryan Chen c95800e65d Add user memory feature for cross-conversation recall
Give the LangChain agent a save_user_memory tool so users can ask it to
remember preferences and personal facts. Memories are stored per-user in
a new user_memories table and injected into the system prompt on each
conversation turn.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 21:53:14 -04:00
Ryan Chen 18ef611134 Merge origin/main: resolve conflicts keeping both email/Mealie and WhatsApp/Mailgun/Obsidian work
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 08:19:50 -04:00
Ryan Chen 0415610d64 Add image upload and vision analysis to Ask Simba chat
Users can now attach images in the web chat for Simba to analyze using
Ollama's gemma3 vision model. Images are stored in Garage (S3-compatible)
and displayed in chat history.

Also fixes aerich migration config by extracting TORTOISE_CONFIG into a
standalone config/db.py module, removing the stale aerich_config.py, and
adding missing MODELS_STATE to migration 3.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 08:03:19 -04:00
ryan 6a7b1369ad Add email channel via Mailgun for Ask Simba
Users can now receive a unique email address (ask+<token>@domain) and
interact with Simba by sending emails. Inbound emails hit a Mailgun
webhook, are authenticated via HMAC token lookup, processed through the
LangChain agent, and replied to via the Mailgun API.

- Extract shared SIMBA_SYSTEM_PROMPT to blueprints/conversation/prompts.py
- Add email_enabled and email_hmac_token fields to User model
- Create blueprints/email with webhook, signature validation, rate limiting
- Add admin endpoints to enable/disable email per user
- Update AdminPanel with Email column, toggle, and copy-address button
- Add Mailgun env vars to .env.example
- Include database migration for new fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-13 16:21:18 -04:00
ryan 86cc269b3a yeet 2026-03-03 08:23:31 -05:00
Ryan Chen 43dd05f9d5 chore(01-01): add FERNET_KEY config and email tables migration
- Add FERNET_KEY to .env.example with generation instructions
- Register email.models in aerich_config.py and app.py
- Register email blueprint in app.py
- Create database migration for email_accounts, email_sync_status, emails
- Migration includes proper foreign keys and indexes
2026-02-08 09:15:23 -05:00
Ryan Chen ad39904dda reorganization 2026-01-31 17:13:27 -05:00
Ryan Chen 70799ffb7d refactor 2025-11-10 15:51:13 -05:00
Ryan Chen 29ac724d50 enabling login btw users 2025-10-25 09:30:54 -04:00