10 Commits

Author SHA1 Message Date
Ryan Chen 5809ad3adb Add durable chat runs, shared agent runtime, reliable jobs, history tools and citations
CI / backend (pull_request) Successful in 42s
CI / frontend (pull_request) Failing after 7s
CI / backend (push) Successful in 41s
CI / frontend (push) Failing after 7s
Shared runtime
- One agent entry point (runtime.invoke_agent_for_user / prepare_agent_turn) for
  web, email, iMessage, WhatsApp, scheduled and cron, so identity, permissions,
  memories, summaries and metrics match across channels.
- Serialize turns per checkpoint thread; /query validates input and returns 409
  while a durable run owns the conversation.

Reconnectable chat (durable runs)
- POST /runs creates an idempotent, owner-scoped run; GET .../events replays an
  ordered SSE log by cursor, so generation survives refresh and device switches.
  Stop cancels; disconnect only detaches. /stream-query is a compatibility adapter.
- Coalesce streamed deltas (150 ms) and wake same-process subscribers instead of
  polling; keep buffered text when a run is stopped.
- Drain runs on shutdown; serve via Hypercorn with a shutdown timeout covering
  the drain (and no longer in debug mode). CHAT_RUN_* settings are forwarded.

Scheduled messages and cron
- Atomic delivery claims, attempt history and explicit retries; ambiguous sends
  become "uncertain" and are never resent automatically.
- Cron triggers need a run_id/Idempotency-Key; legacy jobs need an explicit owner.
- Cancelling never rewrites a started delivery; series stop via tombstones.
  Legacy sent rows are not forked into new series. Malformed bodies return 400.

History, integrations and retrieval
- Message-content search, pinned conversations, JSON/Markdown export and a
  memory editor.
- Integration status panel and YNAB data-freshness provenance.
- Date-filtered document/note search with stable citations and a source viewer.

Migration 20 is generated by aerich.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 23:11:04 -04:00
Ryan Chen 863ebe8e20 Fix EncryptedTextField crashing on plaintext values during model creation
Tortoise calls to_python_value() when setting field values at create() time,
not only when loading from DB. Add _is_fernet_token() to distinguish encrypted
DB values (≥73 decoded bytes, version 0x80) from plaintext being set — the
latter passes through untouched while wrong-key errors on real tokens still raise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-17 19:54:20 -04:00
Ryan Chen 2a81ea8f76 Fix: migrate all remaining main_agent imports to current_app
email, scheduled_messages scheduler, and cron blueprints still imported
main_agent directly from agents. All now use current_app.main_agent so
they read the initialized value set by build_agent() in the lifespan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-09-16 22:59:13 -04:00
Ryan Chen ca37c4a780 Fix history roles and persist tool results for cross-turn context
Two fixes for the agent losing track of conversation context:

- History replay compared msg.speaker (a Speaker enum) to the string
  "user", which is always false, so every prior message — including the
  user's — was sent to the LLM with role "assistant". Compare against
  Speaker.USER instead. Affected web, iMessage, WhatsApp, and email.

- Tool outputs (web searches, RAG chunks) were discarded after each
  turn, so follow-up questions about fetched content had nothing to
  reference. Persist them as speaker="tool" messages (capped at 4000
  chars) and replay them in history as assistant context. The
  conversation GET endpoint serves only their header line so the UI
  tool pill stays compact.

Also bumps the replayed history window from 10 to 20 messages since
tool messages now share it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-12 15:14:33 -04:00
Ryan Chen 3ba93c55f4 Add channel-scoped conversations for iMessage, WhatsApp, and email
Revert get_conversation_for_user to use Conversation.get() with
MultipleObjectsReturned fallback. Add channel field to Conversation
model and get_conversation_for_channel helper so each messaging
channel gets its own isolated conversation per user.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:39:39 -04:00
Ryan Chen 18ef611134 Merge origin/main: resolve conflicts keeping both email/Mealie and WhatsApp/Mailgun/Obsidian work
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 08:19:50 -04:00
ryan 6a7b1369ad Add email channel via Mailgun for Ask Simba
Users can now receive a unique email address (ask+<token>@domain) and
interact with Simba by sending emails. Inbound emails hit a Mailgun
webhook, are authenticated via HMAC token lookup, processed through the
LangChain agent, and replied to via the Mailgun API.

- Extract shared SIMBA_SYSTEM_PROMPT to blueprints/conversation/prompts.py
- Add email_enabled and email_hmac_token fields to User model
- Create blueprints/email with webhook, signature validation, rate limiting
- Add admin endpoints to enable/disable email per user
- Update AdminPanel with Email column, toggle, and copy-address button
- Add Mailgun env vars to .env.example
- Include database migration for new fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-13 16:21:18 -04:00
Ryan Chen e4084276d8 feat(01-02): create email body parser for multipart MIME messages
- Implemented parse_email_body function for RFC822 email parsing
- Uses stdlib email.message_from_bytes with modern EmailMessage API
- Extracts text and HTML bodies using get_body() method
- Prefers plain text over HTML for "preferred" field
- Converts HTML to text using html2text when text body missing
- Extracts all metadata: subject, from, to, date, message_id
- Uses parsedate_to_datetime for proper date parsing
- Handles UnicodeDecodeError gracefully with partial data return
- Follows async patterns and logging conventions from existing codebase
2026-02-08 09:34:47 -05:00
Ryan Chen 6e4ee6c75e feat(01-02): implement IMAP connection service with authentication and folder listing
- Created IMAPService class with async connect/list_folders/close methods
- Uses aioimaplib for async IMAP4_SSL operations
- Implements proper connection cleanup with logout() not close()
- Added aioimaplib and html2text dependencies to pyproject.toml
- Follows async patterns from existing service classes (ynab_service.py, mealie_service.py)
- Includes comprehensive logging with [IMAP] and [IMAP ERROR] prefixes
2026-02-08 09:33:59 -05:00
Ryan Chen bee63d1c60 feat(01-01): create email blueprint with encrypted Tortoise ORM models
- Add EncryptedTextField for transparent Fernet encryption
- Create EmailAccount model with encrypted IMAP credentials
- Create EmailSyncStatus model for sync state tracking
- Create Email model with 30-day retention logic
- Follow existing blueprint patterns from users/conversation
2026-02-08 09:08:32 -05:00