Durable chat runs, shared agent runtime, reliable jobs, history tools and citations #79

Merged

ryan merged 1 commits from feat/application-foundation into main 2026-10-12 23:27:29 -04:00

Conversation 0 Commits 1 Files Changed 68

+7751 -1981

ryan commented 2026-10-12 23:11:35 -04:00

Owner

Copy Link

Copy Source

Priorities 2–7 of the application-foundation batch. Stacked on #78: merge that first; this PR's own change is the top commit (5809ad3).

What's in it

• Shared agent runtime (3): one entry point for web, email, iMessage, WhatsApp, scheduled and cron (same identity, permissions, memories, summaries, metrics). Turns on one checkpoint thread are serialized; /query validates input and returns 409 while a durable run owns the conversation.
• Reconnectable chat (5): idempotent, owner-scoped durable runs with an ordered, cursor-replayable SSE log, so a response survives refresh or a device switch. Stop cancels; disconnect only detaches. /stream-query stays as a compatibility adapter. Deltas are coalesced (150 ms) and same-process subscribers are woken instead of polling. Runs drain on shutdown, and python app.py now serves via Hypercorn with a matching shutdown timeout (it was app.run(debug=True)). See docs/chat-runs.md.
• Scheduled jobs and cron (2): atomic delivery claims, attempt history, explicit retry. Ambiguous sends become uncertain and are never resent automatically. Cron triggers need a run ID or Idempotency-Key; legacy cron jobs need an explicit owner. Cancelling never rewrites a started delivery. See docs/job-delivery.md.
• Integration status (4): status panel, plus YNAB data-freshness provenance in tool output.
• Retrieval and citations (6): date-filtered paperless and Obsidian search, stable /sources/<id> citations, source viewer.
• History (7): message-content search, pinned conversations, JSON/Markdown export, memory editor.

Migration

20_20261012231017_application_foundation.py was generated by aerich and applied cleanly to a scratch Postgres on top of main's migration 19.

⚠️ No migration ever creates scheduled_messages: that's an existing gap on main, and prod has the table from outside aerich. A fresh database fails at migration 20's ALTER TABLE scheduled_messages. Existing deployments are unaffected.

Review

Every workstream had an implementation review plus an independent second review. The findings were fixed with regression tests: thread races, payload 500s, legacy recurrence forking, cancel overwriting delivered rows, series revival on delete, lost buffered text on stop, a delayed flush, subscriber-map growth, a retrieval post-filter mismatch, and null-field 500s.

Validation: 542 unit tests (repeated runs, no order dependence), Ruff, 3/3 context evals, 20 frontend tests, tsc, frontend build.

🤖 Generated with Claude Code

Priorities 2–7 of the application-foundation batch. **Stacked on #78**: merge that first; this PR's own change is the top commit (`5809ad3`). ## What's in it - **Shared agent runtime (3):** one entry point for web, email, iMessage, WhatsApp, scheduled and cron (same identity, permissions, memories, summaries, metrics). Turns on one checkpoint thread are serialized; `/query` validates input and returns 409 while a durable run owns the conversation. - **Reconnectable chat (5):** idempotent, owner-scoped durable runs with an ordered, cursor-replayable SSE log, so a response survives refresh or a device switch. Stop cancels; disconnect only detaches. `/stream-query` stays as a compatibility adapter. Deltas are coalesced (150 ms) and same-process subscribers are woken instead of polling. Runs drain on shutdown, and `python app.py` now serves via Hypercorn with a matching shutdown timeout (it was `app.run(debug=True)`). See `docs/chat-runs.md`. - **Scheduled jobs and cron (2):** atomic delivery claims, attempt history, explicit retry. Ambiguous sends become `uncertain` and are never resent automatically. Cron triggers need a run ID or `Idempotency-Key`; legacy cron jobs need an explicit owner. Cancelling never rewrites a started delivery. See `docs/job-delivery.md`. - **Integration status (4):** status panel, plus YNAB data-freshness provenance in tool output. - **Retrieval and citations (6):** date-filtered paperless and Obsidian search, stable `/sources/<id>` citations, source viewer. - **History (7):** message-content search, pinned conversations, JSON/Markdown export, memory editor. ## Migration `20_20261012231017_application_foundation.py` was generated by aerich and applied cleanly to a scratch Postgres on top of main's migration 19. ⚠️ No migration ever creates `scheduled_messages`: that's an existing gap on main, and prod has the table from outside aerich. A fresh database fails at migration 20's `ALTER TABLE scheduled_messages`. Existing deployments are unaffected. ## Review Every workstream had an implementation review plus an independent second review. The findings were fixed with regression tests: thread races, payload 500s, legacy recurrence forking, cancel overwriting delivered rows, series revival on delete, lost buffered text on stop, a delayed flush, subscriber-map growth, a retrieval post-filter mismatch, and null-field 500s. Validation: 542 unit tests (repeated runs, no order dependence), Ruff, 3/3 context evals, 20 frontend tests, `tsc`, frontend build. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

ryan added 2 commits 2026-10-12 23:11:36 -04:00

Validate production config, add CI, and build from locked dependencies ... de88679f25

- Fail startup on unsafe or malformed production settings (scripts/validate_config.py),
  run it before migrations, and stop on a failed `aerich upgrade`.
- Forward the context-budget, timeout and metrics settings through Compose; accept
  the historical TAVILIY_API_KEY spelling.
- Build the image from uv.lock and yarn.lock in a multi-stage Dockerfile.
- Add Gitea Actions CI: Ruff, unit tests, context evaluation, frontend tests,
  typecheck and build.
- Update README and deployment/development docs to match the current stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Add durable chat runs, shared agent runtime, reliable jobs, history tools and citations ... 5809ad3adb

Shared runtime
- One agent entry point (runtime.invoke_agent_for_user / prepare_agent_turn) for
  web, email, iMessage, WhatsApp, scheduled and cron, so identity, permissions,
  memories, summaries and metrics match across channels.
- Serialize turns per checkpoint thread; /query validates input and returns 409
  while a durable run owns the conversation.

Reconnectable chat (durable runs)
- POST /runs creates an idempotent, owner-scoped run; GET .../events replays an
  ordered SSE log by cursor, so generation survives refresh and device switches.
  Stop cancels; disconnect only detaches. /stream-query is a compatibility adapter.
- Coalesce streamed deltas (150 ms) and wake same-process subscribers instead of
  polling; keep buffered text when a run is stopped.
- Drain runs on shutdown; serve via Hypercorn with a shutdown timeout covering
  the drain (and no longer in debug mode). CHAT_RUN_* settings are forwarded.

Scheduled messages and cron
- Atomic delivery claims, attempt history and explicit retries; ambiguous sends
  become "uncertain" and are never resent automatically.
- Cron triggers need a run_id/Idempotency-Key; legacy jobs need an explicit owner.
- Cancelling never rewrites a started delivery; series stop via tombstones.
  Legacy sent rows are not forked into new series. Malformed bodies return 400.

History, integrations and retrieval
- Message-content search, pinned conversations, JSON/Markdown export and a
  memory editor.
- Integration status panel and YNAB data-freshness provenance.
- Date-filtered document/note search with stable citations and a source viewer.

Migration 20 is generated by aerich.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ryan referenced this pull request 2026-10-12 23:17:07 -04:00

Create scheduled_messages for databases that never had it #80

ryan referenced this pull request 2026-10-12 23:18:42 -04:00

Only presign and accept images in the caller's own conversations #81

ryan merged commit 2a843c29f2 into main 2026-10-12 23:27:29 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

ryan

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ryan/simbarag#79