simbarag/docs/chat-runs.md

# Reconnectable chat runs

A generation belongs to a durable, authenticated run rather than an HTTP connection. Closing a tab, navigating away, or losing the network detaches its subscriber. Explicit Stop sends cancellation to the run. Generation calls the authoritative `runtime.prepare_agent_turn` and `runtime.stream_prepared` under its own application context, without retaining the request context.

## Wiring

- `blueprints.conversation.run_models` is registered in `config/db.py`; the tables come from migration `20_20261012231017_application_foundation.py`.
- `app.py` registers `runs_blueprint`, calls `expire_runs()` after ORM start-up (only already-expired leases; another deployment may still own live runs) and `drain_runs(app)` right after the serving lifecycle's `yield`, while the agent and checkpointer are still open.
- `python app.py` serves through Hypercorn with `shutdown_timeout = CHAT_RUN_DRAIN_SECONDS + 30`, so lifespan shutdown outlasts the drain. `scripts/deploy.sh` gives the old container `CHAT_RUN_DRAIN_SECONDS + 60` (override with `DEPLOY_STOP_SECONDS`).
- Every writer to a conversation checkpoint is serialized: `/runs` and `/stream-query` through `ChatRun.active_key` (cross-process), `/query` returns 409 while a run is active, and all channels share `runtime.thread_lock` (in-process).
- Streamed deltas are coalesced (`FLUSH_SECONDS`, 150 ms) before they are written, and same-process subscribers are woken on each write; other processes fall back to polling every `POLL_SECONDS`.

## API

All endpoints require the existing JWT authentication. Run creation, discovery, replay and cancellation check both owner and conversation. Attachment keys must be under the authorized conversation's storage prefix.

- `POST /api/conversation/runs`: `{conversation_id, query, request_id, image_key?, file_key?}`. Returns a run (`201` new, `200` idempotent replay). Clients must retain the same request ID for ambiguous retries. A reused ID with different input returns 409; another active run in the conversation returns a clear 409 before saving another user message or starting generation. Legacy callers may omit request ID and receive a fresh one.
- `GET /api/conversation/{conversation_id}/runs`: `{run: ... | null}`. Returns the latest run, including terminal status, so loading a conversation can discover active generation or an interrupted attempt. Includes original query and image key for the brief interval before the user message is saved.
- `GET /api/conversation/{conversation_id}/runs/{run_id}/events?cursor=N`: ordered SSE replay followed by live database polling. Only events with sequence greater than N are returned. `Last-Event-ID` is also supported when no query cursor is supplied. Negative, noninteger and ahead-of-log cursors return 400. Event `id:` is the durable sequence; event payload types remain `thinking`, `thinking_delta`, `tool_start`, `tool_end`, `content`, `response`, `error`, and terminal `[DONE]`. Comment heartbeats do not advance the cursor.
- `POST /api/conversation/{conversation_id}/runs/{run_id}/cancel`: requests cancellation; it is idempotent. The database flag reaches the owning process through its heartbeat, normally within one second. Completion may win a race with cancellation. Keep reading until the terminal event.
- `POST /api/conversation/stream-query`: compatibility adapter creating a run and attaching at cursor zero. Existing event types and `[DONE]` remain; `X-Chat-Run-ID` identifies the run. A disconnect detaches this endpoint too.

Run statuses are `running`, `completed`, `failed`, `cancelled`, and `interrupted`. Terminal errors precede `[DONE]`. A model stall retains its existing `stalled: true` event and metrics. Transient subscriber failures are recoverable and are distinct from generation failure.

## Storage and lifecycle

`chat_runs` stores the authenticated owner, input, idempotency key, claim token, lease, cancellation flag, partial text and persistence markers. Its nullable unique `active_key` holds a conversation until terminalization. A composite uniqueness constraint makes duplicate creation safe across processes. `chat_run_events` has unique `(run_id, sequence)` ordering. Database transactions lock the run before appending events, finalizing or saving messages; answer and user persistence markers are committed with their messages. Tool messages/artifacts and the existing streaming metrics/timeouts stay on the established streaming path.

PostgreSQL is the deployment store; real file-backed SQLite exercises the same ORM lifecycle in tests. Subscribers can land on either deployment process because replay reads shared storage. The owning process renews its lease, checks cross-process cancellation, and cancels itself if renewal fails. Expired runs save available partial text once, release admission and emit an interrupted error. A process restart **does not restart generation**. Prior persisted events and partial answers remain available; the user starts a new turn explicitly. Lease expiry assumes failed renewal promptly stops the worker; it is not fencing for independent checkpoint writers.

Defaults, read from environment at module load:

| Setting | Default | Purpose |
| --- | --- | --- |
| `CHAT_RUN_LEASE_SECONDS` | 30 | Abandoned-worker lease; minimum 10 seconds |
| `CHAT_RUN_MAX_SECONDS` | 1800 | Entire generation, including preparation and tools |
| `CHAT_RUN_DRAIN_SECONDS` | 1800 | Wait for tracked generations before interrupting on shutdown |
| `DEPLOY_STOP_SECONDS` | drain + 60 | Deployment container stop grace |
| `TTFT_TIMEOUT_SECONDS` | existing 30 | Existing per-model silent-output timeout |

Subscribers have finite lifetimes. The browser retries with its last successfully handled cursor, bounds retries after consecutive transport failures, and bounds silent reads. A fresh conversation load rebuilds the active turn from cursor zero after excluding saved tool/assistant rows from that turn, avoiding double-rendered output. Unmount and route switching abort only the reader. Stop uses the cancellation endpoint, including when pressed during creation.

## Validation

```sh
.venv/bin/python -m pytest tests/unit/test_chat_runs.py tests/unit/test_agent_runtime.py -q
cd raggr-frontend
node --test tests/runs*.test.cjs
npx tsc --noEmit
```

Tests cover real SQLite creation/claim contention, owner/conversation isolation and JWT endpoints, replay order/cursor, once-only answer persistence, request idempotency, application context, disconnect versus cancellation, file database close/reopen and lease expiry, drain, the actual shared runtime and legacy endpoint, SSE framing/duplicate suppression, authenticated discovery/cancel and cursor retry.

The stall/metrics tests run through the durable worker: persistence happens in `runs.py`, and request cancellation detaches the subscriber rather than stopping generation.