simbarag/docs/deployment.md

Deployment & Migrations Guide

This document covers database migrations and deployment workflows for SimbaRAG.

Migration Workflow

Migrations are managed by Aerich, the migration tool for Tortoise ORM.

Key Principles

1. Generate migrations in Docker - Aerich needs database access to detect schema changes
2. Migrations auto-apply on startup - Both startup.sh and startup-dev.sh run aerich upgrade
3. Commit migrations to git - Migration files must be in the repo for production deploys

Generating a New Migration

Development (Recommended)

With docker-compose.dev.yml, your local services/raggr directory is synced to the container. Migrations generated inside the container appear on your host automatically.

# 1. Start the dev environment
docker compose -f docker-compose.dev.yml up -d

# 2. Generate migration (runs inside container, syncs to host)
docker compose -f docker-compose.dev.yml exec raggr aerich migrate --name describe_your_change

# 3. Verify migration was created
ls services/raggr/migrations/models/

# 4. Commit the migration
git add services/raggr/migrations/
git commit -m "Add migration: describe_your_change"

Production Container

For production, migration files are baked into the image. You must generate migrations in dev first.

# If you need to generate a migration from production (not recommended):
docker compose exec raggr aerich migrate --name describe_your_change

# Copy the file out of the container
docker cp $(docker compose ps -q raggr):/app/migrations/models/ ./services/raggr/migrations/

Applying Migrations

Migrations apply automatically on container start via the startup scripts.

Manual application (if needed):

# Dev
docker compose -f docker-compose.dev.yml exec raggr aerich upgrade

# Production
docker compose exec raggr aerich upgrade

Checking Migration Status

# View applied migrations
docker compose exec raggr aerich history

# View pending migrations
docker compose exec raggr aerich heads

Rolling Back

# Downgrade one migration
docker compose exec raggr aerich downgrade

# Downgrade to specific version
docker compose exec raggr aerich downgrade -v 1

Deployment Workflows

Development

# Start with watch mode (auto-restarts on file changes)
docker compose -f docker-compose.dev.yml up

# Or with docker compose watch (requires Docker Compose v2.22+)
docker compose -f docker-compose.dev.yml watch

The dev environment:

• Syncs services/raggr/ to /app in the container
• Rebuilds frontend on changes
• Auto-applies migrations on startup

Production (zero-downtime blue-green)

The app runs as one of two identical compose services: raggr-blue (host port 8080) or raggr-green (host port 8081), gated behind compose profiles so only one color runs at a time. Caddy health-checks both ports and routes traffic to whichever is healthy.

# Pull latest and swap colors with zero downtime
make redeploy

# Or run the swap directly
./scripts/deploy.sh

# View logs / exec into the active color (Makefile resolves the color)
make logs
make migrate-history

scripts/deploy.sh does the following:

1. Detects the running color via docker compose ps.
2. Builds the inactive color.
3. Starts it with docker compose up -d --wait, which blocks until the container healthcheck (GET /api/health) passes.
4. If the new color never becomes healthy, it is stopped and the old color keeps serving (the script exits non-zero).
5. Otherwise the old color is stopped with a 30s grace period so in-flight SSE streams can finish, then removed.

To target a color manually, enable its profile:

docker compose --profile blue up -d raggr-blue
docker compose --profile blue --profile green ps

Caddy configuration

Caddy needs both upstreams with active health checks:

ask.torrtle.co {
  reverse_proxy host.docker.internal:8080 host.docker.internal:8081 {
    lb_policy first          # prefer blue when both are healthy
    lb_try_duration 5s       # retry on the other upstream during the swap
    lb_try_interval 250ms
    health_uri /api/health
    health_interval 2s
    health_timeout 2s
    fail_duration 10s
  }
}

One-time cutover from the old raggr service

The first deploy after this change lands needs manual steps, because the old raggr container still holds port 8080 and the deploy script doesn't know about it:

# 1. Update the Caddy config to the two-upstream block below and reload Caddy
# 2. On the server:
git pull
docker compose --profile green up -d --build --wait raggr-green   # port 8081, no conflict
docker stop -t 30 $(docker ps -q --filter name=raggr-1)           # old raggr container
docker rm $(docker ps -aq --filter name=raggr-1)
# From here on, `make redeploy` handles swaps (green is now the active color)

Blue-green caveats

• Migrations must be backward-compatible. The new color runs aerich upgrade on startup while the old color is still serving the previous code. Additive changes (new tables/columns) are safe; destructive changes (drop/rename a column the old code reads) need a two-step deploy: ship code that stops using the column first, then drop it in a later deploy.
• Obsidian sync overlap. During the ~30s swap window both containers run pull-only ob sync against the shared ./obvault mount. Both are read-only pulls, so this is low risk, but avoid deploying mid-reindex.

Fresh Deploy (New Database)

On first deploy with an empty database, startup-dev.sh runs aerich init-db instead of aerich upgrade. This creates all tables from the current models.

For production (startup.sh), ensure the database exists and run:

# If aerich table doesn't exist yet
docker compose exec raggr aerich init-db

# Or if migrating from existing schema
docker compose exec raggr aerich upgrade

Troubleshooting

"No migrations found" on startup

The migrations/models/ directory is empty or not copied into the image.

Fix: Ensure migrations are committed and the Dockerfile copies them:

COPY migrations ./migrations

Migration fails with "relation already exists"

The database has tables but aerich doesn't know about them (fresh aerich setup on existing DB).

Fix: Fake the initial migration:

# Mark initial migration as applied without running it
docker compose exec raggr aerich upgrade --fake

Model changes not detected

Aerich compares models against the last migration's state. If state is out of sync:

# Regenerate migration state (dangerous - review carefully)
docker compose exec raggr aerich migrate --name fix_state

Database connection errors

Ensure PostgreSQL is healthy before running migrations:

# Check postgres status
docker compose ps postgres

# Wait for postgres then run migrations
docker compose exec raggr bash -c "sleep 5 && aerich upgrade"

File Reference

File	Purpose
pyproject.toml	Aerich config ([tool.aerich] section)
migrations/models/	Migration files
startup.sh	Production startup (runs aerich upgrade)
startup-dev.sh	Dev startup (runs aerich upgrade or init-db)
app.py	Contains TORTOISE_CONFIG
aerich_config.py	Aerich initialization configuration

Quick Reference

Task	Command
Generate migration	docker compose -f docker-compose.dev.yml exec raggr aerich migrate --name name
Apply migrations	docker compose exec raggr aerich upgrade
View history	docker compose exec raggr aerich history
Rollback	docker compose exec raggr aerich downgrade
Fresh init	docker compose exec raggr aerich init-db