simbarag/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

SimbaRAG is a RAG (Retrieval-Augmented Generation) conversational AI system for querying information about Simba (a cat) and the household. It ingests documents from Paperless-NGX, Obsidian, and Immich, stores embeddings in PostgreSQL via pgvector (Chroma client), uses LLMs (Ollama or OpenAI) to answer questions with a tool-using LangChain agent, and exposes chat over web, iMessage, and WhatsApp.

Commands

Development

# Start environment (raggr runs as raggr-blue/raggr-green behind compose profiles)
docker compose --profile blue up --build

# View logs (Makefile resolves the active color)
make logs

# Reindex vector store (all sources / Obsidian only)
make reindex
make reindex-obsidian

Database Migrations (Aerich/Tortoise ORM)

# Generate migration (must run in Docker with DB access)
make migrate-new

# Apply migrations (auto-runs on startup, manual if needed)
make migrate

# View migration history
make migrate-history

Migrations must be backward-compatible: blue-green deploys run aerich upgrade in the new container while the old code still serves.

Frontend

cd raggr-frontend
yarn install
yarn build      # Production build
yarn dev        # Dev server (rarely needed, backend serves frontend)

Production

Zero-downtime blue-green deploy: raggr-blue (port 8080) and raggr-green (port 8081) alternate behind Caddy health checks. See docs/deployment.md.

make redeploy    # git pull + scripts/deploy.sh (build new color, wait healthy, stop old)

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Docker Compose                          │
├─────────────────────────────────────────────────────────────┤
│  raggr-blue/green (8080/8081)   │  postgres (port 5432)    │
│  ├── Quart backend              │  PostgreSQL 16 + pgvector│
│  ├── React frontend (served)    │                          │
│  └── Twilio webhooks (iMessage/ │                          │
│      WhatsApp)                  │                          │
└─────────────────────────────────────────────────────────────┘

Backend (root directory):

• app.py - Quart application entry, serves API and static frontend
• main.py - RAG logic, document indexing, LLM interaction, LangChain agent
• llm.py - LLM client with Ollama primary, OpenAI fallback
• blueprints/ - API routes organized as Quart blueprints
  • users/ - OIDC auth, JWT tokens, RBAC with LDAP groups
  • conversation/ - Chat conversations, message history, streamed answers
  • rag/ - Document indexing endpoints (admin-only); logic in rag/logic.py
  • email/ - Email integration (Mailgun webhooks, Fernet encryption, IMAP, parsing)
  • imessage/ - Twilio webhook receiver for iMessage (HMAC-validated, number allowlist)
  • whatsapp/ - Twilio webhook receiver for WhatsApp (TWIML replies, SendBlue send support)
  • scheduled_messages/ - Scheduled message delivery with background scheduler
• config/ - Configuration modules
  • db.py - Database setup
  • oidc_config.py - OIDC authentication configuration
• utils/ - Reusable utilities and household integrations
  • chunker.py - Document chunking for embeddings
  • cleaner.py - PDF cleaning and summarization
  • image_process.py - Image description with LLM
  • request.py - Paperless-NGX API client
  • obsidian_service.py, mealie_service.py, ynab_service.py, s3_client.py, simbakit_service.py - integrations with Obsidian notes, Mealie recipes, YNAB budgets, S3, and SimbaKit
• scripts/ - Administrative and utility scripts
  • add_user.py - Create users manually
  • user_message_stats.py - User message statistics
  • manage_vectorstore.py / inspect_vector_store.py - Vector store CLI
  • index_immich.py - Index Immich photos into the vector store
  • link_imessage.py - Link iMessage contacts to users
  • deploy.sh - Blue-green deployment script
• migrations/ - Database migration files
• tests/ - pytest test suite (tests/unit/ covers email/crypto, YNAB, Obsidian, SimbaKit, rate limiting, user model)

simbakit/ (standalone subproject):

• Self-hosted PetKit device poller + web dashboard for Simba's smart devices (feeder, litter box, fountain, purifier)
• Own pyproject.toml, uv.lock, and Dockerfile; Python 3.14 + uv
• Polls PetKit cloud API every 2 min into SQLite; Chart.js dashboard; docs in simbakit/docs/
• Queried from the main app via utils/simbakit_service.py

Frontend (raggr-frontend/):

• React 19 with Rsbuild bundler
• Tailwind CSS for styling
• Built to dist/, served by backend at /
• Key components: ChatScreen, ConversationsPage, LoginScreen, AdminPanel, ScheduledMessagesPanel, ToolBubble (agent tool-call display)

Auth Flow: LLDAP → Authelia (OIDC) → Backend JWT → Frontend localStorage

Data Sources: Paperless-NGX documents, Obsidian vault, Immich photos (indexed via scripts/index_immich.py / make reindex)

Testing

Always run make test before pushing code to ensure all tests pass.

make test       # Run tests
make test-cov   # Run tests with coverage

Note: make test may fail locally (no pytest on PATH); use .venv/bin/pytest tests/ -q instead.

Git Workflow

• The host runs Gitea (git.torrtle.co), not GitHub, despite the remote name. Use the tea CLI for PRs (Gitea flavor: tea pr create, tea pr list, etc. — no tea create).
• Typical flow: commit locally, git push -u origin <branch>, then tea pr create -t "title" -d "body".
• "kermit" is the user's slang for commit (e.g. "kermit and push" = commit and push). There is no kermit tool.
• Keep one logical change per branch/PR; don't ride along on stale feature branches.
• Don't commit .DS_Store, chroma_db/, chromadb/, .planning/ (local artifacts).

Key Patterns

• All endpoints are async (async def)
• Use @jwt_refresh_token_required for authenticated endpoints
• Use @admin_required for admin-only endpoints (checks lldap_admin group)
• Tortoise ORM models in blueprints/*/models.py
• External integrations are service classes in utils/*_service.py
• Frontend API services in raggr-frontend/src/api/

Environment Variables

See .env.example. Key ones:

• DATABASE_URL - PostgreSQL connection
• OIDC_* - Authelia OIDC configuration
• LLAMA_SERVER_URL / LLAMA_MODEL_NAME - Local Llama/Ollama LLM server
• EMBEDDING_SERVER_URL / EMBEDDING_MODEL_NAME - Embedding model
• OPENAI_API_KEY - Fallback LLM; TAVILY_API_KEY - web search tool
• PAPERLESS_TOKEN / BASE_URL - Document source
• TWILIO_*, ALLOWED_IMESSAGE_NUMBERS, ALLOWED_WHATSAPP_NUMBERS - iMessage/WhatsApp webhooks
• MAILGUN_*, EMAIL_HMAC_SECRET, FERNET_KEY - email integration
• SENDBLUE_* - SMS sending for scheduled messages
• YNAB_*, MEALIE_*, OBSIDIAN_*, IMMICH_*, SIMBAKIT_URL - household integrations

Documentation

Project docs live in docs/ (mkdocs): deployment.md, development.md, authentication.md, MIGRATIONS.md, VECTORSTORE.md, TASKS.md, ynab_integration/.