Strip markdown formatting (bold, italic, headers, code, links, lists) from
LLM responses before sending via iMessage. Add scheduled messages feature
with CRUD API, background scheduler loop, and admin frontend panel.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Revert get_conversation_for_user to use Conversation.get() with
MultipleObjectsReturned fallback. Add channel field to Conversation
model and get_conversation_for_channel helper so each messaging
channel gets its own isolated conversation per user.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Validates sb-signing-secret header against SENDBLUE_WEBHOOK_SECRET env var.
Can be disabled with SENDBLUE_SIGNATURE_VALIDATION=false for development.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New imessage blueprint: webhook receives inbound iMessages, runs through
LangChain agent, replies via SendBlue REST API
- Admin-only: only users with lldap_admin group can use iMessage channel
- Admin endpoints to link/unlink imessage_number on user accounts
- Add imessage_number field to User model (needs aerich migration)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch from gws calendar +agenda to gws calendar events list with
explicit timeMin/timeMax and singleEvents=true to include all-day events.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a get_calendar_events agent tool that shells out to `gws calendar +agenda`
for admin users. Controlled by GOOGLE_CALENDAR_ENABLED env var, with OAuth
credentials mounted from credentials.json.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The async/sync engine split caused visibility issues where newly indexed
files weren't found on the next cycle, triggering re-indexing of all 36
files every 60 seconds. Replace with a module-level dict that loads from
DB on cold start and stays in sync via cache updates after each indexing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two fixes:
- Convert wikilinks to display text instead of stripping them entirely.
[[Noah]] becomes "Noah", [[target|display]] becomes "display". This
was causing names and references in wikilinks to be invisible to search.
- Switch _get_obsidian_indexed_files to async engine to avoid stale reads
from the separate sync engine, which caused files to be re-indexed
every cycle.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Empty documents after sanitization caused aadd_documents to issue a
DEFAULT VALUES insert. Guard with an emptiness check. Also increase
similarity search k from 2 to 6 so multi-word queries like full names
have better recall.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YAML frontmatter can contain datetime objects which aren't JSON
serializable. Add _make_serializable() to coerce all metadata values
before storing in pgvector.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace full delete-and-reindex with mtime-based incremental sync that
only re-indexes changed/new files and removes deleted ones. A background
polling task keeps the vector store up-to-date automatically when
OBSIDIAN_CONTINUOUS_SYNC=true.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update from journal/YYYY/YYYY-MM-DD.md to
50 - Journal/YYYY/MM/YYYY-MM-DD.md to match the actual Obsidian vault
folder layout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Disable tiktoken pre-encoding for custom embedding servers. LangChain
was encoding text into OpenAI token IDs then sending them to llama-server
which has a different vocabulary, causing "invalid tokens" errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Indexes chunks one at a time with error logging to identify which
document/chunk causes embedding failures. Also strips Unicode surrogates
and replacement characters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Strips null bytes, control characters, and excessive whitespace from
document content before sending to embedding models. Fixes 400 errors
from BERT-based tokenizers (e.g. nomic-embed-text) on PDF-extracted text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds EMBEDDING_SERVER_URL and EMBEDDING_MODEL_NAME env vars, mirroring
the existing LLAMA_SERVER_URL pattern for LLM configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Shift focus from cat persona to genuine helpfulness. Keep light
cat flavor but prioritize thorough, detailed answers over the
assertive cat act.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
_get_collection_id now catches the UndefinedTable error that occurs
before the first index operation creates the langchain tables.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate onto PostgreSQL by using pgvector instead of a separate
ChromaDB instance. This removes a Docker volume, a large dependency,
and simplifies the stack without meaningful performance impact at
our document scale.
- Swap langchain-chroma for langchain-postgres (PGVector)
- Use pgvector/pgvector:pg16 Docker image with init script
- Lazy-initialize vector store to avoid eager DB connections
- Add SQL helpers for stats/delete/list (replacing _collection access)
- Remove legacy main.py, chunker, petmd scraper, and /api/query endpoint
Re-index required after deploy (POST /api/rag/index + /index-obsidian).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Give the LangChain agent a save_user_memory tool so users can ask it to
remember preferences and personal facts. Memories are stored per-user in
a new user_memories table and injected into the system prompt on each
conversation turn.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Conversations are now returned sorted by most recently updated first.
New conversations are named using the first 100 characters of the
user's initial message instead of a username+timestamp placeholder.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use `claims.get("groups") or []` instead of `claims.get("groups", [])`
so that an explicit `null` value is coerced to an empty list, preventing
a ValueError on the non-nullable ldap_groups field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unused image_url from upload response and TS type
- Remove bare except in serve_image that masked config errors as 404s
- Add error state and broken-image placeholder in QuestionBubble
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Browser <img> tags can't attach JWT headers, causing 401s. The image
endpoint now returns a time-limited presigned S3 URL via authenticated
API call, which the frontend fetches and uses directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Users can now attach images in the web chat for Simba to analyze using
Ollama's gemma3 vision model. Images are stored in Garage (S3-compatible)
and displayed in chat history.
Also fixes aerich migration config by extracting TORTOISE_CONFIG into a
standalone config/db.py module, removing the stale aerich_config.py, and
adding missing MODELS_STATE to migration 3.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move blueprints.email.helpers import from module-level to inside the
endpoint functions that use it, breaking the circular dependency chain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Users can now receive a unique email address (ask+<token>@domain) and
interact with Simba by sending emails. Inbound emails hit a Mailgun
webhook, are authenticated via HMAC token lookup, processed through the
LangChain agent, and replied to via the Mailgun API.
- Extract shared SIMBA_SYSTEM_PROMPT to blueprints/conversation/prompts.py
- Add email_enabled and email_hmac_token fields to User model
- Create blueprints/email with webhook, signature validation, rate limiting
- Add admin endpoints to enable/disable email per user
- Update AdminPanel with Email column, toggle, and copy-address button
- Add Mailgun env vars to .env.example
- Include database migration for new fields
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add /me, /admin/users, and WhatsApp link/unlink endpoints
- Add AdminPanel component with user management UI
- Add userService methods for admin API calls
- Fix simba mode so cat responses appear in the message list
- Fetch userinfo endpoint for groups on OIDC callback (Authelia compat)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Implemented parse_email_body function for RFC822 email parsing
- Uses stdlib email.message_from_bytes with modern EmailMessage API
- Extracts text and HTML bodies using get_body() method
- Prefers plain text over HTML for "preferred" field
- Converts HTML to text using html2text when text body missing
- Extracts all metadata: subject, from, to, date, message_id
- Uses parsedate_to_datetime for proper date parsing
- Handles UnicodeDecodeError gracefully with partial data return
- Follows async patterns and logging conventions from existing codebase
- Created IMAPService class with async connect/list_folders/close methods
- Uses aioimaplib for async IMAP4_SSL operations
- Implements proper connection cleanup with logout() not close()
- Added aioimaplib and html2text dependencies to pyproject.toml
- Follows async patterns from existing service classes (ynab_service.py, mealie_service.py)
- Includes comprehensive logging with [IMAP] and [IMAP ERROR] prefixes
- Add EncryptedTextField for transparent Fernet encryption
- Create EmailAccount model with encrypted IMAP credentials
- Create EmailSyncStatus model for sync state tracking
- Create Email model with 30-day retention logic
- Follow existing blueprint patterns from users/conversation
- Update llm.py to use OpenAI client with custom base_url for llama-server
- Update agents.py to use ChatOpenAI instead of ChatOllama
- Remove unused ollama imports from main.py, chunker.py, query.py
- Add LLAMA_SERVER_URL and LLAMA_MODEL_NAME env vars
- Remove ollama and langchain-ollama dependencies
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Ryan Chen