Commit Graph

184 Commits

Author SHA1 Message Date
Ryan Chen f5203e0466 Add scheduled messages and strip markdown from iMessage responses
Strip markdown formatting (bold, italic, headers, code, links, lists) from
LLM responses before sending via iMessage. Add scheduled messages feature
with CRUD API, background scheduler loop, and admin frontend panel.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 23:25:10 -04:00
Ryan Chen 3ba93c55f4 Add channel-scoped conversations for iMessage, WhatsApp, and email
Revert get_conversation_for_user to use Conversation.get() with
MultipleObjectsReturned fallback. Add channel field to Conversation
model and get_conversation_for_channel helper so each messaging
channel gets its own isolated conversation per user.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:39:39 -04:00
Ryan Chen a693874662 Add SendBlue env vars to docker-compose
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:34:08 -04:00
Ryan Chen 846477075e Add link_imessage management script
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:31:12 -04:00
Ryan Chen 1e753bfaab Add SendBlue webhook signature validation
Validates sb-signing-secret header against SENDBLUE_WEBHOOK_SECRET env var.
Can be disabled with SENDBLUE_SIGNATURE_VALIDATION=false for development.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:28:35 -04:00
Ryan Chen 20576cabf3 Add SendBlue iMessage integration with admin-only access
- New imessage blueprint: webhook receives inbound iMessages, runs through
  LangChain agent, replies via SendBlue REST API
- Admin-only: only users with lldap_admin group can use iMessage channel
- Admin endpoints to link/unlink imessage_number on user accounts
- Add imessage_number field to User model (needs aerich migration)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:22:15 -04:00
Ryan Chen 02dd3df1f9 Use calendar events list API for all-day event support
Switch from gws calendar +agenda to gws calendar events list with
explicit timeMin/timeMax and singleEvents=true to include all-day events.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:12:17 -04:00
Ryan Chen 33f19e704c Fix gws CLI npm package name to @googleworkspace/cli
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:04:10 -04:00
ryan 8eee58de60 ops 2026-06-03 19:02:18 -04:00
Ryan Chen 98c47d5507 Add read-only Google Calendar integration via gws CLI
Adds a get_calendar_events agent tool that shells out to `gws calendar +agenda`
for admin users. Controlled by GOOGLE_CALENDAR_ENABLED env var, with OAuth
credentials mounted from credentials.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 19:01:33 -04:00
Ryan Chen 9a149cdaa6 Use in-memory cache for obsidian indexed files instead of cross-engine DB query
The async/sync engine split caused visibility issues where newly indexed
files weren't found on the next cycle, triggering re-indexing of all 36
files every 60 seconds. Replace with a module-level dict that loads from
DB on cold start and stays in sync via cache updates after each indexing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 07:37:39 -04:00
Ryan Chen 00c9b44c0e Preserve wikilink text in Obsidian indexing and fix duplicate sync
Two fixes:
- Convert wikilinks to display text instead of stripping them entirely.
  [[Noah]] becomes "Noah", [[target|display]] becomes "display". This
  was causing names and references in wikilinks to be invisible to search.
- Switch _get_obsidian_indexed_files to async engine to avoid stale reads
  from the separate sync engine, which caused files to be re-indexed
  every cycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 07:35:24 -04:00
Ryan Chen 73e952c617 Set Obsidian sync to pull-only mode
Prevents local changes from being pushed back to Obsidian servers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 07:29:18 -04:00
Ryan Chen 9f51dc3cdb Fix NOT NULL violation on empty splits and increase search results to k=6
Empty documents after sanitization caused aadd_documents to issue a
DEFAULT VALUES insert. Guard with an emptiness check. Also increase
similarity search k from 2 to 6 so multi-word queries like full names
have better recall.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 07:16:28 -04:00
Ryan Chen 1e6bc536b4 Fix datetime serialization in Obsidian metadata for pgvector
YAML frontmatter can contain datetime objects which aren't JSON
serializable. Add _make_serializable() to coerce all metadata values
before storing in pgvector.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 07:11:29 -04:00
Ryan Chen 869de1c250 Add incremental Obsidian-to-pgvector sync with background watcher
Replace full delete-and-reindex with mtime-based incremental sync that
only re-indexes changed/new files and removes deleted ones. A background
polling task keeps the vector store up-to-date automatically when
OBSIDIAN_CONTINUOUS_SYNC=true.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 07:05:48 -04:00
Ryan Chen 2cd77c68c1 Fix daily note path to match vault structure
Update from journal/YYYY/YYYY-MM-DD.md to
50 - Journal/YYYY/MM/YYYY-MM-DD.md to match the actual Obsidian vault
folder layout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 00:08:40 -04:00
Ryan Chen ad5b889bf1 Remove stale Obsidian sync lock before starting continuous sync
The ob CLI uses a directory lock at .obsidian/.sync.lock that persists
across container restarts via the volume mount, causing "Another sync
instance is already running" errors. Remove it before starting sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 00:07:37 -04:00
Ryan Chen 75e6b09464 Guard against missing Obsidian credentials to prevent startup hang
ob login blocks waiting for interactive input when OBSIDIAN_EMAIL or
OBSIDIAN_PASSWORD is empty. Check required env vars before attempting
login to skip sync gracefully with a warning instead of hanging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-31 00:02:28 -04:00
Ryan Chen 47238f8567 Fix Obsidian sync race condition and block credentials.json from being served
Run ob login and sync-setup in foreground before backgrounding sync to
prevent "Another sync instance is already running" error. Restrict the
catch-all route to only serve whitelisted static file extensions to
prevent sensitive files like credentials.json from being exposed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-30 23:59:29 -04:00
Ryan Chen 5e0e2994c2 Fix Obsidian sync setup in Docker by running login and sync-setup before sync
Replace OBSIDIAN_AUTH_TOKEN with OBSIDIAN_EMAIL/OBSIDIAN_PASSWORD and run
the full ob login → sync-setup → sync sequence on container startup so
fresh containers authenticate properly instead of failing with
"Run 'ob sync-setup' first".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-30 23:52:30 -04:00
Ryan Chen 9629bfcef4 Fix embedding tokenizer mismatch with custom embedding server
Disable tiktoken pre-encoding for custom embedding servers. LangChain
was encoding text into OpenAI token IDs then sending them to llama-server
which has a different vocabulary, causing "invalid tokens" errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 23:42:23 -04:00
Ryan Chen b4097730ef Add per-chunk error logging and broaden text sanitizer
Indexes chunks one at a time with error logging to identify which
document/chunk causes embedding failures. Also strips Unicode surrogates
and replacement characters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 23:38:03 -04:00
Ryan Chen abb06b78e2 Sanitize document text before embedding to fix tokenizer errors
Strips null bytes, control characters, and excessive whitespace from
document content before sending to embedding models. Fixes 400 errors
from BERT-based tokenizers (e.g. nomic-embed-text) on PDF-extracted text.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 23:35:25 -04:00
Ryan Chen 92171cbfb6 Support custom OpenAI-compatible embedding server with OpenAI fallback
Adds EMBEDDING_SERVER_URL and EMBEDDING_MODEL_NAME env vars, mirroring
the existing LLAMA_SERVER_URL pattern for LLM configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 23:24:54 -04:00
ryan 8e884b5e76 Merge pull request 'Refactor frontend to hook-based architecture' (#33) from refactor/frontend-hooks into main
Reviewed-on: #33
2026-04-24 09:12:47 -04:00
ryan ed973357e8 Merge pull request 'Improve Simba system prompt' (#31) from feat/improve-system-prompt into main
Reviewed-on: #31
2026-04-24 09:12:39 -04:00
Ryan Chen 4ac0754ea7 Refactor frontend to hook-based architecture
Extract logic from god components into custom hooks (useAuthCheck,
useConversations, useChat, usePresignedUrl, useAdminUsers, useOIDCAuth).
Eliminate unnecessary useEffects per React guidelines — scroll is now
imperative, isAdmin comes from useAuthCheck instead of a separate fetch.
ConversationList becomes a pure presentational component. Wrap bubble
components in React.memo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 09:11:57 -04:00
Ryan Chen db977270a3 Improve Simba system prompt for more helpful responses
Shift focus from cat persona to genuine helpfulness. Keep light
cat flavor but prioritize thorough, detailed answers over the
assertive cat act.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 08:58:29 -04:00
ryan bac773ae4b Merge pull request 'Enable async_mode on PGVector for async method support' (#30) from refactor/chromadb-to-pgvector into main
Reviewed-on: #30
2026-04-24 08:53:34 -04:00
Ryan Chen 564a9b68a5 Enable async_mode on PGVector for async method support
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 08:53:21 -04:00
ryan 7742673cc0 Merge pull request 'Handle missing pgvector tables on first run' (#29) from refactor/chromadb-to-pgvector into main
Reviewed-on: #29
2026-04-24 08:49:38 -04:00
Ryan Chen c157c37cde Handle missing pgvector tables on first run
_get_collection_id now catches the UndefinedTable error that occurs
before the first index operation creates the langchain tables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 08:49:00 -04:00
ryan 3b8fa3e7a0 Merge pull request 'Replace ChromaDB with pgvector' (#28) from refactor/chromadb-to-pgvector into main
Reviewed-on: #28
2026-04-24 08:44:39 -04:00
Ryan Chen 438399646f Replace ChromaDB with pgvector for vector storage
Consolidate onto PostgreSQL by using pgvector instead of a separate
ChromaDB instance. This removes a Docker volume, a large dependency,
and simplifies the stack without meaningful performance impact at
our document scale.

- Swap langchain-chroma for langchain-postgres (PGVector)
- Use pgvector/pgvector:pg16 Docker image with init script
- Lazy-initialize vector store to avoid eager DB connections
- Add SQL helpers for stats/delete/list (replacing _collection access)
- Remove legacy main.py, chunker, petmd scraper, and /api/query endpoint

Re-index required after deploy (POST /api/rag/index + /index-obsidian).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 08:43:52 -04:00
ryan 9ed4ca126a Merge pull request 'Fix mobile conversation launch resetting to homepage' (#27) from fix/mobile-conversation-launch into main
Reviewed-on: #27
2026-04-09 22:09:55 -04:00
Ryan Chen f3ae76ce68 Fix mobile conversation launch resetting to homepage
Remove the useEffect on selectedConversation.id that race-conditions
with handleQuestionSubmit — it fetches the (still-empty) conversation
and wipes messages, sending the user back to the empty state. Refresh
conversation list after streaming completes instead to pick up the
auto-generated title.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 22:08:26 -04:00
ryan 7ee3bdef84 Merge pull request 'Simplify conversation naming to first message truncation' (#26) from feat/conversation-name-truncation into main
Reviewed-on: #26
2026-04-09 22:04:33 -04:00
Ryan Chen 500c44feb1 Simplify conversation naming to truncate first message
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 22:02:10 -04:00
ryan 896501deb1 Merge pull request 'Add user memory for cross-conversation recall' (#25) from feat/user-memory into main
Reviewed-on: #25
2026-04-09 21:54:04 -04:00
Ryan Chen c95800e65d Add user memory feature for cross-conversation recall
Give the LangChain agent a save_user_memory tool so users can ask it to
remember preferences and personal facts. Memories are stored per-user in
a new user_memories table and injected into the system prompt on each
conversation turn.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 21:53:14 -04:00
ryan 90372a6a6d Merge pull request 'Order conversations by recency and auto-name from first message' (#24) from feat/conversation-ordering-and-naming into main
Reviewed-on: #24
2026-04-05 10:43:09 -04:00
Ryan Chen c01764243f Order conversations by recency and auto-name from first message
Conversations are now returned sorted by most recently updated first.
New conversations are named using the first 100 characters of the
user's initial message instead of a username+timestamp placeholder.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:34:48 -04:00
ryan dfaac4caf8 Merge pull request 'Extend JWT token expiration times' (#23) from extend-jwt-expiration into main
Reviewed-on: #23
2026-04-05 10:13:29 -04:00
ryan 17c3a2f888 Merge pull request 'Add redeploy Makefile target' (#20) from feat/makefile-redeploy into main
Reviewed-on: #20
2026-04-05 10:13:01 -04:00
ryan fa0f68e3b4 Merge pull request 'Fix OIDC login crash when groups claim is null' (#22) from fix/oidc-null-groups into main
Reviewed-on: #22
2026-04-05 10:12:55 -04:00
Ryan Chen a6c698c6bd Fix OIDC login crash when groups claim is null
Use `claims.get("groups") or []` instead of `claims.get("groups", [])`
so that an explicit `null` value is coerced to an empty list, preventing
a ValueError on the non-nullable ldap_groups field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:12:12 -04:00
Ryan Chen 07c272c96a Extend JWT token expiration times
Access tokens now last 1 hour (up from default 15 min) and refresh
tokens last 30 days, reducing frequent re-authentication.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 10:10:47 -04:00
ryan 975a337af4 Merge pull request 'Fix mobile performance degradation during typing and after image upload' (#21) from fix/mobile-input-performance into main
Reviewed-on: #21
2026-04-05 06:59:39 -04:00
Ryan Chen e644def141 Fix mobile performance degradation during typing and after image upload
Memoize blob URL creation to prevent leak on every keystroke, wrap
MessageInput in React.memo with stable useCallback props, remove
expensive backdrop-blur-sm from chat footer, and use instant scroll
during streaming to avoid queuing smooth scroll animations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 06:58:53 -04:00