The OpenAI client defaults to a 600s read timeout, so a wedged
llama-server blocked requests for 10 minutes and never raised, which
meant with_fallbacks() to OpenAI never triggered. Set a per-chunk read
timeout (120s, configurable via LLAMA_TIMEOUT_SECONDS) plus one retry
on the ChatOpenAI and LLMClient clients. Context is preserved because
conversation history is rebuilt from Postgres on every request.
- Update llm.py to use OpenAI client with custom base_url for llama-server
- Update agents.py to use ChatOpenAI instead of ChatOllama
- Remove unused ollama imports from main.py, chunker.py, query.py
- Add LLAMA_SERVER_URL and LLAMA_MODEL_NAME env vars
- Remove ollama and langchain-ollama dependencies
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Ryan Chen