Files
ryan b361377088 Add read timeouts to llama-server clients so hangs fail fast
The OpenAI client defaults to a 600s read timeout, so a wedged
llama-server blocked requests for 10 minutes and never raised, which
meant with_fallbacks() to OpenAI never triggered. Set a per-chunk read
timeout (120s, configurable via LLAMA_TIMEOUT_SECONDS) plus one retry
on the ChatOpenAI and LLMClient clients. Context is preserved because
conversation history is rebuilt from Postgres on every request.
2026-09-11 17:05:10 -04:00
..
2026-01-31 17:13:27 -05:00