ryan/simbarag
1
0
Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Add read timeouts to llama-server clients so hangs fail fast #43

Merged
ryan merged 2 commits from llama-timeouts into main 2026-09-11 17:18:15 -04:00
Conversation 0 Commits 2 Files Changed 3
+30 -1
ryan commented 2026-09-11 17:05:45 -04:00
Owner
Copy Link
Copy Source

The OpenAI client defaults to a 600s read timeout, so a wedged llama-server blocked requests for 10 minutes and never raised — meaning the existing with_fallbacks() to OpenAI never triggered.

  • ChatOpenAI (conversation agent) and LLMClient now use a per-chunk read timeout (120s, configurable via LLAMA_TIMEOUT_SECONDS) with one retry
  • A hang now surfaces as an exception quickly and the conversation continues on the OpenAI fallback; context is preserved because message history is rebuilt from Postgres on every request
The OpenAI client defaults to a 600s read timeout, so a wedged llama-server blocked requests for 10 minutes and never raised — meaning the existing with_fallbacks() to OpenAI never triggered. - ChatOpenAI (conversation agent) and LLMClient now use a per-chunk read timeout (120s, configurable via LLAMA_TIMEOUT_SECONDS) with one retry - A hang now surfaces as an exception quickly and the conversation continues on the OpenAI fallback; context is preserved because message history is rebuilt from Postgres on every request
ryan added 1 commit 2026-09-11 17:05:45 -04:00
Add read timeouts to llama-server clients so hangs fail fast b361377088 
The OpenAI client defaults to a 600s read timeout, so a wedged
llama-server blocked requests for 10 minutes and never raised, which
meant with_fallbacks() to OpenAI never triggered. Set a per-chunk read
timeout (120s, configurable via LLAMA_TIMEOUT_SECONDS) plus one retry
on the ChatOpenAI and LLMClient clients. Context is preserved because
conversation history is rebuilt from Postgres on every request.
ryan added 1 commit 2026-09-11 17:13:13 -04:00
Document git workflow (Gitea, tea CLI, kermit = commit) 56ad5f7046
ryan merged commit 2988482791 into main 2026-09-11 17:18:15 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
ryan
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ryan/simbarag#43
Delete Branch "llama-timeouts"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?