Batch Paperless chunk embeddings during indexing #73

2026-10-12T23:02:52-04:00

ryan commented

2026-10-12 23:02:52 -04:00

index_documents sent one embedding request + insert per chunk. Chunks now go in batches of 32; a failed batch is retried one chunk at a time so the rejected chunk is still named in the logs (same diagnostics as b409773).

Tests: new tests/unit/test_paperless_indexing.py (batching, failure naming, recovery). Full suite passes.

🤖 Generated with Claude Code

index_documents sent one embedding request + insert per chunk. Chunks now go in batches of 32; a failed batch is retried one chunk at a time so the rejected chunk is still named in the logs (same diagnostics as b409773). Tests: new tests/unit/test_paperless_indexing.py (batching, failure naming, recovery). Full suite passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

ryan added 1 commit 2026-10-12 23:02:52 -04:00

Batch Paperless chunk embeddings during indexing c80125f7fe

index_documents sent one embedding request and one insert per chunk.
Send chunks in batches of 32 and, only when a batch fails, retry it one
chunk at a time so the rejected chunk is still named in the logs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ryan merged commit aefae43a6d into main

2026-10-12 23:06:29 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ryan/simbarag#73