ryan/simbarag
1
0
Fork 0
Code Issues Pull Requests 5 Actions 2 Packages Projects Releases Wiki Activity

Cowork E: drop files into the chat (PDF, text, CSV, images) #66

Merged
ryan merged 5 commits from feat/cowork-file-drop into main 2026-10-12 18:47:29 -04:00
Conversation 0 Commits 5 Files Changed 13
+960 -70
ryan commented 2026-10-12 18:47:23 -04:00
Owner
Copy Link
Copy Source

Task E from docs/cowork-ui-plan.md.

What

  • Backend:
    • POST /api/conversation/upload:
      • images reuse process_image
      • PDF/TXT/MD/CSV go to S3 under conversations/<id>/files/, with the original name in object metadata
      • 415 for other types; 422 for scanned/empty files ("no text layer")
      • checks the caller owns the conversation
    • stream-query accepts file_key (restricted to the conversation's prefix). The text is extracted with PyMuPDF off the event loop (asyncio.to_thread), capped at 12k chars, and appended to the agent's turn as <attached file "…">…</attached>. The saved user message records 📎 name.
    • New utils/file_extract.py, plus S3 helpers for non-image objects.
  • Frontend:
    • pendingAttachment (an image or a file) replaces pendingImage.
    • The picker accepts images, PDF, TXT, MD and CSV.
    • A whole-window "Drop to attach" overlay, which sits correctly beside or over the artifact panel and sheet.
    • Attachment chips in the input and in saved question bubbles.
    • Upload errors are shown.
  • Desktop: disable_drag_drop_handler() so HTML5 drops reach the webview in the Tauri app.
  • Upload/drop scenarios folded into the shared test-fixtures/cowork/mock-server.mjs.

Follow-ups (pre-existing, out of scope)

  • stream-query still doesn't check that image_key belongs to the conversation.
  • "Retry" after a stall resends the question without the attachment.

Test plan

  • pytest tests/ -q: 275 passed (extraction, 415/422, ownership/prefix checks, truncation)
  • yarn build; tsc --noEmit clean; cargo build in desktop/src-tauri
  • Playwright, screenshots checked by eye:
    • overlay, file chip, file_key in the request
    • image path, unsupported type
    • drop with the artifact panel open, and with the sheet open at 900 px
  • Coordinator review, 2 passes (event-loop offload, text-less files)

🤖 Generated with Claude Code

Task E from `docs/cowork-ui-plan.md`. ## What - **Backend:** - `POST /api/conversation/upload`: - images reuse `process_image` - PDF/TXT/MD/CSV go to S3 under `conversations/<id>/files/`, with the original name in object metadata - 415 for other types; 422 for scanned/empty files ("no text layer") - checks the caller owns the conversation - `stream-query` accepts `file_key` (restricted to the conversation's prefix). The text is extracted with PyMuPDF **off the event loop** (`asyncio.to_thread`), capped at 12k chars, and appended to the agent's turn as `<attached file "…">…</attached>`. The saved user message records `📎 name`. - New `utils/file_extract.py`, plus S3 helpers for non-image objects. - **Frontend:** - `pendingAttachment` (an image or a file) replaces `pendingImage`. - The picker accepts images, PDF, TXT, MD and CSV. - A whole-window "Drop to attach" overlay, which sits correctly beside or over the artifact panel and sheet. - Attachment chips in the input and in saved question bubbles. - Upload errors are shown. - **Desktop:** `disable_drag_drop_handler()` so HTML5 drops reach the webview in the Tauri app. - Upload/drop scenarios folded into the shared `test-fixtures/cowork/mock-server.mjs`. ## Follow-ups (pre-existing, out of scope) - `stream-query` still doesn't check that `image_key` belongs to the conversation. - "Retry" after a stall resends the question without the attachment. ## Test plan - [x] `pytest tests/ -q`: 275 passed (extraction, 415/422, ownership/prefix checks, truncation) - [x] `yarn build`; `tsc --noEmit` clean; `cargo build` in `desktop/src-tauri` - [x] Playwright, screenshots checked by eye: - overlay, file chip, `file_key` in the request - image path, unsupported type - drop with the artifact panel open, and with the sheet open at 900 px - [x] Coordinator review, 2 passes (event-loop offload, text-less files) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
ryan added 5 commits 2026-10-12 18:47:23 -04:00
Add conversation file upload and file_key in stream-query b2f0b9f9f1 
POST /api/conversation/upload takes an image (via process_image) or a
PDF/txt/md/csv file, checks the conversation belongs to the caller, and
stores files at conversations/<id>/files/<uuid>.<ext> with the original
name in S3 metadata. Anything else gets a 415.

stream-query accepts file_key (must sit under this conversation's
files/ prefix), extracts its text capped at 12k chars, appends it to the
agent's user turn as <attached file> and records 📎 name on the saved
message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Desktop: let HTML5 file drops reach the webview 3e50aecb96 
Tauri v2 intercepts drag-and-drop by default, so the chat's drop overlay
never saw dropped files. Disable its handler on the main window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frontend: attach files by picker or drop, with an attachment chip 35faeb0fb7 
- pendingImage becomes pendingAttachment (image or PDF/TXT/MD/CSV), uploaded
  through POST /api/conversation/upload and sent as image_key or file_key.
- MessageInput shows an icon/name/size chip for files, an inline error for
  unsupported or oversized files, and a paperclip picker.
- DropOverlay covers the chat area with "Drop to attach" while files are
  dragged over the window; drags without files are ignored.
- QuestionBubble renders a saved trailing "📎 name" as a chip.
- test-fixtures/cowork/mock-server.mjs: dependency-free mock API (static
  dist/, conversations, contract-shaped SSE, uploads, artifacts) for
  Playwright runs without Docker or OIDC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extract attachment text off the event loop; reject text-less files cc29ae1ef8 
- extract_text (PyMuPDF for PDFs) now runs via asyncio.to_thread in both
  /upload and stream-query, so a large PDF doesn't stall other streams.
- /upload returns 422 when extraction yields no text: a scanned PDF
  without a text layer, or an empty text file.
- Mock server: *scan*.pdf uploads return that 422.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop overlay sits above the artifact sheet below lg 62416fcf2e 
Beside the panel (lg and up) it still covers only the chat column. Below
lg the panel is a fixed z-40 sheet, so the overlay covers the window at
z-[45] (under the z-50 mobile drawer) instead of being hidden by it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ryan merged commit 37a8b95887 into main 2026-10-12 18:47:29 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
ryan
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: ryan/simbarag#66
Delete Branch "feat/cowork-file-drop"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?