refactor: replace custom chunk manager with TUS protocol (tus-py-server)
Summary
app/utils/chunk_manager.py (266 lines) implements a custom resumable upload system — chunk storage in /tmp/chunks/, assembly, progress tracking, and missing-chunk detection. This reinvents the TUS resumable upload protocol, a well-established open standard with server and client libraries.
Problem
- Custom temp-file orchestration in
/tmpwith manual cleanup (also a Celery maintenance task) - No standard client contract — frontend must use our bespoke chunk API, not any off-the-shelf uploader
- Resume logic is home-grown: chunk index tracking, completeness validation, and ordering are all custom
- Bugs in edge cases (network interruption mid-chunk, duplicate chunks) require our team to diagnose and fix
- The Celery maintenance task (
app/tasks/maintenance.py) exists solely to clean up after this module
Proposed Solution
Adopt tus-py-server or a FastAPI-compatible TUS server implementation. TUS is a protocol standard — any TUS-compliant client (Uppy, tus-js-client, tus-android-client) works out of the box.
Library signals:
- Protocol: Open standard (MIT), widely adopted
- Client ecosystem: Uppy (JS), tus-js-client, tus-java-client, tus-android-client
- Maintenance: Active community
- Test suite: Protocol compliance tests exist
- CVEs: None known
Impact
| Dimension | Current | After |
|---|---|---|
| LOC | 266 custom lines + maintenance task | Protocol library + thin config |
| Client compatibility | Bespoke API only | Any TUS-compliant client |
| Resume reliability | Custom logic | Protocol-guaranteed |
| Temp file cleanup | Dedicated Celery task | Handled by protocol expiry |
| Interoperability | None | Works with Uppy, tus-js-client, etc. |
Files Affected
-
app/utils/chunk_manager.py— replace -
app/tasks/maintenance.py— remove chunk cleanup task -
app/api/v1/upload endpoints — update to TUS endpoint handlers