feat: add S3/object storage bucket upload support
Summary
This merge request adds S3/object storage bucket upload support to Corpus CLI, allowing users to upload files directly from s3:// URIs.
Changes
S3/Object Storage Integration
- Add
aioboto3andbotocoredependencies for S3 integration - Implement S3 download producer for
s3://URI support - Add S3 credential management and interactive credential setup
- Add
ProgressTrackerfor upload progress monitoring - Add S3 buffer management and cleanup utilities
- Update CLI to accept
s3://URIs in upload command - Add comprehensive test coverage for S3 upload functionality
Upload Fixes
- Fix macOS buffer cleanup: delete
.DS_Storebeforermdir, useshutil.rmtreefor final cleanup instead ofos.rmdir
Testing
- Added comprehensive test coverage for S3 upload functionality
- Fixed mypy type errors and ruff linting issues
- Updated existing tests to match new code paths