Skip to content

feat: file uploads and Patch using CSV files

Bhaskar Battula requested to merge csv-file-uploads into develop

What is this MR?

This MR introduces CSV-based support for bulk data ingestion and updates. It allows users to provide structured CSV input for creating and updating records efficiently.

What does it do?

This change enables the system to:

  • Accept CSV files as input for bulk record creation
  • Automatically map CSV headers to corresponding fields
  • Introduce a --patch option to update existing records using CSV data
  • Use uid in each CSV row to identify records for updates

Features Implemented

1. CSV File Upload Support

  • Supports bulk record creation using CSV input files
  • Automatically maps CSV columns to system fields

2. Patch Mode (--patch)

  • Enables updating existing records using CSV input
  • Each row must include a uid
  • Supports partial updates (only provided fields are modified)

3. Improved CLI Workflow

  • Faster bulk operations
  • Reduced manual input
  • Automation-friendly data ingestion

Test Cases

1. Valid CSV Upload (Create Mode)

  • Input: Valid CSV with required fields
  • Expected: Records are created successfully

2. Valid CSV Patch Update

  • Input: CSV containing uid and fields to update
  • Expected: Matching records are updated successfully

3. Missing UID in Patch Mode

  • Input: CSV row without uid
  • Expected: Row is skipped or validation error is returned

4. Invalid CSV Format

  • Input: Malformed CSV file
  • Expected: Parsing error is returned

5. Partial Field Update

  • Input: CSV with only some fields provided
  • Expected: Only provided fields are updated; others remain unchanged

Closes

Closes #26 (closed)

How to Use the Feature

1. Create records using CSV

corpus-client update-files --csv < CSV file path >

### 2. Update existing records using Patch command

corpus-client patch < CSV file path >

Merge request reports