Skip to content

Fix CI pipeline — type checking, linting, security, and test stages

Praneeth Ashish requested to merge fix/type-checking into develop

Description

What does this MR do?

This MR resolves all CI pipeline stage failures on the fix/type-checking branch. It fixes pyrefly type errors across the entire codebase, resolves bandit security scan findings, fixes ruff lint violations, and ensures all CI jobs have the dev dependencies they need.

Why is this needed?

The pipeline was failing on every stage after recent changes, blocking all merges. Root causes:

  1. CI jobs lacked uv sync --extra devdependencies in their before_script
  2. pyrefly type errors from SQLModel column access patterns, missing return types, and nullable field handling
  3. Bandit flagged real security issues (random for OTP, missing request timeout) plus false positives
  4. Ruff flagged a line-length violation

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • CI/CD improvement
  • Security fix
  • Refactoring (type annotations, no behavioral changes)

How Has This Been Tested?

All pipeline stages verified locally:

Stage Command Result
validate pre-commit run --all-files Passed
format ruff format --check . 212 files formatted
lint ruff check . All checks passed
type-check pyrefly check . --project-excludes tests/ test_async_validation.py 0 errors
security bandit -r app/ -c pyproject.toml No issues identified
test pytest -v --cov=. -n auto --cov-fail-under=70 1716 passed, 72.47% coverage

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • All existing tests pass
  • CI pipeline stages pass locally
  • No new warnings introduced
  • Changes are backward compatible

Key Changes by Area

CI/CD (.gitlab-ci.yml)

  • Added uv sync --frozen --extra devdependencies to all job before_script blocks
  • Added libmagic1 apt package to pytest job
  • Expanded pyrefly check scope

Security (app/services/otp_service.py, app/core/validators.py, app/utils/create_categories.py, pyproject.toml)

  • OTP generation now uses secrets.randbelow (cryptographically secure)
  • Username suffix generation uses secrets.randbelow
  • Added timeout=30 to HTTP request
  • Configured bandit skip list for documented false positives

Type Safety (across 50 files)

  • SQLModel col() for typed column references
  • Explicit return type annotations on all scripts
  • cast() and null guards where type inference fails
  • Literal[...] types replacing deprecated const=True
  • Typed dict aliases in backfill modules

Models & Schemas

  • TokenResponse.username optional
  • RoleEnum.system added
  • sa_column fix for ARRAY fields
  • Pydantic v2 validator arg fixes

Services

  • Null guards for Record.uid in history and streak services
  • list() wrapper on .exec().all() results
  • Richer daily activity counts from streak service

Screenshots

image

bandit: No issues identified. Exit code: 0
ruff: All checks passed! Exit code: 0
pyrefly: 0 errors. Exit code: 0
pytest: 1716 passed, 254 skipped. Coverage: 72.47%. Exit code: 0

Related Issues

Closes #136 (closed)

Notes for Reviewers

  • Focus review on the col() migration pattern — ensure all SQLModel queries still work as expected
  • The upload_file_to_hetzner change from async to sync is intentional (MinIO client is synchronous); verify the caller in records.py endpoint
  • Bandit skips in pyproject.toml are for false positives only — real issues (B311, B113) were fixed in code
  • Backfill module refactoring is purely for type safety — no behavioral changes
Edited by Praneeth Ashish

Merge request reports