Skip to content

Migration from python-gitlab to glab-flow

Bikkumalla Sai Krishna requested to merge try2-glabflow into dev

What does this MR do?

This MR migrates the GitLab API client from python-gitlab to glabflow, a modern async-first GitLab API client. This involves a comprehensive rewrite of the infrastructure layer to support async operations natively.

Why is this change needed?

  1. Performance: glabflow provides native async support with better concurrency handling, eliminating the need for asyncio.to_thread() workarounds
  2. Better Rate Limit Handling: Built-in support for RateLimitError with automatic retry-after headers
  3. Connection Management: Proper connection pooling with configurable concurrency limits
  4. Memory Efficiency: Uses msgspec for faster JSON parsing (5-10x faster than standard json module)
  5. Modern Stack: Aligns with async-first design patterns used throughout the application

Key Changes

Dependencies (pyproject.toml)

  • Replace python-gitlab with glabflow>=0.1.0a4
  • Bump minimum Python version: >=3.11 → >=3.13
  • Add msgspec for high-performance JSON decoding
  • Add vulture>=2.14 for dead code detection
  • Bump uv>=0.11.6 Infrastructure Layer (infrastructure/gitlab/)
  • client.py:Complete rewrite of GitLabClient
    • glabflow.Client initialized as async context manager in dedicated background thread
    • Isolated event loop to avoid conflicts with Streamlit's execution model
    • New async methods: _async_get(), _async_get_paginated() using glabflow's native pagination
    • Updated retry logic to handle glabflow.RateLimitError, ServerError, TransientError, NotFoundError
    • Proper cleanup in del to exit glabflow context
  • api_helper.py:
    • Migrate get_user_from_token(), get_user_groups_by_token(), get_project_branches() to async
    • Add _decode() helper for msgspec JSON decoding
  • network.py:
    • Rewrite get_user_from_token() and get_user_groups() using async glabflow client
    • Use asyncio.run() for standalone async execution
  • files_reader.py:
    • read_file_content() now uses REST API with base64 decoding
    • list_all_files() uses paginated /repository/tree endpoint
  • parse_uvlock.py:
    • extract_dependencies_from_project() updated to use REST API
  • retry_helper.py:
    • Simplified retry logic for REST API calls
    • Removed python-gitlab specific exception handling
  • users.py:
    • Enhanced get_user_by_username() with search fallback for case-insensitive matching
  • projects.py:
    • get_project_with_retries() uses URL-encoded project paths Services Layer (services/batch/api_helper.py)
  • Updated all helper functions to accept gl_client and project_id instead of project objects
  • Functions updated:
    • get_project_branches()
    • list_all_files()
    • check_vscode_settings(), check_vscode_file_exists()
    • check_extensions_json_for_ruff()
    • list_markdown_files_in_folder()
    • check_templates_presence()
    • check_license_content()
    • check_project_compliance() Internationalization (messages.pot)
  • Regenerated with Babel 2.10.3 Architecture Details Async Client Management

glabflow Client runs in isolated background thread

self._loop = asyncio.new_event_loop() self._thread = threading.Thread(target=self._run_event_loop, daemon=True)

This prevents conflicts with Streamlit's event loop

gl._use_global_connector = False # Critical for Streamlit compatibility JSON Decoding with msgspec _JSON_DECODER = msgspec.json.Decoder() def _decode_json(data): if isinstance(data, (bytes, bytearray)): return _JSON_DECODER.decode(data) # Fast JSON parsing return data Coroutine Factory Pattern

Retry logic uses coroutine factory for deferred execution

async def safe_api_call_async(coro_factory, *args, **kwargs): return await coro_factory(*args, **kwargs) Breaking Changes

  • Minimum Python version is now 3.13
  • GitLabClient internal API changed- external consumers using client.projects.get() pattern must update to use REST API methods (_get(), _get_paginated())
  • Functions that previously accepted project objects now require gl_client and project_id parameters Testing Instructions
  1. Install dependencies: uv sync

  2. Verify Python version: python --version # Should be >= 3.13

  3. Run tests: uv run pytest

  4. Manual testing:

    • Connect to GitLab instance with valid token
    • Test user authentication (get_user_from_token)
    • Test group listing (get_user_groups_by_token)
    • Test project operations (branches, files, compliance checks)
    • Verify rate limiting handling works correctly Files Changed
  • pyproject.toml
  • messages.pot
  • src/gitlab_compliance_checker/infrastructure/gitlab/client.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/api_helper.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/network.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/files_reader.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/parse_uvlock.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/retry_helper.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/users.py
  • src/gitlab_compliance_checker/infrastructure/gitlab/projects.py
  • src/gitlab_compliance_checker/services/batch/api_helper.py

also fixes #64 (closed)

Edited by Ahlad Pataparla

Merge request reports

Loading