Skip to content

Bug : MRs and Commits sometimes fail to fetch after a team edit, but then reappear after a rerun in the team-leaderboard

Root Cause Analysis: Intermittent Data Fetching

This report explains why MRs and Commits sometimes fail to fetch after a team edit, but then reappear after a rerun.

1. Technical Root Cause: Client/Thread Proliferation

The primary issue is how the GitLabClient is managed within the Streamlit lifecycle.

The "Zombie Thread" Problem

Every time you interact with the UI (editing a user, clicking a button, or uploading a file), Streamlit reruns the script from top to bottom. In ui/main.py, a new GitLabClient is created on every single rerun:

# From ui/main.py (Line 47)
client = GitLabClient(gitlab_url, gitlab_token, ssl_verify=ssl_verify)

Inside the GitLabClient constructor:

  1. A new background event loop is created.
  2. A new background thread is started to run that loop.
  3. A new async HTTP session (glabflow) is initialized.

Result: If you edit a team multiple times, you may have 10-20 "zombie" background threads all running concurrently, possibly competing for the same network resources or hitting GitLab's connection limits. This causes transient "timeouts" where one request fails (returning 0 items) while another succeeds later.

2. Resource Contention in Parallel Processing

The application uses nested parallelism:

  1. Leaderboard loops through teams.
  2. process_batch_users uses 10 threads to process users.
  3. Each user's processing uses 5 more threads to fetch Commits, MRs, and Issues simultaneously.

When combined with the "Zombie Thread" issue above, your machine may be attempting 50-100+ concurrent network connections to GitLab. When this overload happens:

  • Some threads receive a "Connection Reset" or "Timeout".
  • The code handles these exceptions by returning an empty list [] to prevent a total crash.
  • This is why you see "0 MRs" or "0 Commits" intermittently.

3. Recommended Fixes

To stabilize the application, we should implement the following architectural changes:

  1. Client Persistence: Store the GitLabClient in st.session_state so only one instance (and one background thread) exists for the entire user session.
  2. Concurrency Control: Reduce the number of nested threads and rely more on the async semaphore already present in the client.
  3. Global Connection Pool: Ensure the background glabflow client uses a managed connection pool that survives reruns.

[!TIP] I have updated the implementation plan to include these stability fixes alongside the strict matching rules.