refactor(queue): remove KYP from queue workflow and transition vitals directly to waiting

Merge Request

Overview

Remove "Know Your Patient (KYP)" from the consultation queue workflow. KYP is no longer a mandatory queue stage between VITALS and WAITING. KYP data fields remain fully intact as optional patient profile data that can be captured during or after registration.

What does this MR do and why?

Motivation: KYP was incorrectly treated as a recurring operational queue stage (VITALS → KYP → WAITING), forcing every patient through a KYP step even when KYP data was already captured or not applicable. This created unnecessary bottlenecks in the queue workflow.

Approach: Removed KYP from the queue transition map so the default flow is now VITALS → WAITING. Added a safe drain transition (know_your_patient → waiting) so any legacy records stuck in KYP status can progress. KYP remains available as a QueueStatusEnum value for backward compatibility but is no longer used in active queue logic.

Trade-offs: Analytics, websocket managers, and doctor/medicine services still reference know_your_patient in their status filter lists. These references are now harmless (no records will match) and were left untouched to minimize blast radius. They can be cleaned up in a follow-up MR.

Changes Made

Files Modified

File	Change
app/services/consultation_queue_service.py	Removed KYP from VALID_STATUS_TRANSITIONS main path. Added know_your_patient → waiting as a drain-only transition for legacy records. Updated _get_initial_queue_status to return waiting instead of know_your_patient. Updated docstrings.
app/services/patient_service.py	Renamed _auto_transition_queue_to_know_your_patient → _auto_transition_queue_to_waiting. Auto-transition after vitals now targets waiting.
app/api/v1/routes/consultation_queue_routes.py	Removed backward-compat normalization block that converted vitals → waiting into vitals → know_your_patient (would have caused runtime errors). Replaced unreachable KYP permission checks with a direct vitals → waiting permission check.
app/api/v1/routes/patient_routes.py	Removed know_your_patient from queue_status_order and status_order lists in the patient status endpoint.
tests/test_api/test_consultation_queue_routes.py	Removed obsolete KYP transition tests (~82 lines of dead test code).
tests/test_services/test_consultation_queue_service.py	Updated test_update_queue_status_to_waiting_sets_estimated_time to start from vitals status with proper vitals mock.
tests/test_services/test_patient_service.py	Updated function name references and expected status value from know_your_patient to waiting.
tests/test_api/test_patient_routes.py	Updated 4 status progression tests to use waiting instead of know_your_patient. Removed KYP assertions from status response checks.

Technical Details

Before:

vitals_waiting → vitals → know_your_patient → waiting → in_consultation → ...

After:

vitals_waiting → vitals → waiting → in_consultation → ...
know_your_patient → waiting  (drain-only path for legacy DB records)
![Screenshot_from_2026-04-10_09-53-01](/uploads/26869835da3509782478770841b5a427/Screenshot_from_2026-04-10_09-53-01.png)
## Documentation

- [ ] README.md updated (if setup steps changed)
- [ ] `.env.example` updated (if new env vars added)
- [ ] API documentation updated (docstrings, OpenAPI specs)
- [ ] CHANGELOG.md will be updated (if applicable)
- [x] Code comments explain complex logic

## Known Limitations / Technical Debt

1. **`know_your_patient` still in status filter lists** — Analytics, websocket managers, doctor/medicine services still include `know_your_patient` in their query filters. No records will match since it's unreachable. Safe to clean up in a follow-up MR.

2. **Analytics `patients_kyp_done` will always return 0** — Since KYP is no longer a reachable queue status, the analytics count query will always return zero. The field remains in the response schema for backward compatibility.

3. **No DB migration for legacy records** — Existing records with `status = 'know_your_patient'` in the database are handled via the drain transition but no migration script exists to backfill them to `waiting`. Recommended follow-up.

## Additional Notes

- `QueueStatusEnum.know_your_patient` is intentionally kept in the enum definition to prevent `AttributeError` crashes in any code that references it directly (analytics, websockets, etc.).
- The backward-compat normalization block in `consultation_queue_routes.py` was removed because it would have caused runtime `ValueError` exceptions (normalizing to a status that no longer exists in transitions).
- KYP patient data (`allergies`, `life_style`, `work_profile`, `chronic_history`) is completely unaffected by this change.

---

## MR Acceptance Checklist

### Quality & Correctness
- [ ] Code works as intended and solves the stated problem
- [ ] No bugs introduced (existing functionality not broken)
- [ ] Edge cases handled appropriately

### Maintainability
- [ ] Code is readable and well-organized
- [ ] Code is testable and well-tested
- [ ] Follows project patterns and conventions

### Acceptance Review
- [ ] Reviewed by at least 1 teammate
- [ ] Reviewed by product owner

**Edge case handling:** If a database record has `status = 'know_your_patient'` (legacy or manually set), it can now transition to `waiting` via the drain entry. It cannot be reached from any other status since `vitals → know_your_patient` is no longer a valid transition.

## Type of Change

- [x] 💥 Breaking change (fix or feature that would cause existing functionality to change)
- [x] ♻️ Refactor (no functional changes)
- [x] 🧪 Test update
- [x] 🗑️ Deprecation (removing deprecated queue workflow behavior)

## Related Issues / References

<related issue #95 >

## Screenshots or Screen Recordings

### Queue Transitions (verified via Python)

vitals -> ['waiting'] know_your_patient -> ['waiting'] (drain path for legacy records) know_your_patient in any active transition? False

### Test Results

186 passed, 53 warnings in 0.88s

## How to Validate Locally

### Previous Behavior
- After saving vitals, queue auto-transitioned: `vitals → know_your_patient`
- Volunteers had to manually move patients: `know_your_patient → waiting`
- API status endpoint included `know_your_patient` in the workflow progression

### Changes Made
- Queue auto-transition now goes: `vitals → waiting`
- `_get_initial_queue_status` returns `waiting` when vitals already exist
- Removed dead normalization code that would have crashed at runtime
- Patient status endpoint no longer lists KYP in workflow progression

### New Behavior
- Save vitals → queue moves to `waiting` automatically
- No KYP step required in the queue workflow
- KYP patient data fields (`allergies`, `life_style`, `work_profile`, `chronic_history`) remain fully functional for patient registration/updates
- Legacy records stuck in `know_your_patient` status can transition to `waiting`

### Validate Steps
```bash
# 1. Start Docker services
cd /home/suma/ehrs-fastapi
POSTGRES_SERVER=db docker compose up -d

# 2. Verify API is running
curl http://localhost:8000/docs  # Should return 200

# 3. Check queue transitions
.venv/bin/python -c "
from app.services.consultation_queue_service import VALID_STATUS_TRANSITIONS
from app.models.consultation_queue import QueueStatusEnum as Q
print('vitals ->', [t.value for t in VALID_STATUS_TRANSITIONS[Q.vitals]])
print('kyp drain ->', VALID_STATUS_TRANSITIONS.get(Q.know_your_patient, []))
"
# Expected: vitals -> ['waiting'], kyp drain -> [<QueueStatusEnum.waiting: 'waiting'>]

# 4. Run tests
.venv/bin/python -m pytest tests/test_services/test_consultation_queue_service.py \
    tests/test_services/test_patient_service.py \
    tests/test_api/test_consultation_queue_routes.py \
    tests/test_api/test_patient_routes.py -v

Testing Done

• Unit tests added/updated
• API endpoint tests passing

Test Cases Covered:

Scenario	Expected Result	Status
Save vitals → auto-transition	Queue moves to waiting (not KYP)	✅
Queue creation with existing vitals	Starts at waiting	✅
Manual vitals → waiting transition	Succeeds with proper permission	✅
Patient status API response	Does not include know_your_patient	✅
Legacy KYP record transition	Can drain to waiting	✅
KYP patient data update	Still works independently	✅

Test Commands Run:

# Run all relevant tests
.venv/bin/python -m pytest tests/test_services/test_consultation_queue_service.py \
    tests/test_services/test_patient_service.py \
    tests/test_api/test_consultation_queue_routes.py \
    tests/test_api/test_patient_routes.py -v

# Result: 186 passed, 53 warnings

Code Quality Checklist

Code Standards

• Code follows project conventions (naming, structure, formatting)
• No debug statements or commented-out code left
• No unused imports, variables, or functions
• No duplicate code (DRY principle followed)
• Type hints are properly defined
• Ruff checks pass (no new errors introduced)

Python & FastAPI Best Practices

• Functions follow single-responsibility principle
• Dependency injection used appropriately
• Pydantic models used for request/response validation
• SQLAlchemy queries are optimized
• Error handling is comprehensive

API Design

• RESTful conventions followed
• Proper HTTP status codes returned
• Input validation implemented
• Authentication/authorization enforced
• Role-based access control maintained

Database & Migrations

• No schema changes required (existing fields reused)
• No raw SQL queries (using SQLAlchemy ORM)
• Data integrity constraints maintained
• Database migration needed for existing know_your_patient records (follow-up)

Security

• No sensitive data logged
• SQL injection prevention verified (ORM used)
• Authentication tokens handled securely

Error Handling

• Errors are caught and handled gracefully
• User-friendly error messages returned
• HTTP error responses follow API standards

Documentation

• README.md updated (if setup steps changed)
• .env.example updated (if new env vars added)
• API documentation updated (docstrings, OpenAPI specs)
• CHANGELOG.md will be updated (if applicable)
• Code comments explain complex logic

Known Limitations / Technical Debt

1. know_your_patient still in status filter lists — Analytics, websocket managers, doctor/medicine services still include know_your_patient in their query filters. No records will match since it's unreachable. Safe to clean up in a follow-up MR.

2. Analytics patients_kyp_done will always return 0 — Since KYP is no longer a reachable queue status, the analytics count query will always return zero. The field remains in the response schema for backward compatibility.

3. No DB migration for legacy records — Existing records with status = 'know_your_patient' in the database are handled via the drain transition but no migration script exists to backfill them to waiting. Recommended follow-up.

Additional Notes

• QueueStatusEnum.know_your_patient is intentionally kept in the enum definition to prevent AttributeError crashes in any code that references it directly (analytics, websockets, etc.).
• The backward-compat normalization block in consultation_queue_routes.py was removed because it would have caused runtime ValueError exceptions (normalizing to a status that no longer exists in transitions).
• KYP patient data (allergies, life_style, work_profile, chronic_history) is completely unaffected by this change.

---

MR Acceptance Checklist

Quality & Correctness

• Code works as intended and solves the stated problem
• No bugs introduced (existing functionality not broken)
• Edge cases handled appropriately

Maintainability

• Code is readable and well-organized
• Code is testable and well-tested
• Follows project patterns and conventions

Acceptance Review

• Reviewed by at least 1 teammate
• Reviewed by product owner

closes #95 (closed)