feat: implement background task architecture for code quality scans
- Add status fields to ArchitectureScan model (status, started_at, completed_at, error_message, progress_message) - Create database migration for new status fields - Create background task function execute_code_quality_scan() - Update API to return 202 with job IDs and support polling - Add code quality scans to unified BackgroundTasksService - Integrate scans into background tasks API and page - Implement frontend polling with 3-second interval - Add progress banner showing scan status - Users can navigate away while scans run in background - Document the implementation in architecture docs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
316
.architecture-rules/background_tasks.yaml
Normal file
316
.architecture-rules/background_tasks.yaml
Normal file
@@ -0,0 +1,316 @@
|
||||
# Background Tasks Architecture Rules
|
||||
# ====================================
|
||||
# Enforces consistent patterns for all background tasks in the application.
|
||||
# See docs/architecture/background-tasks.md for full specification.
|
||||
|
||||
background_task_rules:
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# Model Rules
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
|
||||
- id: BG-001
|
||||
name: Background task models must have status field
|
||||
description: |
|
||||
All database models for background tasks must include a 'status' field
|
||||
with proper indexing for efficient querying.
|
||||
severity: error
|
||||
patterns:
|
||||
# Models that should have status field
|
||||
- file: "models/database/marketplace_import_job.py"
|
||||
must_contain: "status = Column"
|
||||
- file: "models/database/letzshop.py"
|
||||
must_contain: "status = Column"
|
||||
- file: "models/database/test_run.py"
|
||||
must_contain: "status = Column"
|
||||
- file: "models/database/architecture_scan.py"
|
||||
must_contain: "status = Column"
|
||||
suggestion: |
|
||||
Add: status = Column(String(30), nullable=False, default="pending", index=True)
|
||||
|
||||
- id: BG-002
|
||||
name: Background task models must have timestamp fields
|
||||
description: |
|
||||
All background task models must include created_at, started_at, and
|
||||
completed_at timestamp fields for proper lifecycle tracking.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "models/database/*_job.py"
|
||||
must_contain:
|
||||
- "started_at"
|
||||
- "completed_at"
|
||||
- file: "models/database/test_run.py"
|
||||
must_contain:
|
||||
- "started_at"
|
||||
- "completed_at"
|
||||
suggestion: |
|
||||
Add timestamp fields:
|
||||
created_at = Column(DateTime, default=lambda: datetime.now(UTC))
|
||||
started_at = Column(DateTime, nullable=True)
|
||||
completed_at = Column(DateTime, nullable=True)
|
||||
|
||||
- id: BG-003
|
||||
name: Background task models must have error_message field
|
||||
description: |
|
||||
All background task models must include an error_message field
|
||||
to store failure details.
|
||||
severity: warning
|
||||
patterns:
|
||||
- file: "models/database/marketplace_import_job.py"
|
||||
must_contain: "error_message"
|
||||
- file: "models/database/letzshop.py"
|
||||
must_contain: "error_message"
|
||||
- file: "models/database/test_run.py"
|
||||
must_contain: "error_message"
|
||||
suggestion: |
|
||||
Add: error_message = Column(Text, nullable=True)
|
||||
|
||||
- id: BG-004
|
||||
name: Background task models must have triggered_by field
|
||||
description: |
|
||||
All background task models must track who/what triggered the task
|
||||
for audit purposes.
|
||||
severity: warning
|
||||
patterns:
|
||||
- file: "models/database/*_job.py"
|
||||
must_contain: "triggered_by"
|
||||
- file: "models/database/test_run.py"
|
||||
must_contain: "triggered_by"
|
||||
suggestion: |
|
||||
Add: triggered_by = Column(String(100), nullable=True)
|
||||
Format: "manual:username", "scheduled", "api:client_id"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# Status Value Rules
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
|
||||
- id: BG-010
|
||||
name: Use standard status values
|
||||
description: |
|
||||
Background task status must use standard values:
|
||||
- pending: Task created, not started
|
||||
- running: Task actively executing
|
||||
- completed: Task finished successfully
|
||||
- failed: Task failed with error
|
||||
- completed_with_warnings: Task completed with non-fatal issues
|
||||
|
||||
Legacy values to migrate:
|
||||
- "processing" -> "running"
|
||||
- "fetching" -> "running"
|
||||
- "passed" -> "completed"
|
||||
- "error" -> "failed"
|
||||
severity: warning
|
||||
anti_patterns:
|
||||
- pattern: 'status.*=.*["\']processing["\']'
|
||||
message: "Use 'running' instead of 'processing'"
|
||||
- pattern: 'status.*=.*["\']fetching["\']'
|
||||
message: "Use 'running' instead of 'fetching'"
|
||||
suggestion: |
|
||||
Use the standard status enum:
|
||||
class TaskStatus(str, Enum):
|
||||
PENDING = "pending"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
COMPLETED_WITH_WARNINGS = "completed_with_warnings"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# API Rules
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
|
||||
- id: BG-020
|
||||
name: Task trigger endpoints must return job_id
|
||||
description: |
|
||||
Endpoints that trigger background tasks must return the job ID
|
||||
so the frontend can poll for status.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "app/api/v1/**/marketplace.py"
|
||||
endpoint_pattern: "@router.post.*import"
|
||||
must_return: "job_id"
|
||||
- file: "app/api/v1/**/tests.py"
|
||||
endpoint_pattern: "@router.post.*run"
|
||||
must_return: "run_id"
|
||||
suggestion: |
|
||||
Return a response like:
|
||||
{
|
||||
"job_id": job.id,
|
||||
"status": "pending",
|
||||
"message": "Task queued successfully"
|
||||
}
|
||||
|
||||
- id: BG-021
|
||||
name: Use FastAPI BackgroundTasks for async execution
|
||||
description: |
|
||||
All long-running tasks must use FastAPI's BackgroundTasks
|
||||
for async execution, not synchronous blocking calls.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "app/api/v1/**/*.py"
|
||||
must_contain: "BackgroundTasks"
|
||||
when_contains: ["import_", "export_", "run_scan", "execute_"]
|
||||
anti_patterns:
|
||||
- pattern: "subprocess\\.run.*wait.*True"
|
||||
message: "Don't wait synchronously for subprocesses in API handlers"
|
||||
- pattern: "time\\.sleep"
|
||||
file: "app/api/**/*.py"
|
||||
message: "Don't use time.sleep in API handlers"
|
||||
suggestion: |
|
||||
Use BackgroundTasks:
|
||||
async def trigger_task(background_tasks: BackgroundTasks):
|
||||
job = create_job_record(db)
|
||||
background_tasks.add_task(execute_task, job.id)
|
||||
return {"job_id": job.id}
|
||||
|
||||
- id: BG-022
|
||||
name: Tasks must be registered in BackgroundTasksService
|
||||
description: |
|
||||
All background task types must be registered in the
|
||||
BackgroundTasksService for unified monitoring.
|
||||
severity: warning
|
||||
patterns:
|
||||
- file: "app/services/background_tasks_service.py"
|
||||
must_reference:
|
||||
- "MarketplaceImportJob"
|
||||
- "LetzshopHistoricalImportJob"
|
||||
- "TestRun"
|
||||
- "ArchitectureScan"
|
||||
suggestion: |
|
||||
Add task model to TASK_MODELS in BackgroundTasksService:
|
||||
TASK_MODELS = {
|
||||
'product_import': MarketplaceImportJob,
|
||||
'order_import': LetzshopHistoricalImportJob,
|
||||
'test_run': TestRun,
|
||||
'code_quality_scan': ArchitectureScan,
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# Task Implementation Rules
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
|
||||
- id: BG-030
|
||||
name: Tasks must update status on start
|
||||
description: |
|
||||
Background task functions must set status to 'running'
|
||||
and record started_at timestamp at the beginning.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "app/tasks/*.py"
|
||||
must_contain:
|
||||
- 'status = "running"'
|
||||
- "started_at"
|
||||
suggestion: |
|
||||
At task start:
|
||||
job.status = "running"
|
||||
job.started_at = datetime.now(UTC)
|
||||
db.commit()
|
||||
|
||||
- id: BG-031
|
||||
name: Tasks must update status on completion
|
||||
description: |
|
||||
Background task functions must set appropriate final status
|
||||
and record completed_at timestamp.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "app/tasks/*.py"
|
||||
must_contain:
|
||||
- "completed_at"
|
||||
must_contain_one_of:
|
||||
- 'status = "completed"'
|
||||
- 'status = "failed"'
|
||||
suggestion: |
|
||||
On completion:
|
||||
job.status = "completed" # or "failed"
|
||||
job.completed_at = datetime.now(UTC)
|
||||
db.commit()
|
||||
|
||||
- id: BG-032
|
||||
name: Tasks must handle exceptions
|
||||
description: |
|
||||
Background tasks must catch exceptions, set status to 'failed',
|
||||
store error message, and optionally notify admins.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "app/tasks/*.py"
|
||||
must_contain:
|
||||
- "try:"
|
||||
- "except"
|
||||
- 'status = "failed"'
|
||||
- "error_message"
|
||||
suggestion: |
|
||||
Use try/except pattern:
|
||||
try:
|
||||
# Task logic
|
||||
job.status = "completed"
|
||||
except Exception as e:
|
||||
job.status = "failed"
|
||||
job.error_message = str(e)
|
||||
logger.error(f"Task failed: {e}")
|
||||
finally:
|
||||
job.completed_at = datetime.now(UTC)
|
||||
db.commit()
|
||||
|
||||
- id: BG-033
|
||||
name: Tasks must use separate database sessions
|
||||
description: |
|
||||
Background tasks must create their own database session
|
||||
since FastAPI request sessions are closed after response.
|
||||
severity: error
|
||||
patterns:
|
||||
- file: "app/tasks/*.py"
|
||||
must_contain: "SessionLocal()"
|
||||
anti_patterns:
|
||||
- pattern: "def.*task.*db.*Session"
|
||||
message: "Don't pass request db session to background tasks"
|
||||
suggestion: |
|
||||
Create session in task:
|
||||
async def my_task(job_id: int):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
# Task logic
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# Frontend Rules
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
|
||||
- id: BG-040
|
||||
name: Pages with tasks must show status on return
|
||||
description: |
|
||||
Pages that trigger background tasks must check for active/recent
|
||||
tasks on load and display appropriate status banners.
|
||||
severity: info
|
||||
patterns:
|
||||
- file: "static/admin/js/*.js"
|
||||
when_contains: "BackgroundTasks"
|
||||
must_contain:
|
||||
- "init"
|
||||
- "activeTask"
|
||||
suggestion: |
|
||||
In Alpine component init():
|
||||
async init() {
|
||||
// Check for active tasks for this page
|
||||
await this.checkActiveTask();
|
||||
if (this.activeTask) {
|
||||
this.startPolling();
|
||||
}
|
||||
}
|
||||
|
||||
- id: BG-041
|
||||
name: Use consistent polling interval
|
||||
description: |
|
||||
Polling for background task status should use 3-5 second intervals
|
||||
to balance responsiveness with server load.
|
||||
severity: info
|
||||
patterns:
|
||||
- file: "static/admin/js/*.js"
|
||||
when_contains: "setInterval"
|
||||
should_match: "setInterval.*[3-5]000"
|
||||
anti_patterns:
|
||||
- pattern: "setInterval.*1000"
|
||||
message: "1 second polling is too frequent"
|
||||
- pattern: "setInterval.*10000"
|
||||
message: "10 second polling may feel unresponsive"
|
||||
suggestion: |
|
||||
Use 3-5 second polling:
|
||||
this.pollInterval = setInterval(() => this.pollStatus(), 5000);
|
||||
Reference in New Issue
Block a user