feat: add Celery/Redis task queue with feature flag support
Migrate background tasks from FastAPI BackgroundTasks to Celery with Redis for persistent task queuing, retries, and scheduled jobs. Key changes: - Add Celery configuration with Redis broker/backend - Create task dispatcher with USE_CELERY feature flag for gradual rollout - Add Celery task wrappers for all background operations: - Marketplace imports - Letzshop historical imports - Product exports - Code quality scans - Test runs - Subscription scheduled tasks (via Celery Beat) - Add celery_task_id column to job tables for Flower integration - Add Flower dashboard link to admin background tasks page - Update docker-compose.yml with worker, beat, and flower services - Add Makefile targets: celery-worker, celery-beat, celery-dev, flower When USE_CELERY=false (default), system falls back to FastAPI BackgroundTasks for development without Redis dependency. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
16
app/tasks/celery_tasks/__init__.py
Normal file
16
app/tasks/celery_tasks/__init__.py
Normal file
@@ -0,0 +1,16 @@
|
||||
# app/tasks/celery_tasks/__init__.py
|
||||
"""
|
||||
Celery task modules for Wizamart.
|
||||
|
||||
This package contains Celery task wrappers for background processing:
|
||||
- marketplace: Product import tasks
|
||||
- letzshop: Historical import tasks
|
||||
- subscription: Scheduled subscription management
|
||||
- export: Product export tasks
|
||||
- code_quality: Code quality scan tasks
|
||||
- test_runner: Test execution tasks
|
||||
"""
|
||||
|
||||
from app.tasks.celery_tasks.base import DatabaseTask
|
||||
|
||||
__all__ = ["DatabaseTask"]
|
||||
Reference in New Issue
Block a user