feat: add Celery/Redis task queue with feature flag support

Migrate background tasks from FastAPI BackgroundTasks to Celery with Redis
for persistent task queuing, retries, and scheduled jobs.

Key changes:
- Add Celery configuration with Redis broker/backend
- Create task dispatcher with USE_CELERY feature flag for gradual rollout
- Add Celery task wrappers for all background operations:
  - Marketplace imports
  - Letzshop historical imports
  - Product exports
  - Code quality scans
  - Test runs
  - Subscription scheduled tasks (via Celery Beat)
- Add celery_task_id column to job tables for Flower integration
- Add Flower dashboard link to admin background tasks page
- Update docker-compose.yml with worker, beat, and flower services
- Add Makefile targets: celery-worker, celery-beat, celery-dev, flower

When USE_CELERY=false (default), system falls back to FastAPI BackgroundTasks
for development without Redis dependency.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-11 17:35:16 +01:00
parent 879ac0caea
commit 2792414395
30 changed files with 2218 additions and 79 deletions

View File

@@ -11,13 +11,13 @@ DESCRIPTION=Multi-tenants multi-themes ecommerce application
VERSION=2.2.0
# =============================================================================
# DATABASE CONFIGURATION
# DATABASE CONFIGURATION (PostgreSQL required)
# =============================================================================
# For development (SQLite)
DATABASE_URL=sqlite:///./wizamart.db
# Default works with: docker-compose up -d db
DATABASE_URL=postgresql://wizamart_user:secure_password@localhost:5432/wizamart_db
# For production (PostgreSQL)
# DATABASE_URL=postgresql://username:password@localhost:5432/wizamart_db
# For production, use your PostgreSQL connection string:
# DATABASE_URL=postgresql://username:password@production-host:5432/wizamart_db
# =============================================================================
# ADMIN INITIALIZATION
@@ -145,3 +145,21 @@ SEED_DEMO_VENDORS=3
SEED_CUSTOMERS_PER_VENDOR=15
SEED_PRODUCTS_PER_VENDOR=20
SEED_ORDERS_PER_VENDOR=10
# =============================================================================
# CELERY / REDIS TASK QUEUE
# =============================================================================
# Redis connection URL (used for Celery broker and backend)
# Default works with: docker-compose up -d redis
REDIS_URL=redis://localhost:6379/0
# Enable Celery for background tasks (set to false to use FastAPI BackgroundTasks)
# Recommended: false for development, true for production
USE_CELERY=false
# Flower monitoring dashboard URL (for admin panel links)
FLOWER_URL=http://localhost:5555
# Flower basic authentication password
# ⚠️ CHANGE THIS IN PRODUCTION!
FLOWER_PASSWORD=changeme