docs: update documentation with Celery/Redis and Makefile changes

- Update Makefile help sections with correct target names (platform-install)
- Add Celery infrastructure section to background-tasks.md
- Update installation.md with Celery setup instructions and environment vars
- Update index.md with comprehensive development commands including Celery
- Add Flower monitoring and task queue commands to docs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-11 18:56:46 +01:00
parent 63fd891f36
commit 0d204f39a8
4 changed files with 146 additions and 22 deletions

View File

@@ -4,6 +4,77 @@
This document defines the harmonized architecture for all background tasks in the application. Background tasks are long-running operations that execute asynchronously, allowing users to continue browsing while the task completes.
## Task Queue Infrastructure
Wizamart uses **Celery with Redis** for production-grade background task processing:
| Component | Purpose | Port |
|-----------|---------|------|
| **Celery Worker** | Executes background tasks | - |
| **Celery Beat** | Scheduled task scheduler | - |
| **Redis** | Message broker & result backend | 6379 |
| **Flower** | Web-based task monitoring | 5555 |
### Configuration
```bash
# Environment variables (.env)
REDIS_URL=redis://localhost:6379/0
USE_CELERY=true # false = use FastAPI BackgroundTasks
FLOWER_URL=http://localhost:5555
FLOWER_PASSWORD=changeme
```
### Running Celery
```bash
# Start Redis
docker compose up -d redis
# Start Celery worker (processes tasks)
make celery-worker
# Start Celery beat (scheduled tasks)
make celery-beat
# Start both together (development)
make celery-dev
# Start Flower monitoring
make flower
```
### Feature Flag: USE_CELERY
The system supports gradual migration via the `USE_CELERY` feature flag:
- **`USE_CELERY=false` (default)**: Uses FastAPI's `BackgroundTasks` for immediate execution without Redis dependency. Suitable for development.
- **`USE_CELERY=true`**: Routes tasks through Celery for persistent queuing, retries, and scheduling. Required for production.
### Task Routing
Tasks are routed to queues based on their characteristics:
| Queue | Tasks | Characteristics |
|-------|-------|-----------------|
| `default` | Product exports | Fast, non-blocking |
| `long_running` | Imports, code quality scans, tests | May take 10+ minutes |
| `scheduled` | Subscription maintenance | Triggered by Celery Beat |
### Celery Task ID Tracking
All background job models include a `celery_task_id` field for cross-referencing with Flower:
```python
class MarketplaceImportJob(Base):
celery_task_id = Column(String(255), nullable=True)
```
This enables:
- Deep linking to Flower for task details
- Task cancellation via Flower API
- Correlation between database records and Celery events
## Current State Analysis
| Task Type | Database Model | Status Values | Tracked in BG Tasks Page |