sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update documentation with Celery/Redis and Makefile changes

Browse Source 
- Update Makefile help sections with correct target names (platform-install)
- Add Celery infrastructure section to background-tasks.md
- Update installation.md with Celery setup instructions and environment vars
- Update index.md with comprehensive development commands including Celery
- Add Flower monitoring and task queue commands to docs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Samir Boulahtit
2026-01-11 18:56:46 +01:00
parent 63fd891f36
commit 0d204f39a8
4 changed files with 146 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
Makefile
View File

@@ -520,7 +520,7 @@ help:
@echo " migrate-up - Apply pending migrations"
@echo " migrate-down - Rollback last migration"
@echo " migrate-status - Show migration status"
@echo " install - First-time setup (validates config + migrate + init)"
@echo " platform-install - First-time setup (validates config + migrate + init)"
@echo " init-prod - Initialize platform (admin, CMS, pages, emails)"
@echo " seed-demo - Seed demo data (3 companies + vendors)"
@echo " seed-demo-minimal - Seed minimal demo (1 company + vendor)"
@@ -594,7 +594,7 @@ help-db:
@echo ""
@echo "FIRST-TIME INSTALLATION:"
@echo "──────────────────────────────────────────────────────────"
@echo " install - Complete installation wizard:"
@echo " platform-install - Complete installation wizard:"
@echo " - Validates .env configuration"
@echo " - Checks Stripe, Email, Security settings"
@echo " - Runs database migrations"
@@ -630,7 +630,7 @@ help-db:
@echo "TYPICAL FIRST-TIME SETUP (Development):"
@echo "──────────────────────────────────────────────────────────"
@echo " 1. cp .env.example .env # Configure environment"
@echo " 2. make install # Validates config + initializes platform"
@echo " 2. make platform-install # Validates config + initializes platform"
@echo " 3. make seed-demo # Add demo data (optional)"
@echo " 4. make dev # Start development server"
@echo ""
@@ -642,5 +642,5 @@ help-db:
@echo " - STRIPE_SECRET_KEY, STRIPE_PUBLISHABLE_KEY"
@echo " - Email provider settings (SMTP/SendGrid/Mailgun/SES)"
@echo " - ADMIN_PASSWORD (strong password)"
@echo " 2. make install # Validates + initializes"
@echo " 2. make platform-install # Validates + initializes"
@echo " 3. DO NOT run seed-demo in production!"

71
docs/architecture/background-tasks.md
View File

@@ -4,6 +4,77 @@
This document defines the harmonized architecture for all background tasks in the application. Background tasks are long-running operations that execute asynchronously, allowing users to continue browsing while the task completes.
## Task Queue Infrastructure
Wizamart uses **Celery with Redis** for production-grade background task processing:
| Component | Purpose | Port |
|-----------|---------|------|
| **Celery Worker** | Executes background tasks | - |
| **Celery Beat** | Scheduled task scheduler | - |
| **Redis** | Message broker & result backend | 6379 |
| **Flower** | Web-based task monitoring | 5555 |
### Configuration
```bash
# Environment variables (.env)
REDIS_URL=redis://localhost:6379/0
USE_CELERY=true # false = use FastAPI BackgroundTasks
FLOWER_URL=http://localhost:5555
FLOWER_PASSWORD=changeme
```
### Running Celery
```bash
# Start Redis
docker compose up -d redis
# Start Celery worker (processes tasks)
make celery-worker
# Start Celery beat (scheduled tasks)
make celery-beat
# Start both together (development)
make celery-dev
# Start Flower monitoring
make flower
```
### Feature Flag: USE_CELERY
The system supports gradual migration via the `USE_CELERY` feature flag:
- **`USE_CELERY=false` (default)**: Uses FastAPI's `BackgroundTasks` for immediate execution without Redis dependency. Suitable for development.
- **`USE_CELERY=true`**: Routes tasks through Celery for persistent queuing, retries, and scheduling. Required for production.
### Task Routing
Tasks are routed to queues based on their characteristics:
| Queue | Tasks | Characteristics |
|-------|-------|-----------------|
| `default` | Product exports | Fast, non-blocking |
| `long_running` | Imports, code quality scans, tests | May take 10+ minutes |
| `scheduled` | Subscription maintenance | Triggered by Celery Beat |
### Celery Task ID Tracking
All background job models include a `celery_task_id` field for cross-referencing with Flower:
```python
class MarketplaceImportJob(Base):
celery_task_id = Column(String(255), nullable=True)
```
This enables:
- Deep linking to Flower for task details
- Task cancellation via Flower API
- Correlation between database records and Celery events
## Current State Analysis
| Task Type | Database Model | Status Values | Tracked in BG Tasks Page |

66
docs/getting-started/installation.md
View File

@@ -6,8 +6,8 @@ This guide will help you set up the Wizamart Platform for development or product
Before you begin, ensure you have the following installed:
- **Python 3.10 or higher**
- **Docker and Docker Compose** (required for database)
- **Python 3.11 or higher**
- **Docker and Docker Compose** (required for database and Redis)
- **Git**
- **PostgreSQL client tools** (optional, for debugging)
@@ -97,27 +97,53 @@ docker run --name wizamart-postgres \
-d postgres:15
```
### 5. Initialize Database
### 5. Initialize Platform
```bash
# Run database migrations
python -m alembic upgrade head
# Option A: Full installation wizard (recommended)
make platform-install
# Create initial admin user (optional)
python scripts/create_admin.py
# Option B: Manual steps
make migrate-up # Run database migrations
make init-prod # Initialize admin, CMS, email templates
```
### 6. Run the Application
### 6. Start Background Task Queue (Optional)
For production or to test background tasks:
```bash
# Start Redis (required for Celery)
docker compose up -d redis
# Start Celery worker (in separate terminal)
make celery-worker
# Start Celery beat scheduler (in separate terminal, for scheduled tasks)
make celery-beat
# Or start both together for development
make celery-dev
```
!!! tip "Development without Celery"
Set `USE_CELERY=false` in `.env` to use FastAPI BackgroundTasks instead.
This is the default and doesn't require Redis.
### 7. Run the Application
```bash
# Start development server
uvicorn main:app --reload --host 0.0.0.0 --port 8000 # or make dev
make dev
```
The application will be available at:
- **API**: http://localhost:8000
- **Interactive API Docs**: http://localhost:8000/docs
- **Alternative API Docs**: http://localhost:8000/redoc
- **Admin Panel**: http://localhost:8000/admin
- **Flower** (if Celery enabled): http://localhost:5555
## Docker Setup
@@ -192,15 +218,31 @@ pytest -m integration
## Environment Variables
### Core Settings
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `DATABASE_URL` | PostgreSQL connection string | - | ✅ |
| `SECRET_KEY` | Application secret key | - | ✅ |
| `JWT_SECRET_KEY` | JWT token secret | - | ✅ |
| `ENVIRONMENT` | Environment (dev/staging/prod) | `development` | ❌ |
| `DEBUG` | Enable debug mode | `False` | ❌ |
| `LOG_LEVEL` | Logging level | `INFO` | ❌ |
| `CORS_ORIGINS` | Allowed CORS origins | `["*"]` | ❌ |
### Celery / Redis
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `REDIS_URL` | Redis connection string | `redis://localhost:6379/0` | ❌ |
| `USE_CELERY` | Enable Celery task queue | `false` | ❌ |
| `FLOWER_URL` | Flower dashboard URL | `http://localhost:5555` | ❌ |
| `FLOWER_PASSWORD` | Flower authentication password | `changeme` | ❌ |
### Stripe Billing
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `STRIPE_SECRET_KEY` | Stripe secret key | - | ❌ |
| `STRIPE_PUBLISHABLE_KEY` | Stripe publishable key | - | ❌ |
| `STRIPE_WEBHOOK_SECRET` | Stripe webhook secret | - | ❌ |
## Troubleshooting

23
docs/index.md
View File

@@ -354,26 +354,37 @@ Quick reference for common development tasks:
```bash
# Setup
make install-all # Install all dependencies
make install # Install production dependencies
make install-all # Install all dependencies (dev + test + docs)
make platform-install # First-time setup wizard (validates config)
make setup # Complete setup (install + migrate + seed)
# Development
make dev # Start development server
make docs-serve # Start documentation server
make dev # Start development server (port 8000)
make docs-serve # Start documentation server (port 9991)
# Database
make migrate-up # Run migrations
make migrate-create message="description" # Create new migration
make seed-demo # Seed demo data
make db-setup # Complete database setup
# Background Tasks (Celery)
make celery-worker # Start Celery worker
make celery-beat # Start Celery scheduler
make celery-dev # Start worker + beat together (dev)
make flower # Start Flower monitoring (port 5555)
# Testing
make test # Run all tests
make test-unit # Run unit tests only
make test-coverage # Run with coverage report
# Code Quality
make format # Format code (black + isort)
make lint # Run linters
make check # Format + lint
make format # Format code (ruff)
make lint # Run linters (ruff + mypy)
make arch-check # Validate architecture patterns
make check # Format + lint + verify imports
# Documentation
make docs-build # Build documentation