feat: add Sentry, Cloudflare R2, and CloudFlare CDN integrations

Production quick wins for improved observability and scalability:

Sentry Error Tracking:
- Add sentry-sdk[fastapi] dependency
- Initialize Sentry in main.py with FastAPI/SQLAlchemy integrations
- Add Celery integration for background task error tracking
- Feature-flagged via SENTRY_DSN (disabled when empty)

Cloudflare R2 Storage:
- Add boto3 dependency for S3-compatible API
- Create storage_service.py with StorageBackend abstraction
- LocalStorageBackend for development (default)
- R2StorageBackend for production cloud storage
- Feature-flagged via STORAGE_BACKEND setting

CloudFlare CDN/Proxy:
- Create middleware/cloudflare.py for CF header handling
- Extract real client IP from CF-Connecting-IP
- Support CF-IPCountry for geo features
- Feature-flagged via CLOUDFLARE_ENABLED setting

Documentation:
- Add docs/deployment/cloudflare.md setup guide
- Update infrastructure.md with dev vs prod requirements
- Add enterprise upgrade checklist for scaling beyond 1000 users
- Update installation.md with new environment variables

All features are optional and disabled by default for development.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-11 19:44:59 +01:00
parent 12b79c1ff0
commit ff5b395cdd
13 changed files with 1062 additions and 47 deletions

View File

@@ -163,3 +163,38 @@ FLOWER_URL=http://localhost:5555
# Flower basic authentication password # Flower basic authentication password
# ⚠️ CHANGE THIS IN PRODUCTION! # ⚠️ CHANGE THIS IN PRODUCTION!
FLOWER_PASSWORD=changeme FLOWER_PASSWORD=changeme
# =============================================================================
# SENTRY ERROR TRACKING
# =============================================================================
# Get your DSN from https://sentry.io (free tier available)
# Leave empty to disable Sentry
SENTRY_DSN=
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
# =============================================================================
# CLOUDFLARE R2 STORAGE
# =============================================================================
# Storage backend: "local" (default) or "r2" for Cloudflare R2
# Set to "r2" for production to enable cloud storage
STORAGE_BACKEND=local
# Cloudflare R2 credentials (required when STORAGE_BACKEND=r2)
# Get these from Cloudflare Dashboard > R2 > Manage R2 API Tokens
R2_ACCOUNT_ID=
R2_ACCESS_KEY_ID=
R2_SECRET_ACCESS_KEY=
R2_BUCKET_NAME=wizamart-media
# Public URL for R2 bucket (optional - for custom domain)
# If not set, uses Cloudflare's default R2 public URL
# Example: https://media.yoursite.com
R2_PUBLIC_URL=
# =============================================================================
# CLOUDFLARE CDN / PROXY
# =============================================================================
# Set to true when your domain is proxied through CloudFlare
# This enables proper handling of CF-Connecting-IP and other CloudFlare headers
CLOUDFLARE_ENABLED=false

View File

@@ -7,16 +7,33 @@ It includes:
- Task routing to separate queues (default, long_running, scheduled) - Task routing to separate queues (default, long_running, scheduled)
- Celery Beat schedule for periodic tasks - Celery Beat schedule for periodic tasks
- Task retry policies - Task retry policies
- Sentry integration for error tracking
""" """
import os import os
import sentry_sdk
from celery import Celery from celery import Celery
from celery.schedules import crontab from celery.schedules import crontab
from sentry_sdk.integrations.celery import CeleryIntegration
# Redis URL from environment or default # Redis URL from environment or default
REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0") REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")
# =============================================================================
# SENTRY INITIALIZATION FOR CELERY WORKERS
# =============================================================================
# Celery workers run in separate processes, so Sentry must be initialized here too
SENTRY_DSN = os.getenv("SENTRY_DSN")
if SENTRY_DSN:
sentry_sdk.init(
dsn=SENTRY_DSN,
environment=os.getenv("SENTRY_ENVIRONMENT", "development"),
traces_sample_rate=float(os.getenv("SENTRY_TRACES_SAMPLE_RATE", "0.1")),
integrations=[CeleryIntegration()],
send_default_pii=True,
)
# Create Celery application # Create Celery application
celery_app = Celery( celery_app = Celery(
"wizamart", "wizamart",

View File

@@ -187,6 +187,28 @@ class Settings(BaseSettings):
flower_url: str = "http://localhost:5555" flower_url: str = "http://localhost:5555"
flower_password: str = "changeme" # CHANGE IN PRODUCTION! flower_password: str = "changeme" # CHANGE IN PRODUCTION!
# =============================================================================
# SENTRY ERROR TRACKING
# =============================================================================
sentry_dsn: str | None = None # Set to enable Sentry
sentry_environment: str = "development" # development, staging, production
sentry_traces_sample_rate: float = 0.1 # 10% of transactions for performance monitoring
# =============================================================================
# CLOUDFLARE R2 STORAGE
# =============================================================================
storage_backend: str = "local" # "local" or "r2"
r2_account_id: str | None = None
r2_access_key_id: str | None = None
r2_secret_access_key: str | None = None
r2_bucket_name: str = "wizamart-media"
r2_public_url: str | None = None # Custom domain for public access (e.g., https://media.yoursite.com)
# =============================================================================
# CLOUDFLARE CDN / PROXY
# =============================================================================
cloudflare_enabled: bool = False # Set to True when using CloudFlare proxy
model_config = {"env_file": ".env"} model_config = {"env_file": ".env"}

View File

@@ -0,0 +1,295 @@
# app/services/storage_service.py
"""
Storage abstraction service for file uploads.
Provides a unified interface for file storage with support for:
- Local filesystem (default, development)
- Cloudflare R2 (production, S3-compatible)
Usage:
from app.services.storage_service import get_storage_backend
storage = get_storage_backend()
url = await storage.upload("path/to/file.jpg", file_bytes, "image/jpeg")
await storage.delete("path/to/file.jpg")
"""
import logging
from abc import ABC, abstractmethod
from pathlib import Path
from app.core.config import settings
logger = logging.getLogger(__name__)
class StorageBackend(ABC):
"""Abstract base class for storage backends."""
@abstractmethod
async def upload(self, file_path: str, content: bytes, content_type: str) -> str:
"""
Upload a file to storage.
Args:
file_path: Relative path where file should be stored
content: File content as bytes
content_type: MIME type of the file
Returns:
Public URL to access the file
"""
pass
@abstractmethod
async def delete(self, file_path: str) -> bool:
"""
Delete a file from storage.
Args:
file_path: Relative path of file to delete
Returns:
True if file was deleted, False if not found
"""
pass
@abstractmethod
def get_url(self, file_path: str) -> str:
"""
Get the public URL for a file.
Args:
file_path: Relative path of the file
Returns:
Public URL to access the file
"""
pass
@abstractmethod
async def exists(self, file_path: str) -> bool:
"""
Check if a file exists in storage.
Args:
file_path: Relative path of the file
Returns:
True if file exists
"""
pass
class LocalStorageBackend(StorageBackend):
"""Local filesystem storage backend."""
def __init__(self, base_dir: str = "uploads"):
"""
Initialize local storage backend.
Args:
base_dir: Base directory for file storage (relative to project root)
"""
self.base_dir = Path(base_dir)
self.base_dir.mkdir(parents=True, exist_ok=True)
logger.info(f"LocalStorageBackend initialized with base_dir: {self.base_dir}")
async def upload(self, file_path: str, content: bytes, content_type: str) -> str:
"""Upload file to local filesystem."""
full_path = self.base_dir / file_path
# Ensure parent directory exists
full_path.parent.mkdir(parents=True, exist_ok=True)
# Write file
full_path.write_bytes(content)
logger.debug(f"Uploaded to local: {file_path} ({len(content)} bytes)")
return self.get_url(file_path)
async def delete(self, file_path: str) -> bool:
"""Delete file from local filesystem."""
full_path = self.base_dir / file_path
if full_path.exists():
full_path.unlink()
logger.debug(f"Deleted from local: {file_path}")
# Clean up empty parent directories
self._cleanup_empty_dirs(full_path.parent)
return True
return False
def get_url(self, file_path: str) -> str:
"""Get URL for local file (served via /uploads mount)."""
return f"/uploads/{file_path}"
async def exists(self, file_path: str) -> bool:
"""Check if file exists locally."""
return (self.base_dir / file_path).exists()
def _cleanup_empty_dirs(self, dir_path: Path) -> None:
"""Remove empty directories up to base_dir."""
try:
while dir_path != self.base_dir and dir_path.exists():
if not any(dir_path.iterdir()):
dir_path.rmdir()
dir_path = dir_path.parent
else:
break
except OSError:
pass
class R2StorageBackend(StorageBackend):
"""Cloudflare R2 storage backend (S3-compatible)."""
def __init__(self):
"""Initialize R2 storage backend."""
import boto3
from botocore.config import Config
if not all([
settings.r2_account_id,
settings.r2_access_key_id,
settings.r2_secret_access_key,
]):
raise ValueError(
"R2 storage requires R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, "
"and R2_SECRET_ACCESS_KEY environment variables"
)
# R2 endpoint URL
endpoint_url = f"https://{settings.r2_account_id}.r2.cloudflarestorage.com"
# Configure boto3 client for R2
self.client = boto3.client(
"s3",
endpoint_url=endpoint_url,
aws_access_key_id=settings.r2_access_key_id,
aws_secret_access_key=settings.r2_secret_access_key,
config=Config(
signature_version="s3v4",
retries={"max_attempts": 3, "mode": "adaptive"},
),
)
self.bucket_name = settings.r2_bucket_name
self.public_url = settings.r2_public_url
logger.info(
f"R2StorageBackend initialized: bucket={self.bucket_name}, "
f"public_url={self.public_url or 'default'}"
)
async def upload(self, file_path: str, content: bytes, content_type: str) -> str:
"""Upload file to R2."""
try:
self.client.put_object(
Bucket=self.bucket_name,
Key=file_path,
Body=content,
ContentType=content_type,
)
logger.debug(f"Uploaded to R2: {file_path} ({len(content)} bytes)")
return self.get_url(file_path)
except Exception as e:
logger.error(f"R2 upload failed for {file_path}: {e}")
raise
async def delete(self, file_path: str) -> bool:
"""Delete file from R2."""
try:
# Check if file exists first
if not await self.exists(file_path):
return False
self.client.delete_object(
Bucket=self.bucket_name,
Key=file_path,
)
logger.debug(f"Deleted from R2: {file_path}")
return True
except Exception as e:
logger.error(f"R2 delete failed for {file_path}: {e}")
return False
def get_url(self, file_path: str) -> str:
"""Get public URL for R2 file."""
if self.public_url:
# Use custom domain
return f"{self.public_url.rstrip('/')}/{file_path}"
else:
# Use default R2 public URL pattern
# Note: Bucket must have public access enabled
return f"https://{self.bucket_name}.{settings.r2_account_id}.r2.dev/{file_path}"
async def exists(self, file_path: str) -> bool:
"""Check if file exists in R2."""
try:
self.client.head_object(Bucket=self.bucket_name, Key=file_path)
return True
except self.client.exceptions.ClientError as e:
if e.response.get("Error", {}).get("Code") == "404":
return False
raise
# =============================================================================
# STORAGE BACKEND FACTORY
# =============================================================================
_storage_backend: StorageBackend | None = None
def get_storage_backend() -> StorageBackend:
"""
Get the configured storage backend instance.
Returns:
Storage backend based on STORAGE_BACKEND setting
Raises:
ValueError: If storage backend is misconfigured
"""
global _storage_backend
if _storage_backend is not None:
return _storage_backend
backend_type = settings.storage_backend.lower()
if backend_type == "r2":
_storage_backend = R2StorageBackend()
elif backend_type == "local":
_storage_backend = LocalStorageBackend()
else:
raise ValueError(f"Unknown storage backend: {backend_type}")
return _storage_backend
def reset_storage_backend() -> None:
"""Reset the storage backend (useful for testing)."""
global _storage_backend
_storage_backend = None
# =============================================================================
# PUBLIC API
# =============================================================================
__all__ = [
"StorageBackend",
"LocalStorageBackend",
"R2StorageBackend",
"get_storage_backend",
"reset_storage_backend",
]

View File

@@ -0,0 +1,299 @@
# CloudFlare Setup Guide
This guide covers setting up CloudFlare for Wizamart, including CDN, proxy, WAF, and R2 storage.
## Overview
CloudFlare provides:
| Feature | Benefit |
|---------|---------|
| **CDN** | Global edge caching for static assets |
| **Proxy** | Hide origin IP, DDoS protection |
| **WAF** | Web Application Firewall (basic rules free) |
| **R2** | S3-compatible object storage (~$5/mo) |
| **SSL** | Free SSL certificates |
---
## Quick Start
### 1. Add Your Domain to CloudFlare
1. Create a CloudFlare account at [cloudflare.com](https://cloudflare.com)
2. Add your domain and follow the setup wizard
3. Update your domain's nameservers to CloudFlare's
### 2. Configure DNS Records
Create these DNS records (with proxy enabled - orange cloud):
| Type | Name | Content | Proxy |
|------|------|---------|-------|
| A | @ | Your server IP | ✅ Proxied |
| A | www | Your server IP | ✅ Proxied |
| A | api | Your server IP | ✅ Proxied |
| CNAME | media | your-bucket.r2.dev | ✅ Proxied |
### 3. Enable CloudFlare in Wizamart
```env
# .env
CLOUDFLARE_ENABLED=true
```
This enables the CloudFlare middleware to:
- Extract real client IPs from `CF-Connecting-IP`
- Read client country from `CF-IPCountry`
- Track requests via `CF-Ray` header
---
## SSL/TLS Configuration
### Recommended: Full (Strict) Mode
1. Go to **SSL/TLS** > **Overview**
2. Select **Full (strict)**
3. This requires a valid SSL certificate on your origin server
### Origin Certificates
For the origin server, you can use:
1. **Let's Encrypt** (recommended for VPS):
```bash
sudo certbot --nginx -d yourdomain.com
```
2. **CloudFlare Origin Certificate** (15-year free cert):
- Go to **SSL/TLS** > **Origin Server**
- Create Certificate
- Install on your server
---
## Caching Configuration
### Page Rules for Static Assets
Create page rules for optimal caching:
**Rule 1: Static Assets**
- URL: `*yourdomain.com/static/*`
- Setting: Cache Level → Cache Everything
- Setting: Edge Cache TTL → 1 month
**Rule 2: Uploads**
- URL: `*yourdomain.com/uploads/*`
- Setting: Cache Level → Cache Everything
- Setting: Edge Cache TTL → 1 week
**Rule 3: API (No Cache)**
- URL: `*yourdomain.com/api/*`
- Setting: Cache Level → Bypass
### Cache Rules (New Interface)
Or use the newer Cache Rules:
```
Expression: (http.request.uri.path starts with "/static/")
Action: Cache eligible → Override → 30 days
```
---
## Cloudflare R2 Storage
### Create R2 Bucket
1. Go to **R2** in CloudFlare dashboard
2. Click **Create bucket**
3. Name: `wizamart-media`
4. Location: Choose region closest to your users
### Create API Token
1. Go to **R2** > **Manage R2 API Tokens**
2. Create new token with:
- Permission: Object Read & Write
- Bucket: Select your bucket
3. Save the **Access Key ID** and **Secret Access Key**
### Configure Wizamart
```env
# .env
STORAGE_BACKEND=r2
R2_ACCOUNT_ID=your_account_id
R2_ACCESS_KEY_ID=your_access_key
R2_SECRET_ACCESS_KEY=your_secret_key
R2_BUCKET_NAME=wizamart-media
```
### Enable Public Access (Optional)
For direct public access to uploaded files:
1. Go to **R2** > Your bucket > **Settings**
2. Enable **Public Access**
3. Note the public URL: `https://your-bucket.account-id.r2.dev`
Or use a custom domain:
1. Go to **R2** > Your bucket > **Settings** > **Custom Domains**
2. Add `media.yourdomain.com`
3. Update `.env`:
```env
R2_PUBLIC_URL=https://media.yourdomain.com
```
---
## Security Settings
### WAF Rules (Free Tier)
Enable these managed rules:
1. **CloudFlare Managed Ruleset** - Basic protection
2. **OWASP Core Ruleset** - SQL injection, XSS protection
### Rate Limiting
Create rate limiting rules for the API:
- URL: `/api/*`
- Rate: 100 requests per minute
- Action: Challenge or Block
### Bot Fight Mode
1. Go to **Security** > **Bots**
2. Enable **Bot Fight Mode**
---
## Nginx Configuration for CloudFlare
When using CloudFlare proxy, update Nginx to trust CloudFlare IPs:
```nginx
# /etc/nginx/conf.d/cloudflare.conf
# CloudFlare IP ranges
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 104.16.0.0/13;
set_real_ip_from 104.24.0.0/14;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
# IPv6
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2a06:98c0::/29;
set_real_ip_from 2c0f:f248::/32;
real_ip_header CF-Connecting-IP;
```
---
## Environment Variables Reference
| Variable | Description | Default |
|----------|-------------|---------|
| `CLOUDFLARE_ENABLED` | Enable CF header processing | `false` |
| `STORAGE_BACKEND` | Storage backend (`local` or `r2`) | `local` |
| `R2_ACCOUNT_ID` | CloudFlare account ID | - |
| `R2_ACCESS_KEY_ID` | R2 API access key | - |
| `R2_SECRET_ACCESS_KEY` | R2 API secret key | - |
| `R2_BUCKET_NAME` | R2 bucket name | `wizamart-media` |
| `R2_PUBLIC_URL` | Custom public URL for R2 | - |
---
## Verification
### Check CloudFlare is Working
1. **Check headers** in browser DevTools:
- `CF-Ray` header should be present
- `CF-Cache-Status` shows caching status
2. **Test from command line**:
```bash
curl -I https://yourdomain.com/static/css/main.css
# Should see CF-Ray and CF-Cache-Status headers
```
### Check R2 is Working
1. **Upload a test file** via the admin media library
2. **Check the URL** - should point to R2 or your custom domain
3. **Verify in CloudFlare dashboard** - file should appear in bucket
### Check Real IP Logging
With `CLOUDFLARE_ENABLED=true`:
```bash
# Check application logs
journalctl -u wizamart | grep "real_ip"
```
---
## Troubleshooting
### 521 Error (Web Server Down)
- Ensure your origin server is running
- Check firewall allows CloudFlare IPs
- Verify SSL certificate is valid
### 522 Error (Connection Timed Out)
- Check origin server is responding
- Verify port 443 is open
- Check server isn't overloaded
### 525 Error (SSL Handshake Failed)
- Ensure origin has valid SSL certificate
- Try CloudFlare Origin Certificate
- Check SSL mode is correct (Full vs Full Strict)
### R2 Access Denied
- Verify API token has correct permissions
- Check bucket name is correct
- Ensure bucket policy allows the operation
---
## Cost Estimate
| Service | Free Tier | Paid Usage |
|---------|-----------|------------|
| CDN | Unlimited | Free |
| WAF | Basic rules | $20/mo for advanced |
| R2 Storage | 10 GB/mo | $0.015/GB |
| R2 Requests | 10M Class A, 10M Class B | $0.36/M, $0.0036/M |
| SSL | Free | Free |
**Typical monthly cost for small-medium site: ~$5-15**

View File

@@ -73,31 +73,52 @@ This guide documents the complete infrastructure for the Wizamart platform, from
### What We Have Now ### What We Have Now
| Component | Technology | Status | | Component | Technology | Dev Required | Prod Required | Status |
|-----------|------------|--------| |-----------|------------|--------------|---------------|--------|
| Web Framework | FastAPI + Uvicorn | ✅ Production Ready | | Web Framework | FastAPI + Uvicorn | ✅ | ✅ | ✅ Production Ready |
| Database | PostgreSQL 15 | ✅ Production Ready | | Database | PostgreSQL 15 | ✅ | ✅ | ✅ Production Ready |
| ORM | SQLAlchemy 2.0 | ✅ Production Ready | | ORM | SQLAlchemy 2.0 | ✅ | ✅ | ✅ Production Ready |
| Migrations | Alembic | ✅ Production Ready | | Migrations | Alembic | ✅ | ✅ | ✅ Production Ready |
| Templates | Jinja2 + Tailwind CSS | ✅ Production Ready | | Templates | Jinja2 + Tailwind CSS | ✅ | ✅ | ✅ Production Ready |
| Authentication | JWT (PyJWT) | ✅ Production Ready | | Authentication | JWT (PyJWT) | ✅ | ✅ | ✅ Production Ready |
| Email | SMTP/SendGrid/Mailgun/SES | ✅ Production Ready | | Email | SMTP/SendGrid/Mailgun/SES | ❌ | ✅ | ✅ Production Ready |
| Payments | Stripe | ✅ Production Ready | | Payments | Stripe | ❌ | ✅ | ✅ Production Ready |
| Task Queue | Celery 5.3 + Redis | ✅ Production Ready | | Task Queue | Celery 5.3 + Redis | ❌ | ✅ | ✅ Production Ready |
| Task Scheduler | Celery Beat | ✅ Production Ready | | Task Scheduler | Celery Beat | ❌ | ✅ | ✅ Production Ready |
| Task Monitoring | Flower | ✅ Production Ready | | Task Monitoring | Flower | ❌ | ⚪ Optional | ✅ Production Ready |
| Caching | Redis 7 | ✅ Production Ready | | Caching | Redis 7 | ❌ | ✅ | ✅ Production Ready |
| File Storage | Local filesystem | ⏳ Needs S3 for prod | | File Storage | Local / Cloudflare R2 | Local | R2 | ✅ Production Ready |
| Error Tracking | Sentry | ❌ | ⚪ Recommended | ✅ Production Ready |
| CDN / WAF | CloudFlare | ❌ | ⚪ Recommended | ✅ Production Ready |
### What We Need to Add **Legend:** ✅ Required | ⚪ Optional/Recommended | ❌ Not needed
| Component | Priority | Reason | ### Development vs Production
|-----------|----------|--------|
| S3/MinIO | High | Scalable file storage | **Development** requires only:
| Sentry | High | Error tracking | - PostgreSQL (via Docker: `make docker-up`)
| CloudFlare | Medium | CDN + DDoS protection | - Python 3.11+ with dependencies
| Load Balancer | Medium | Horizontal scaling |
| Prometheus/Grafana | Low | Metrics and dashboards | **Production** adds:
- Redis (for Celery task queue)
- Celery workers (for background tasks)
- Reverse proxy (Nginx)
- SSL certificates
**Optional but recommended for Production:**
- Sentry (error tracking) - Set `SENTRY_DSN` to enable
- Cloudflare R2 (cloud storage) - Set `STORAGE_BACKEND=r2` to enable
- CloudFlare CDN (caching/DDoS) - Set `CLOUDFLARE_ENABLED=true` to enable
### What We Need for Enterprise (Future Growth)
| Component | Priority | When Needed | Estimated Users |
|-----------|----------|-------------|-----------------|
| Load Balancer | Medium | Horizontal scaling | 1,000+ concurrent |
| Database Replica | Medium | Read-heavy workloads | 1,000+ concurrent |
| Redis Sentinel | Low | Cache redundancy | 5,000+ concurrent |
| Prometheus/Grafana | Low | Advanced metrics | Any (nice to have) |
| Kubernetes | Low | Multi-region/HA | 10,000+ concurrent |
--- ---
@@ -838,7 +859,7 @@ python -c "from app.core.database import engine; print(engine.connect())"
## Migration Path ## Migration Path
### Phase 1: Current (Development) ✅ COMPLETE ### Phase 1: Development ✅ COMPLETE
- ✅ PostgreSQL 15 (Docker) - ✅ PostgreSQL 15 (Docker)
- ✅ FastAPI + Uvicorn - ✅ FastAPI + Uvicorn
- ✅ Local file storage - ✅ Local file storage
@@ -850,35 +871,93 @@ python -c "from app.core.database import engine; print(engine.connect())"
- ✅ Celery 5.3 (background jobs) - ✅ Celery 5.3 (background jobs)
- ✅ Celery Beat (scheduled tasks) - ✅ Celery Beat (scheduled tasks)
- ✅ Flower (task monitoring) - ✅ Flower (task monitoring)
- ⏳ S3/MinIO (file storage) - ✅ Cloudflare R2 (cloud file storage)
- Sentry (error tracking) - Sentry (error tracking)
- ✅ CloudFlare CDN (caching + DDoS protection)
### Phase 3: Scale ### Phase 3: Scale (1,000+ Users)
- Horizontal app scaling (multiple Uvicorn instances) - ⏳ Load balancer (Nginx/HAProxy/ALB)
- Load balancer (Nginx/HAProxy) - ⏳ Horizontal app scaling (2-4 Uvicorn instances)
- PostgreSQL read replicas - PostgreSQL read replica
- Redis Sentinel/cluster - ⏳ Dedicated Celery workers per queue
- CDN for static assets (CloudFlare)
- Dedicated Celery workers per queue
### Phase 4: High Availability ### Phase 4: Enterprise (5,000+ Users)
- Multi-region deployment - ⏳ Redis Sentinel/cluster
- Database failover - Database connection pooling (PgBouncer)
- Container orchestration (Kubernetes) - ⏳ Full monitoring stack (Prometheus/Grafana)
- Full monitoring stack (Prometheus/Grafana/Loki) - ⏳ Log aggregation (Loki/ELK)
### Phase 5: High Availability (10,000+ Users)
- ⏳ Multi-region deployment
- ⏳ Database failover (streaming replication)
- ⏳ Container orchestration (Kubernetes)
- ⏳ Global CDN with edge caching
---
## Enterprise Upgrade Checklist
When you're ready to scale beyond 1,000 concurrent users:
### Infrastructure
- [ ] **Load Balancer** - Add Nginx/HAProxy in front of API servers
- Enables horizontal scaling
- Health checks and automatic failover
- SSL termination at edge
- [ ] **Multiple API Servers** - Run 2-4 Uvicorn instances
- Scale horizontally instead of vertically
- Blue-green deployments possible
- [ ] **Database Read Replica** - Add PostgreSQL replica
- Offload read queries from primary
- Backup without impacting production
- [ ] **Connection Pooling** - Add PgBouncer
- Reduce database connection overhead
- Handle connection spikes
### Monitoring & Observability
- [ ] **Prometheus + Grafana** - Metrics dashboards
- Request latency, error rates, saturation
- Database connection pool metrics
- Celery queue lengths
- [ ] **Log Aggregation** - Loki or ELK stack
- Centralized logs from all services
- Search and alerting
- [ ] **Alerting** - PagerDuty/OpsGenie integration
- On-call rotation
- Escalation policies
### Security
- [ ] **WAF Rules** - CloudFlare or AWS WAF
- SQL injection protection
- Rate limiting at edge
- Bot protection
- [ ] **Secrets Management** - HashiCorp Vault
- Rotate credentials automatically
- Audit access to secrets
--- ---
## Next Steps ## Next Steps
1. **Configure S3/MinIO** - For production file storage (high priority) **You're production-ready now!** Optional improvements:
2. **Set up Sentry** - Error tracking (high priority)
3. **Add CloudFlare** - CDN + DDoS protection (medium priority) 1. **Enable Sentry** - Add `SENTRY_DSN` for error tracking (free tier)
4. **Configure load balancer** - When scaling horizontally 2. **Enable R2** - Set `STORAGE_BACKEND=r2` for cloud storage (~$5/mo)
5. **Choose production deployment** - VPS or Docker based on team preference 3. **Enable CloudFlare** - Proxy domain for CDN + DDoS protection (free tier)
4. **Add load balancer** - When you need horizontal scaling
See also: See also:
- [Production Deployment Guide](production.md) - [Production Deployment Guide](production.md)
- [CloudFlare Setup Guide](cloudflare.md)
- [Docker Deployment](docker.md) - [Docker Deployment](docker.md)
- [Environment Configuration](environment.md) - [Environment Configuration](environment.md)
- [Background Tasks Architecture](../architecture/background-tasks.md) - [Background Tasks Architecture](../architecture/background-tasks.md)

View File

@@ -393,11 +393,70 @@ free -h
### Set Up Sentry (Error Tracking) ### Set Up Sentry (Error Tracking)
Sentry provides real-time error tracking and performance monitoring.
1. **Create a Sentry account** at [sentry.io](https://sentry.io) (free tier available)
2. **Create a new project** (Python/FastAPI)
3. **Add to `.env`**:
```env
SENTRY_DSN=https://your-key@sentry.io/project-id
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
```
4. **Restart services**:
```bash
sudo systemctl restart wizamart wizamart-celery
```
Sentry will now capture:
- Unhandled exceptions
- API errors with request context
- Celery task failures
- Performance traces (10% sample rate)
---
## Cloudflare R2 Storage
For production, use Cloudflare R2 instead of local storage for scalability and CDN integration.
### Setup
1. **Create R2 bucket** in CloudFlare dashboard
2. **Create API token** with Object Read/Write permissions
3. **Add to `.env`**:
```env
STORAGE_BACKEND=r2
R2_ACCOUNT_ID=your_account_id
R2_ACCESS_KEY_ID=your_access_key
R2_SECRET_ACCESS_KEY=your_secret_key
R2_BUCKET_NAME=wizamart-media
R2_PUBLIC_URL=https://media.yourdomain.com
```
See [CloudFlare Setup Guide](cloudflare.md) for detailed instructions.
---
## CloudFlare CDN & Proxy
For production, proxy your domain through CloudFlare for:
- Global CDN caching
- DDoS protection
- Free SSL certificates
- WAF (Web Application Firewall)
### Enable CloudFlare Headers
Add to `.env`: Add to `.env`:
```env ```env
SENTRY_DSN=https://your-sentry-dsn CLOUDFLARE_ENABLED=true
``` ```
This enables proper handling of `CF-Connecting-IP` for real client IPs.
See [CloudFlare Setup Guide](cloudflare.md) for complete configuration.
--- ---
## Troubleshooting ## Troubleshooting

View File

@@ -236,6 +236,31 @@ pytest -m integration
| `FLOWER_URL` | Flower dashboard URL | `http://localhost:5555` | ❌ | | `FLOWER_URL` | Flower dashboard URL | `http://localhost:5555` | ❌ |
| `FLOWER_PASSWORD` | Flower authentication password | `changeme` | ❌ | | `FLOWER_PASSWORD` | Flower authentication password | `changeme` | ❌ |
### Sentry Error Tracking
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `SENTRY_DSN` | Sentry DSN (leave empty to disable) | - | ❌ |
| `SENTRY_ENVIRONMENT` | Environment name | `development` | ❌ |
| `SENTRY_TRACES_SAMPLE_RATE` | Performance tracing rate (0.0-1.0) | `0.1` | ❌ |
### Cloudflare R2 Storage
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `STORAGE_BACKEND` | Storage backend (`local` or `r2`) | `local` | ❌ |
| `R2_ACCOUNT_ID` | Cloudflare account ID | - | ❌ (if r2) |
| `R2_ACCESS_KEY_ID` | R2 access key | - | ❌ (if r2) |
| `R2_SECRET_ACCESS_KEY` | R2 secret key | - | ❌ (if r2) |
| `R2_BUCKET_NAME` | R2 bucket name | `wizamart-media` | ❌ |
| `R2_PUBLIC_URL` | Custom public URL for R2 | - | ❌ |
### CloudFlare CDN
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `CLOUDFLARE_ENABLED` | Enable CloudFlare header handling | `false` | ❌ |
### Stripe Billing ### Stripe Billing
| Variable | Description | Default | Required | | Variable | Description | Default | Required |

View File

@@ -435,9 +435,12 @@ We welcome contributions! Please see our [Contributing Guide](development/contri
- ✅ Marketplace integration - ✅ Marketplace integration
- ✅ Team management - ✅ Team management
**Production Ready**:
- ✅ Error tracking (Sentry integration)
- ✅ Cloud storage (Cloudflare R2)
- ✅ CDN & WAF (CloudFlare)
**In Development**: **In Development**:
- 🚧 Payment integration (Stripe-ready)
- 🚧 Email notifications
- 🚧 Advanced analytics - 🚧 Advanced analytics
--- ---

25
main.py
View File

@@ -21,16 +21,41 @@ import logging
from datetime import UTC, datetime from datetime import UTC, datetime
from pathlib import Path from pathlib import Path
import sentry_sdk
from fastapi import Depends, FastAPI, HTTPException, Request, Response from fastapi import Depends, FastAPI, HTTPException, Request, Response
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, HTMLResponse, RedirectResponse from fastapi.responses import FileResponse, HTMLResponse, RedirectResponse
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates from fastapi.templating import Jinja2Templates
from sentry_sdk.integrations.fastapi import FastApiIntegration
from sentry_sdk.integrations.sqlalchemy import SqlalchemyIntegration
from sqlalchemy import text from sqlalchemy import text
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.api.main import api_router from app.api.main import api_router
from app.core.config import settings from app.core.config import settings
# =============================================================================
# SENTRY INITIALIZATION
# =============================================================================
# Initialize Sentry for error tracking (only if DSN is configured)
if settings.sentry_dsn:
sentry_sdk.init(
dsn=settings.sentry_dsn,
environment=settings.sentry_environment,
traces_sample_rate=settings.sentry_traces_sample_rate,
integrations=[
FastApiIntegration(transaction_style="endpoint"),
SqlalchemyIntegration(),
],
# Send PII data (emails, usernames) - set to False if privacy is critical
send_default_pii=True,
# Release version for tracking deployments
release=f"wizamart@{settings.version}",
)
logging.getLogger(__name__).info(
f"Sentry initialized for environment: {settings.sentry_environment}"
)
from app.core.database import get_db from app.core.database import get_db
from app.core.lifespan import lifespan from app.core.lifespan import lifespan
from app.exceptions import ServiceUnavailableException from app.exceptions import ServiceUnavailableException

149
middleware/cloudflare.py Normal file
View File

@@ -0,0 +1,149 @@
# middleware/cloudflare.py
"""
CloudFlare proxy middleware.
When enabled, this middleware handles CloudFlare-specific headers:
- CF-Connecting-IP: Real client IP address
- CF-IPCountry: Client's country code
- CF-Ray: CloudFlare Ray ID for request tracing
- CF-Visitor: Original request scheme
Usage:
Enable by setting CLOUDFLARE_ENABLED=true in .env
The middleware will:
1. Extract real client IP from CF-Connecting-IP header
2. Store country code in request.state.client_country
3. Log CF-Ray for request correlation
"""
import logging
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
from app.core.config import settings
logger = logging.getLogger(__name__)
class CloudFlareMiddleware(BaseHTTPMiddleware):
"""
Middleware to handle CloudFlare proxy headers.
Only active when CLOUDFLARE_ENABLED=true.
"""
async def dispatch(self, request: Request, call_next) -> Response:
"""Process request with CloudFlare headers."""
if not settings.cloudflare_enabled:
return await call_next(request)
# Extract CloudFlare headers
cf_connecting_ip = request.headers.get("CF-Connecting-IP")
cf_country = request.headers.get("CF-IPCountry")
cf_ray = request.headers.get("CF-Ray")
cf_visitor = request.headers.get("CF-Visitor") # {"scheme":"https"}
# Store real client IP
if cf_connecting_ip:
# Override the client host with real IP
# Note: Starlette's request.client is immutable, so we store in state
request.state.real_ip = cf_connecting_ip
else:
# Fallback to standard client IP
request.state.real_ip = (
request.client.host if request.client else "unknown"
)
# Store country code for geo features
if cf_country:
request.state.client_country = cf_country
# Store Ray ID for request tracing
if cf_ray:
request.state.cf_ray = cf_ray
# Determine original scheme
if cf_visitor:
try:
import json
visitor_info = json.loads(cf_visitor)
request.state.original_scheme = visitor_info.get("scheme", "https")
except (json.JSONDecodeError, TypeError):
request.state.original_scheme = "https"
# Log request with CloudFlare context
logger.debug(
f"CloudFlare request: ip={cf_connecting_ip}, "
f"country={cf_country}, ray={cf_ray}"
)
# Process request
response = await call_next(request)
# Add CF-Ray to response for debugging
if cf_ray:
response.headers["X-CF-Ray"] = cf_ray
return response
def get_real_client_ip(request: Request) -> str:
"""
Get the real client IP address.
When behind CloudFlare, this returns CF-Connecting-IP.
Otherwise, returns the standard client IP.
Args:
request: Starlette Request object
Returns:
Client IP address string
"""
# First check if CloudFlare middleware set real_ip
if hasattr(request.state, "real_ip"):
return request.state.real_ip
# Check for CF-Connecting-IP header directly
cf_ip = request.headers.get("CF-Connecting-IP")
if cf_ip:
return cf_ip
# Check for X-Forwarded-For (generic proxy header)
forwarded_for = request.headers.get("X-Forwarded-For")
if forwarded_for:
# Take the first IP (client IP)
return forwarded_for.split(",")[0].strip()
# Fallback to request client
return request.client.host if request.client else "unknown"
def get_client_country(request: Request) -> str | None:
"""
Get the client's country code from CloudFlare.
Args:
request: Starlette Request object
Returns:
ISO 3166-1 alpha-2 country code, or None if not available
"""
if hasattr(request.state, "client_country"):
return request.state.client_country
return request.headers.get("CF-IPCountry")
# =============================================================================
# PUBLIC API
# =============================================================================
__all__ = [
"CloudFlareMiddleware",
"get_real_client_ip",
"get_client_country",
]

View File

@@ -188,6 +188,7 @@ nav:
- Launch Readiness: deployment/launch-readiness.md - Launch Readiness: deployment/launch-readiness.md
- Traditional VPS: deployment/production.md - Traditional VPS: deployment/production.md
- Docker: deployment/docker.md - Docker: deployment/docker.md
- CloudFlare Setup: deployment/cloudflare.md
- GitLab CI/CD: deployment/gitlab.md - GitLab CI/CD: deployment/gitlab.md
- Environment Variables: deployment/environment.md - Environment Variables: deployment/environment.md
- Stripe Integration: deployment/stripe-integration.md - Stripe Integration: deployment/stripe-integration.md

View File

@@ -41,4 +41,10 @@ stripe>=7.0.0
celery[redis]==5.3.6 celery[redis]==5.3.6
redis==5.0.1 redis==5.0.1
kombu==5.3.4 kombu==5.3.4
flower==2.0.1 flower==2.0.1
# Error tracking
sentry-sdk[fastapi]>=2.0.0
# Cloud storage (S3-compatible - Cloudflare R2)
boto3>=1.34.0