diff --git a/docs/features/subscription-billing.md b/docs/features/subscription-billing.md index 3f14e65e..817a8fd7 100644 --- a/docs/features/subscription-billing.md +++ b/docs/features/subscription-billing.md @@ -11,6 +11,8 @@ The billing system enables: - **Stripe Integration**: Checkout sessions, customer portal, and webhook handling - **Self-Service Billing**: Vendor-facing billing page for subscription management - **Add-ons**: Optional purchasable items (domains, SSL, email packages) +- **Capacity Forecasting**: Growth trends and scaling recommendations +- **Background Jobs**: Automated subscription lifecycle management ## Architecture @@ -26,23 +28,32 @@ All subscription models are defined in `models/database/subscription.py`: | `VendorAddOn` | Add-ons purchased by each vendor | | `StripeWebhookEvent` | Idempotency tracking for webhooks | | `BillingHistory` | Invoice and payment history | +| `CapacitySnapshot` | Daily platform capacity metrics for forecasting | ### Services | Service | Location | Purpose | |---------|----------|---------| | `BillingService` | `app/services/billing_service.py` | Subscription operations, checkout, portal | -| `SubscriptionService` | `app/services/subscription_service.py` | Limit checks, usage tracking | +| `SubscriptionService` | `app/services/subscription_service.py` | Limit checks, usage tracking, tier info | | `StripeService` | `app/services/stripe_service.py` | Core Stripe API operations | +| `CapacityForecastService` | `app/services/capacity_forecast_service.py` | Growth trends, projections | +| `PlatformHealthService` | `app/services/platform_health_service.py` | Subscription capacity aggregation | -### Handlers +### Background Tasks -| Handler | Location | Purpose | -|---------|----------|---------| -| `StripeWebhookHandler` | `app/handlers/stripe_webhook.py` | Webhook event processing | +| Task | Location | Schedule | Purpose | +|------|----------|----------|---------| +| `reset_period_counters` | `app/tasks/subscription_tasks.py` | Daily | Reset order counters at period end | +| `check_trial_expirations` | `app/tasks/subscription_tasks.py` | Daily | Expire trials without payment method | +| `sync_stripe_status` | `app/tasks/subscription_tasks.py` | Hourly | Sync status with Stripe | +| `cleanup_stale_subscriptions` | `app/tasks/subscription_tasks.py` | Weekly | Clean up old cancelled subscriptions | +| `capture_capacity_snapshot` | `app/tasks/subscription_tasks.py` | Daily | Capture capacity metrics snapshot | ### API Endpoints +#### Vendor Billing API + All billing endpoints are under `/api/v1/vendor/billing`: | Endpoint | Method | Purpose | @@ -52,11 +63,28 @@ All billing endpoints are under `/api/v1/vendor/billing`: | `/billing/checkout` | POST | Create Stripe checkout session | | `/billing/portal` | POST | Create Stripe customer portal session | | `/billing/invoices` | GET | Invoice history | +| `/billing/upcoming-invoice` | GET | Preview next invoice | +| `/billing/change-tier` | POST | Upgrade/downgrade tier | | `/billing/addons` | GET | Available add-on products | | `/billing/my-addons` | GET | Vendor's purchased add-ons | +| `/billing/addons/purchase` | POST | Purchase an add-on | +| `/billing/addons/{id}` | DELETE | Cancel an add-on | | `/billing/cancel` | POST | Cancel subscription | | `/billing/reactivate` | POST | Reactivate cancelled subscription | +#### Admin Platform Health API + +Capacity endpoints under `/api/v1/admin/platform-health`: + +| Endpoint | Method | Purpose | +|----------|--------|---------| +| `/platform-health/health` | GET | Full platform health report | +| `/platform-health/capacity` | GET | Capacity-focused metrics | +| `/platform-health/subscription-capacity` | GET | Subscription-based capacity vs usage | +| `/platform-health/trends` | GET | Growth trends over time | +| `/platform-health/recommendations` | GET | Scaling recommendations | +| `/platform-health/snapshot` | POST | Manually capture capacity snapshot | + ## Subscription Tiers ### Default Tiers @@ -68,6 +96,32 @@ All billing endpoints are under `/api/v1/vendor/billing`: | Business | €199/mo | Unlimited | 2000 | 10 | | Enterprise | Custom | Unlimited | Unlimited | Unlimited | +### Database-Driven Tiers + +Tiers are stored in the `subscription_tiers` table and queried via `SubscriptionService`: + +```python +# Get tier from database with legacy fallback +tier_info = subscription_service.get_tier_info("professional", db=db) + +# Query all active tiers +all_tiers = subscription_service.get_all_tiers(db=db) +``` + +The service maintains backward compatibility with a legacy fallback: + +```python +def get_tier_info(self, tier_code: str, db: Session | None = None) -> TierInfo: + """Get full tier information. Queries database if db session provided.""" + if db is not None: + db_tier = self.get_tier_by_code(db, tier_code) + if db_tier: + return TierInfo(...) + + # Fallback to hardcoded TIER_LIMITS during migration + return self._get_tier_from_legacy(tier_code) +``` + ### Tier Features Each tier includes specific features stored in the `features` JSON column: @@ -150,6 +204,7 @@ The vendor billing page is at `/vendor/{vendor_code}/billing`: 3. **Change Plan**: Upgrade/downgrade options 4. **Payment Method**: Link to Stripe portal 5. **Invoice History**: Recent invoices with PDF links +6. **Add-ons**: Available and purchased add-ons ### JavaScript Component @@ -161,6 +216,8 @@ function billingData() { subscription: null, tiers: [], invoices: [], + addons: [], + myAddons: [], async init() { await this.loadData(); @@ -177,6 +234,18 @@ function billingData() { async openPortal() { const response = await this.apiPost('/billing/portal', {}); window.location.href = response.portal_url; + }, + + async purchaseAddon(addon) { + const response = await this.apiPost('/billing/addons/purchase', { + addon_code: addon.code + }); + window.location.href = response.checkout_url; + }, + + async cancelAddon(addon) { + await this.apiDelete(`/billing/addons/${addon.id}`); + await this.loadMyAddons(); } }; } @@ -201,6 +270,163 @@ function billingData() { 3. Create Stripe checkout session with add-on price 4. On webhook success: create `VendorAddOn` record +## Background Tasks + +### Task Definitions + +```python +# app/tasks/subscription_tasks.py + +async def reset_period_counters(): + """ + Reset order counters for subscriptions whose billing period has ended. + Should run daily. Resets orders_this_period to 0 and updates period dates. + """ + +async def check_trial_expirations(): + """ + Check for expired trials and update their status. + Trials without payment method -> expired + Trials with payment method -> active + """ + +async def sync_stripe_status(): + """ + Sync subscription status with Stripe. + Fetches current status and updates local records. + """ + +async def cleanup_stale_subscriptions(): + """ + Clean up subscriptions in inconsistent states. + Marks old cancelled subscriptions as expired. + """ + +async def capture_capacity_snapshot(): + """ + Capture a daily snapshot of platform capacity metrics. + Used for growth trending and capacity forecasting. + """ +``` + +### Scheduling + +Configure with your scheduler of choice (Celery, APScheduler, cron): + +```python +# Example with APScheduler +scheduler.add_job(reset_period_counters, 'cron', hour=0, minute=5) +scheduler.add_job(check_trial_expirations, 'cron', hour=1, minute=0) +scheduler.add_job(sync_stripe_status, 'cron', minute=0) # Every hour +scheduler.add_job(cleanup_stale_subscriptions, 'cron', day_of_week=0) # Weekly +scheduler.add_job(capture_capacity_snapshot, 'cron', hour=0, minute=0) +``` + +## Capacity Forecasting + +### Subscription-Based Capacity + +Track theoretical vs actual capacity: + +```python +capacity = platform_health_service.get_subscription_capacity(db) + +# Returns: +{ + "total_subscriptions": 150, + "tier_distribution": { + "essential": 80, + "professional": 50, + "business": 18, + "enterprise": 2 + }, + "products": { + "actual": 125000, + "theoretical_limit": 500000, + "utilization_percent": 25.0, + "headroom": 375000 + }, + "orders_monthly": { + "actual": 45000, + "theoretical_limit": 300000, + "utilization_percent": 15.0 + }, + "team_members": { + "actual": 320, + "theoretical_limit": 1500, + "utilization_percent": 21.3 + } +} +``` + +### Growth Trends + +Analyze growth over time: + +```python +trends = capacity_forecast_service.get_growth_trends(db, days=30) + +# Returns: +{ + "period_days": 30, + "snapshots_available": 30, + "trends": { + "vendors": { + "start_value": 140, + "current_value": 150, + "change": 10, + "growth_rate_percent": 7.14, + "daily_growth_rate": 0.238, + "monthly_projection": 161 + }, + "products": { + "start_value": 115000, + "current_value": 125000, + "change": 10000, + "growth_rate_percent": 8.7, + "monthly_projection": 136000 + } + } +} +``` + +### Scaling Recommendations + +Get automated scaling advice: + +```python +recommendations = capacity_forecast_service.get_scaling_recommendations(db) + +# Returns: +[ + { + "category": "capacity", + "severity": "warning", + "title": "Product capacity approaching limit", + "description": "Currently at 85% of theoretical product capacity", + "action": "Consider upgrading vendor tiers or adding capacity" + }, + { + "category": "infrastructure", + "severity": "info", + "title": "Current tier: Medium", + "description": "Next upgrade trigger: 300 vendors", + "action": "Monitor growth and plan for infrastructure scaling" + } +] +``` + +### Infrastructure Scaling Reference + +| Clients | vCPU | RAM | Storage | Database | Monthly Cost | +|---------|------|-----|---------|----------|--------------| +| 1-50 | 2 | 4GB | 100GB | SQLite | €30 | +| 50-100 | 4 | 8GB | 250GB | PostgreSQL | €80 | +| 100-300 | 4 | 16GB | 500GB | PostgreSQL | €150 | +| 300-500 | 8 | 32GB | 1TB | PostgreSQL + Redis | €350 | +| 500-1000 | 16 | 64GB | 2TB | PostgreSQL + Redis | €700 | +| 1000+ | 32+ | 128GB+ | 4TB+ | PostgreSQL cluster | €1,500+ | + ## Exception Handling Custom exceptions for billing operations (`app/exceptions/billing.py`): @@ -252,6 +478,43 @@ def seed_subscription_tiers(op): ]) ``` +### Capacity Snapshots Table + +The `capacity_snapshots` table stores daily metrics: + +```python +# alembic/versions/l0a1b2c3d4e5_add_capacity_snapshots_table.py +class CapacitySnapshot(Base): + id: int + snapshot_date: datetime # Unique per day + + # Vendor metrics + total_vendors: int + active_vendors: int + trial_vendors: int + + # Subscription metrics + total_subscriptions: int + active_subscriptions: int + + # Resource metrics + total_products: int + total_orders_month: int + total_team_members: int + + # Storage metrics + storage_used_gb: Decimal + db_size_mb: Decimal + + # Capacity metrics + theoretical_products_limit: int + theoretical_orders_limit: int + theoretical_team_limit: int + + # Tier distribution + tier_distribution: dict # JSON +``` + ### Setting Up Stripe 1. Create products and prices in Stripe Dashboard @@ -273,3 +536,10 @@ tier.stripe_price_annual_id = "price_yyy" - Idempotency keys prevent duplicate event processing - Customer portal links are session-based and expire - Stripe API key stored securely in environment variables +- Background tasks run with database session isolation + +## Related Documentation + +- [Capacity Monitoring](../operations/capacity-monitoring.md) - Detailed monitoring guide +- [Capacity Planning](../architecture/capacity-planning.md) - Infrastructure sizing +- [Stripe Integration](../deployment/stripe-integration.md) - Payment setup for vendors diff --git a/docs/operations/capacity-monitoring.md b/docs/operations/capacity-monitoring.md index f46625f5..7913d807 100644 --- a/docs/operations/capacity-monitoring.md +++ b/docs/operations/capacity-monitoring.md @@ -1,10 +1,28 @@ # Capacity Monitoring -Detailed guide for monitoring and managing platform capacity. +Detailed guide for monitoring and managing platform capacity, including growth forecasting and scaling recommendations. ## Overview -The Capacity Monitoring page (`/admin/platform-health/capacity`) provides insights into resource consumption and helps plan infrastructure scaling. +The Capacity Monitoring system provides insights into resource consumption and helps plan infrastructure scaling. It includes: + +- **Real-time metrics**: Current resource usage and health status +- **Subscription capacity**: Theoretical vs actual capacity based on vendor subscriptions +- **Growth forecasting**: Historical trends and future projections +- **Scaling recommendations**: Automated advice for infrastructure planning + +## API Endpoints + +All capacity endpoints are under `/api/v1/admin/platform-health`: + +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/health` | GET | Full platform health report | +| `/capacity` | GET | Capacity-focused metrics | +| `/subscription-capacity` | GET | Subscription-based capacity analysis | +| `/trends` | GET | Growth trends over specified period | +| `/recommendations` | GET | Prioritized scaling recommendations | +| `/snapshot` | POST | Manually capture capacity snapshot | ## Key Metrics @@ -17,6 +35,41 @@ The Capacity Monitoring page (`/admin/platform-health/capacity`) provides insigh | Products per Client | Average products per vendor | Tier compliance | | Monthly Orders | Order volume this month | Performance impact | +### Subscription Capacity + +Track theoretical vs actual capacity based on all vendor subscriptions: + +```python +# GET /api/v1/admin/platform-health/subscription-capacity +{ + "total_subscriptions": 150, + "tier_distribution": { + "essential": 80, + "professional": 50, + "business": 18, + "enterprise": 2 + }, + "products": { + "actual": 125000, + "theoretical_limit": 500000, + "utilization_percent": 25.0, + "headroom": 375000 + }, + "orders_monthly": { + "actual": 45000, + "theoretical_limit": 300000, + "utilization_percent": 15.0, + "headroom": 255000 + }, + "team_members": { + "actual": 320, + "theoretical_limit": 1500, + "utilization_percent": 21.3, + "headroom": 1180 + } +} +``` + ### Storage Metrics | Metric | Description | Warning | Critical | @@ -35,26 +88,148 @@ The Capacity Monitoring page (`/admin/platform-health/capacity`) provides insigh | Cache Hit Rate | > 90% | 70-90% | < 70% | | Connection Pool Usage | < 70% | 70-90% | > 90% | +## Growth Forecasting + +### Capacity Snapshots + +Daily snapshots are captured automatically by the `capture_capacity_snapshot` background task: + +```python +# Captured daily at midnight +class CapacitySnapshot: + snapshot_date: datetime + + # Vendor metrics + total_vendors: int + active_vendors: int + trial_vendors: int + + # Subscription metrics + total_subscriptions: int + active_subscriptions: int + + # Resource metrics + total_products: int + total_orders_month: int + total_team_members: int + + # Storage metrics + storage_used_gb: Decimal + db_size_mb: Decimal + + # Capacity metrics + theoretical_products_limit: int + theoretical_orders_limit: int + theoretical_team_limit: int + + # Tier distribution + tier_distribution: dict +``` + +### Growth Trends + +Analyze growth over any period: + +```python +# GET /api/v1/admin/platform-health/trends?days=30 +{ + "period_days": 30, + "snapshots_available": 30, + "start_date": "2025-11-26", + "end_date": "2025-12-26", + "trends": { + "vendors": { + "start_value": 140, + "current_value": 150, + "change": 10, + "growth_rate_percent": 7.14, + "daily_growth_rate": 0.238, + "monthly_projection": 161 + }, + "products": { + "start_value": 115000, + "current_value": 125000, + "change": 10000, + "growth_rate_percent": 8.7, + "daily_growth_rate": 0.29, + "monthly_projection": 136000 + }, + "orders": { + "start_value": 40000, + "current_value": 45000, + "change": 5000, + "growth_rate_percent": 12.5, + "monthly_projection": 51000 + }, + "team_members": {...}, + "storage_gb": { + "start_value": 150.5, + "current_value": 165.2, + "change": 14.7 + } + } +} +``` + +### Days Until Threshold + +Calculate when a metric will reach a specific threshold: + +```python +# Service method +days = capacity_forecast_service.get_days_until_threshold( + db, + metric="total_products", + threshold=500000 +) +# Returns: 120 (days until products reach 500K) +``` + ## Scaling Recommendations -The system provides automatic scaling recommendations based on current usage: - -### Example Recommendations +The system generates automated recommendations based on current capacity and growth: +```python +# GET /api/v1/admin/platform-health/recommendations +[ + { + "category": "capacity", + "severity": "warning", + "title": "Product capacity approaching limit", + "description": "Currently at 85% of theoretical product capacity", + "action": "Consider upgrading vendor tiers or adding capacity" + }, + { + "category": "infrastructure", + "severity": "info", + "title": "Current tier: Medium", + "description": "Next upgrade trigger: 300 vendors", + "action": "Monitor growth and plan for infrastructure scaling" + }, + { + "category": "growth", + "severity": "info", + "title": "High vendor growth rate", + "description": "Vendor base growing at 15.2% over last 30 days", + "action": "Ensure infrastructure can scale to meet demand" + }, + { + "category": "storage", + "severity": "warning", + "title": "Storage usage high", + "description": "Image storage at 850 GB", + "action": "Plan for storage expansion or implement cleanup policies" + } +] ``` -Current Infrastructure: MEDIUM (100-300 clients) -Current Usage: 85% of capacity -Recommendations: -1. [WARNING] Approaching product limit (420K of 500K) - → Consider upgrading to LARGE tier +### Severity Levels -2. [INFO] Database size growing 5GB/month - → Plan storage expansion in 3 months - -3. [OK] API response times within normal range - → No action needed -``` +| Severity | Description | Action Required | +|----------|-------------|-----------------| +| `critical` | Immediate action needed | Within 24 hours | +| `warning` | Plan action soon | Within 1-2 weeks | +| `info` | Informational | Monitor and plan | ## Threshold Configuration @@ -90,13 +265,63 @@ CAPACITY_THRESHOLDS = { } ``` -## Historical Trends +## Infrastructure Scaling Reference -View growth trends to plan ahead: +| Clients | vCPU | RAM | Storage | Database | Monthly Cost | +|---------|------|-----|---------|----------|--------------| +| 1-50 | 2 | 4GB | 100GB | SQLite | €30 | +| 50-100 | 4 | 8GB | 250GB | PostgreSQL | €80 | +| 100-300 | 4 | 16GB | 500GB | PostgreSQL | €150 | +| 300-500 | 8 | 32GB | 1TB | PostgreSQL + Redis | €350 | +| 500-1000 | 16 | 64GB | 2TB | PostgreSQL + Redis | €700 | +| 1000+ | 32+ | 128GB+ | 4TB+ | PostgreSQL cluster | €1,500+ | -- **30-day growth rate**: Products, storage, clients -- **Projected capacity date**: When limits will be reached -- **Seasonal patterns**: Order volume fluctuations +## Background Tasks + +### Capacity Snapshot Task + +```python +# app/tasks/subscription_tasks.py + +async def capture_capacity_snapshot(): + """ + Capture a daily snapshot of platform capacity metrics. + Should run daily at midnight. + """ + from app.services.capacity_forecast_service import capacity_forecast_service + + db = SessionLocal() + try: + snapshot = capacity_forecast_service.capture_daily_snapshot(db) + db.commit() + return { + "snapshot_id": snapshot.id, + "snapshot_date": snapshot.snapshot_date.isoformat(), + "total_vendors": snapshot.total_vendors, + "total_products": snapshot.total_products, + } + finally: + db.close() +``` + +### Manual Snapshot + +Capture a snapshot on demand: + +```bash +# Via API +curl -X POST /api/v1/admin/platform-health/snapshot \ + -H "Authorization: Bearer $TOKEN" + +# Response +{ + "id": 42, + "snapshot_date": "2025-12-26T00:00:00Z", + "total_vendors": 150, + "total_products": 125000, + "message": "Snapshot captured successfully" +} +``` ## Alerts @@ -106,6 +331,29 @@ Capacity alerts trigger when: 2. **Critical (Red)**: 95% of any threshold 3. **Exceeded**: 100%+ of threshold (immediate action) +## Historical Data + +### Viewing Historical Trends + +Use the `/trends` endpoint with different day ranges: + +```bash +# Last 7 days +GET /api/v1/admin/platform-health/trends?days=7 + +# Last 30 days (default) +GET /api/v1/admin/platform-health/trends?days=30 + +# Last 90 days +GET /api/v1/admin/platform-health/trends?days=90 +``` + +### Data Retention + +- Snapshots are stored indefinitely by default +- Consider implementing cleanup for snapshots older than 2 years +- At minimum, keep monthly aggregates for long-term trending + ## Export Reports Generate capacity reports for planning: @@ -114,8 +362,47 @@ Generate capacity reports for planning: - **Monthly capacity report**: Detailed analysis - **Projection report**: 3/6/12 month forecasts +## Usage Examples + +### Check Current Capacity + +```python +from app.services.platform_health_service import platform_health_service +from app.services.capacity_forecast_service import capacity_forecast_service + +# Get subscription capacity +capacity = platform_health_service.get_subscription_capacity(db) +print(f"Products: {capacity['products']['actual']} / {capacity['products']['theoretical_limit']}") +print(f"Utilization: {capacity['products']['utilization_percent']}%") + +# Get growth trends +trends = capacity_forecast_service.get_growth_trends(db, days=30) +print(f"Vendor growth: {trends['trends']['vendors']['growth_rate_percent']}%") + +# Get recommendations +recommendations = capacity_forecast_service.get_scaling_recommendations(db) +for rec in recommendations: + print(f"[{rec['severity']}] {rec['title']}: {rec['action']}") +``` + +### Project Future Capacity + +```python +# Calculate days until product limit +days = capacity_forecast_service.get_days_until_threshold( + db, + metric="total_products", + threshold=500000 +) +if days: + print(f"Products will reach 500K in approximately {days} days") +else: + print("Insufficient data or no growth detected") +``` + ## Related Documentation +- [Subscription & Billing](../features/subscription-billing.md) - Complete billing system - [Capacity Planning](../architecture/capacity-planning.md) - Full sizing guide - [Platform Health](platform-health.md) - Real-time health monitoring - [Image Storage](image-storage.md) - Image system details