sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

docs: update subscription billing and capacity monitoring documentation

Browse Source 
- Expand subscription-billing.md with complete system documentation
  - Add background tasks section with scheduling examples
  - Add capacity forecasting with API examples
  - Document all new API endpoints (trends, recommendations, snapshot)
  - Add CapacitySnapshot model documentation
  - Include infrastructure scaling reference table

- Update capacity-monitoring.md with forecasting features
  - Add subscription capacity tracking section
  - Document growth trends API with example responses
  - Add scaling recommendations with severity levels
  - Include usage examples for capacity planning
  - Add historical data and export options

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Samir Boulahtit
2025-12-26 20:56:22 +01:00
parent c6e7f4087f
commit 25279a03d4
2 changed files with 583 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

280
docs/features/subscription-billing.md
View File

@@ -11,6 +11,8 @@ The billing system enables:
- **Stripe Integration**: Checkout sessions, customer portal, and webhook handling
- **Self-Service Billing**: Vendor-facing billing page for subscription management
- **Add-ons**: Optional purchasable items (domains, SSL, email packages)
- **Capacity Forecasting**: Growth trends and scaling recommendations
- **Background Jobs**: Automated subscription lifecycle management
## Architecture
@@ -26,23 +28,32 @@ All subscription models are defined in `models/database/subscription.py`:
| `VendorAddOn` | Add-ons purchased by each vendor |
| `StripeWebhookEvent` | Idempotency tracking for webhooks |
| `BillingHistory` | Invoice and payment history |
| `CapacitySnapshot` | Daily platform capacity metrics for forecasting |
### Services
| Service | Location | Purpose |
|---------|----------|---------|
| `BillingService` | `app/services/billing_service.py` | Subscription operations, checkout, portal |
| `SubscriptionService` | `app/services/subscription_service.py` | Limit checks, usage tracking |
| `SubscriptionService` | `app/services/subscription_service.py` | Limit checks, usage tracking, tier info |
| `StripeService` | `app/services/stripe_service.py` | Core Stripe API operations |
| `CapacityForecastService` | `app/services/capacity_forecast_service.py` | Growth trends, projections |
| `PlatformHealthService` | `app/services/platform_health_service.py` | Subscription capacity aggregation |
### Handlers
### Background Tasks
| Handler | Location | Purpose |
|---------|----------|---------|
| `StripeWebhookHandler` | `app/handlers/stripe_webhook.py` | Webhook event processing |
| Task | Location | Schedule | Purpose |
|------|----------|----------|---------|
| `reset_period_counters` | `app/tasks/subscription_tasks.py` | Daily | Reset order counters at period end |
| `check_trial_expirations` | `app/tasks/subscription_tasks.py` | Daily | Expire trials without payment method |
| `sync_stripe_status` | `app/tasks/subscription_tasks.py` | Hourly | Sync status with Stripe |
| `cleanup_stale_subscriptions` | `app/tasks/subscription_tasks.py` | Weekly | Clean up old cancelled subscriptions |
| `capture_capacity_snapshot` | `app/tasks/subscription_tasks.py` | Daily | Capture capacity metrics snapshot |
### API Endpoints
#### Vendor Billing API
All billing endpoints are under `/api/v1/vendor/billing`:
| Endpoint | Method | Purpose |
@@ -52,11 +63,28 @@ All billing endpoints are under `/api/v1/vendor/billing`:
| `/billing/checkout` | POST | Create Stripe checkout session |
| `/billing/portal` | POST | Create Stripe customer portal session |
| `/billing/invoices` | GET | Invoice history |
| `/billing/upcoming-invoice` | GET | Preview next invoice |
| `/billing/change-tier` | POST | Upgrade/downgrade tier |
| `/billing/addons` | GET | Available add-on products |
| `/billing/my-addons` | GET | Vendor's purchased add-ons |
| `/billing/addons/purchase` | POST | Purchase an add-on |
| `/billing/addons/{id}` | DELETE | Cancel an add-on |
| `/billing/cancel` | POST | Cancel subscription |
| `/billing/reactivate` | POST | Reactivate cancelled subscription |
#### Admin Platform Health API
Capacity endpoints under `/api/v1/admin/platform-health`:
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/platform-health/health` | GET | Full platform health report |
| `/platform-health/capacity` | GET | Capacity-focused metrics |
| `/platform-health/subscription-capacity` | GET | Subscription-based capacity vs usage |
| `/platform-health/trends` | GET | Growth trends over time |
| `/platform-health/recommendations` | GET | Scaling recommendations |
| `/platform-health/snapshot` | POST | Manually capture capacity snapshot |
## Subscription Tiers
### Default Tiers
@@ -68,6 +96,32 @@ All billing endpoints are under `/api/v1/vendor/billing`:
| Business | €199/mo | Unlimited | 2000 | 10 |
| Enterprise | Custom | Unlimited | Unlimited | Unlimited |
### Database-Driven Tiers
Tiers are stored in the `subscription_tiers` table and queried via `SubscriptionService`:
```python
# Get tier from database with legacy fallback
tier_info = subscription_service.get_tier_info("professional", db=db)
# Query all active tiers
all_tiers = subscription_service.get_all_tiers(db=db)
```
The service maintains backward compatibility with a legacy fallback:
```python
def get_tier_info(self, tier_code: str, db: Session | None = None) -> TierInfo:
"""Get full tier information. Queries database if db session provided."""
if db is not None:
db_tier = self.get_tier_by_code(db, tier_code)
if db_tier:
return TierInfo(...)
# Fallback to hardcoded TIER_LIMITS during migration
return self._get_tier_from_legacy(tier_code)
```
### Tier Features
Each tier includes specific features stored in the `features` JSON column:
@@ -150,6 +204,7 @@ The vendor billing page is at `/vendor/{vendor_code}/billing`:
3. **Change Plan**: Upgrade/downgrade options
4. **Payment Method**: Link to Stripe portal
5. **Invoice History**: Recent invoices with PDF links
6. **Add-ons**: Available and purchased add-ons
### JavaScript Component
@@ -161,6 +216,8 @@ function billingData() {
subscription: null,
tiers: [],
invoices: [],
addons: [],
myAddons: [],
async init() {
await this.loadData();
@@ -177,6 +234,18 @@ function billingData() {
async openPortal() {
const response = await this.apiPost('/billing/portal', {});
window.location.href = response.portal_url;
},
async purchaseAddon(addon) {
const response = await this.apiPost('/billing/addons/purchase', {
addon_code: addon.code
});
window.location.href = response.checkout_url;
},
async cancelAddon(addon) {
await this.apiDelete(`/billing/addons/${addon.id}`);
await this.loadMyAddons();
}
};
}
@@ -201,6 +270,163 @@ function billingData() {
3. Create Stripe checkout session with add-on price
4. On webhook success: create `VendorAddOn` record
## Background Tasks
### Task Definitions
```python
# app/tasks/subscription_tasks.py
async def reset_period_counters():
"""
Reset order counters for subscriptions whose billing period has ended.
Should run daily. Resets orders_this_period to 0 and updates period dates.
"""
async def check_trial_expirations():
"""
Check for expired trials and update their status.
Trials without payment method -> expired
Trials with payment method -> active
"""
async def sync_stripe_status():
"""
Sync subscription status with Stripe.
Fetches current status and updates local records.
"""
async def cleanup_stale_subscriptions():
"""
Clean up subscriptions in inconsistent states.
Marks old cancelled subscriptions as expired.
"""
async def capture_capacity_snapshot():
"""
Capture a daily snapshot of platform capacity metrics.
Used for growth trending and capacity forecasting.
"""
```
### Scheduling
Configure with your scheduler of choice (Celery, APScheduler, cron):
```python
# Example with APScheduler
scheduler.add_job(reset_period_counters, 'cron', hour=0, minute=5)
scheduler.add_job(check_trial_expirations, 'cron', hour=1, minute=0)
scheduler.add_job(sync_stripe_status, 'cron', minute=0) # Every hour
scheduler.add_job(cleanup_stale_subscriptions, 'cron', day_of_week=0) # Weekly
scheduler.add_job(capture_capacity_snapshot, 'cron', hour=0, minute=0)
```
## Capacity Forecasting
### Subscription-Based Capacity
Track theoretical vs actual capacity:
```python
capacity = platform_health_service.get_subscription_capacity(db)
# Returns:
{
"total_subscriptions": 150,
"tier_distribution": {
"essential": 80,
"professional": 50,
"business": 18,
"enterprise": 2
},
"products": {
"actual": 125000,
"theoretical_limit": 500000,
"utilization_percent": 25.0,
"headroom": 375000
},
"orders_monthly": {
"actual": 45000,
"theoretical_limit": 300000,
"utilization_percent": 15.0
},
"team_members": {
"actual": 320,
"theoretical_limit": 1500,
"utilization_percent": 21.3
}
}
```
### Growth Trends
Analyze growth over time:
```python
trends = capacity_forecast_service.get_growth_trends(db, days=30)
# Returns:
{
"period_days": 30,
"snapshots_available": 30,
"trends": {
"vendors": {
"start_value": 140,
"current_value": 150,
"change": 10,
"growth_rate_percent": 7.14,
"daily_growth_rate": 0.238,
"monthly_projection": 161
},
"products": {
"start_value": 115000,
"current_value": 125000,
"change": 10000,
"growth_rate_percent": 8.7,
"monthly_projection": 136000
}
}
}
```
### Scaling Recommendations
Get automated scaling advice:
```python
recommendations = capacity_forecast_service.get_scaling_recommendations(db)
# Returns:
[
{
"category": "capacity",
"severity": "warning",
"title": "Product capacity approaching limit",
"description": "Currently at 85% of theoretical product capacity",
"action": "Consider upgrading vendor tiers or adding capacity"
},
{
"category": "infrastructure",
"severity": "info",
"title": "Current tier: Medium",
"description": "Next upgrade trigger: 300 vendors",
"action": "Monitor growth and plan for infrastructure scaling"
}
]
```
### Infrastructure Scaling Reference
| Clients | vCPU | RAM | Storage | Database | Monthly Cost |
|---------|------|-----|---------|----------|--------------|
| 1-50 | 2 | 4GB | 100GB | SQLite | €30 |
| 50-100 | 4 | 8GB | 250GB | PostgreSQL | €80 |
| 100-300 | 4 | 16GB | 500GB | PostgreSQL | €150 |
| 300-500 | 8 | 32GB | 1TB | PostgreSQL + Redis | €350 |
| 500-1000 | 16 | 64GB | 2TB | PostgreSQL + Redis | €700 |
| 1000+ | 32+ | 128GB+ | 4TB+ | PostgreSQL cluster | €1,500+ |
## Exception Handling
Custom exceptions for billing operations (`app/exceptions/billing.py`):
@@ -252,6 +478,43 @@ def seed_subscription_tiers(op):
])
```
### Capacity Snapshots Table
The `capacity_snapshots` table stores daily metrics:
```python
# alembic/versions/l0a1b2c3d4e5_add_capacity_snapshots_table.py
class CapacitySnapshot(Base):
id: int
snapshot_date: datetime # Unique per day
# Vendor metrics
total_vendors: int
active_vendors: int
trial_vendors: int
# Subscription metrics
total_subscriptions: int
active_subscriptions: int
# Resource metrics
total_products: int
total_orders_month: int
total_team_members: int
# Storage metrics
storage_used_gb: Decimal
db_size_mb: Decimal
# Capacity metrics
theoretical_products_limit: int
theoretical_orders_limit: int
theoretical_team_limit: int
# Tier distribution
tier_distribution: dict # JSON
```
### Setting Up Stripe
1. Create products and prices in Stripe Dashboard
@@ -273,3 +536,10 @@ tier.stripe_price_annual_id = "price_yyy"
- Idempotency keys prevent duplicate event processing
- Customer portal links are session-based and expire
- Stripe API key stored securely in environment variables
- Background tasks run with database session isolation
## Related Documentation
- [Capacity Monitoring](../operations/capacity-monitoring.md) - Detailed monitoring guide
- [Capacity Planning](../architecture/capacity-planning.md) - Infrastructure sizing
- [Stripe Integration](../deployment/stripe-integration.md) - Payment setup for vendors

329
docs/operations/capacity-monitoring.md
View File

@@ -1,10 +1,28 @@
# Capacity Monitoring
Detailed guide for monitoring and managing platform capacity.
Detailed guide for monitoring and managing platform capacity, including growth forecasting and scaling recommendations.
## Overview
The Capacity Monitoring page (`/admin/platform-health/capacity`) provides insights into resource consumption and helps plan infrastructure scaling.
The Capacity Monitoring system provides insights into resource consumption and helps plan infrastructure scaling. It includes:
- **Real-time metrics**: Current resource usage and health status
- **Subscription capacity**: Theoretical vs actual capacity based on vendor subscriptions
- **Growth forecasting**: Historical trends and future projections
- **Scaling recommendations**: Automated advice for infrastructure planning
## API Endpoints
All capacity endpoints are under `/api/v1/admin/platform-health`:
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Full platform health report |
| `/capacity` | GET | Capacity-focused metrics |
| `/subscription-capacity` | GET | Subscription-based capacity analysis |
| `/trends` | GET | Growth trends over specified period |
| `/recommendations` | GET | Prioritized scaling recommendations |
| `/snapshot` | POST | Manually capture capacity snapshot |
## Key Metrics
@@ -17,6 +35,41 @@ The Capacity Monitoring page (`/admin/platform-health/capacity`) provides insigh
| Products per Client | Average products per vendor | Tier compliance |
| Monthly Orders | Order volume this month | Performance impact |
### Subscription Capacity
Track theoretical vs actual capacity based on all vendor subscriptions:
```python
# GET /api/v1/admin/platform-health/subscription-capacity
{
"total_subscriptions": 150,
"tier_distribution": {
"essential": 80,
"professional": 50,
"business": 18,
"enterprise": 2
},
"products": {
"actual": 125000,
"theoretical_limit": 500000,
"utilization_percent": 25.0,
"headroom": 375000
},
"orders_monthly": {
"actual": 45000,
"theoretical_limit": 300000,
"utilization_percent": 15.0,
"headroom": 255000
},
"team_members": {
"actual": 320,
"theoretical_limit": 1500,
"utilization_percent": 21.3,
"headroom": 1180
}
}
```
### Storage Metrics
| Metric | Description | Warning | Critical |
@@ -35,26 +88,148 @@ The Capacity Monitoring page (`/admin/platform-health/capacity`) provides insigh
| Cache Hit Rate | > 90% | 70-90% | < 70% |
| Connection Pool Usage | < 70% | 70-90% | > 90% |
## Growth Forecasting
### Capacity Snapshots
Daily snapshots are captured automatically by the `capture_capacity_snapshot` background task:
```python
# Captured daily at midnight
class CapacitySnapshot:
snapshot_date: datetime
# Vendor metrics
total_vendors: int
active_vendors: int
trial_vendors: int
# Subscription metrics
total_subscriptions: int
active_subscriptions: int
# Resource metrics
total_products: int
total_orders_month: int
total_team_members: int
# Storage metrics
storage_used_gb: Decimal
db_size_mb: Decimal
# Capacity metrics
theoretical_products_limit: int
theoretical_orders_limit: int
theoretical_team_limit: int
# Tier distribution
tier_distribution: dict
```
### Growth Trends
Analyze growth over any period:
```python
# GET /api/v1/admin/platform-health/trends?days=30
{
"period_days": 30,
"snapshots_available": 30,
"start_date": "2025-11-26",
"end_date": "2025-12-26",
"trends": {
"vendors": {
"start_value": 140,
"current_value": 150,
"change": 10,
"growth_rate_percent": 7.14,
"daily_growth_rate": 0.238,
"monthly_projection": 161
},
"products": {
"start_value": 115000,
"current_value": 125000,
"change": 10000,
"growth_rate_percent": 8.7,
"daily_growth_rate": 0.29,
"monthly_projection": 136000
},
"orders": {
"start_value": 40000,
"current_value": 45000,
"change": 5000,
"growth_rate_percent": 12.5,
"monthly_projection": 51000
},
"team_members": {...},
"storage_gb": {
"start_value": 150.5,
"current_value": 165.2,
"change": 14.7
}
}
}
```
### Days Until Threshold
Calculate when a metric will reach a specific threshold:
```python
# Service method
days = capacity_forecast_service.get_days_until_threshold(
db,
metric="total_products",
threshold=500000
)
# Returns: 120 (days until products reach 500K)
```
## Scaling Recommendations
The system provides automatic scaling recommendations based on current usage:
### Example Recommendations
The system generates automated recommendations based on current capacity and growth:
```python
# GET /api/v1/admin/platform-health/recommendations
[
{
"category": "capacity",
"severity": "warning",
"title": "Product capacity approaching limit",
"description": "Currently at 85% of theoretical product capacity",
"action": "Consider upgrading vendor tiers or adding capacity"
},
{
"category": "infrastructure",
"severity": "info",
"title": "Current tier: Medium",
"description": "Next upgrade trigger: 300 vendors",
"action": "Monitor growth and plan for infrastructure scaling"
},
{
"category": "growth",
"severity": "info",
"title": "High vendor growth rate",
"description": "Vendor base growing at 15.2% over last 30 days",
"action": "Ensure infrastructure can scale to meet demand"
},
{
"category": "storage",
"severity": "warning",
"title": "Storage usage high",
"description": "Image storage at 850 GB",
"action": "Plan for storage expansion or implement cleanup policies"
}
]
```
Current Infrastructure: MEDIUM (100-300 clients)
Current Usage: 85% of capacity
Recommendations:
1. [WARNING] Approaching product limit (420K of 500K)
→ Consider upgrading to LARGE tier
### Severity Levels
2. [INFO] Database size growing 5GB/month
→ Plan storage expansion in 3 months
3. [OK] API response times within normal range
→ No action needed
```
| Severity | Description | Action Required |
|----------|-------------|-----------------|
| `critical` | Immediate action needed | Within 24 hours |
| `warning` | Plan action soon | Within 1-2 weeks |
| `info` | Informational | Monitor and plan |
## Threshold Configuration
@@ -90,13 +265,63 @@ CAPACITY_THRESHOLDS = {
}
```
## Historical Trends
## Infrastructure Scaling Reference
View growth trends to plan ahead:
| Clients | vCPU | RAM | Storage | Database | Monthly Cost |
|---------|------|-----|---------|----------|--------------|
| 1-50 | 2 | 4GB | 100GB | SQLite | €30 |
| 50-100 | 4 | 8GB | 250GB | PostgreSQL | €80 |
| 100-300 | 4 | 16GB | 500GB | PostgreSQL | €150 |
| 300-500 | 8 | 32GB | 1TB | PostgreSQL + Redis | €350 |
| 500-1000 | 16 | 64GB | 2TB | PostgreSQL + Redis | €700 |
| 1000+ | 32+ | 128GB+ | 4TB+ | PostgreSQL cluster | €1,500+ |
- **30-day growth rate**: Products, storage, clients
- **Projected capacity date**: When limits will be reached
- **Seasonal patterns**: Order volume fluctuations
## Background Tasks
### Capacity Snapshot Task
```python
# app/tasks/subscription_tasks.py
async def capture_capacity_snapshot():
"""
Capture a daily snapshot of platform capacity metrics.
Should run daily at midnight.
"""
from app.services.capacity_forecast_service import capacity_forecast_service
db = SessionLocal()
try:
snapshot = capacity_forecast_service.capture_daily_snapshot(db)
db.commit()
return {
"snapshot_id": snapshot.id,
"snapshot_date": snapshot.snapshot_date.isoformat(),
"total_vendors": snapshot.total_vendors,
"total_products": snapshot.total_products,
}
finally:
db.close()
```
### Manual Snapshot
Capture a snapshot on demand:
```bash
# Via API
curl -X POST /api/v1/admin/platform-health/snapshot \
-H "Authorization: Bearer $TOKEN"
# Response
{
"id": 42,
"snapshot_date": "2025-12-26T00:00:00Z",
"total_vendors": 150,
"total_products": 125000,
"message": "Snapshot captured successfully"
}
```
## Alerts
@@ -106,6 +331,29 @@ Capacity alerts trigger when:
2. **Critical (Red)**: 95% of any threshold
3. **Exceeded**: 100%+ of threshold (immediate action)
## Historical Data
### Viewing Historical Trends
Use the `/trends` endpoint with different day ranges:
```bash
# Last 7 days
GET /api/v1/admin/platform-health/trends?days=7
# Last 30 days (default)
GET /api/v1/admin/platform-health/trends?days=30
# Last 90 days
GET /api/v1/admin/platform-health/trends?days=90
```
### Data Retention
- Snapshots are stored indefinitely by default
- Consider implementing cleanup for snapshots older than 2 years
- At minimum, keep monthly aggregates for long-term trending
## Export Reports
Generate capacity reports for planning:
@@ -114,8 +362,47 @@ Generate capacity reports for planning:
- **Monthly capacity report**: Detailed analysis
- **Projection report**: 3/6/12 month forecasts
## Usage Examples
### Check Current Capacity
```python
from app.services.platform_health_service import platform_health_service
from app.services.capacity_forecast_service import capacity_forecast_service
# Get subscription capacity
capacity = platform_health_service.get_subscription_capacity(db)
print(f"Products: {capacity['products']['actual']} / {capacity['products']['theoretical_limit']}")
print(f"Utilization: {capacity['products']['utilization_percent']}%")
# Get growth trends
trends = capacity_forecast_service.get_growth_trends(db, days=30)
print(f"Vendor growth: {trends['trends']['vendors']['growth_rate_percent']}%")
# Get recommendations
recommendations = capacity_forecast_service.get_scaling_recommendations(db)
for rec in recommendations:
print(f"[{rec['severity']}] {rec['title']}: {rec['action']}")
```
### Project Future Capacity
```python
# Calculate days until product limit
days = capacity_forecast_service.get_days_until_threshold(
db,
metric="total_products",
threshold=500000
)
if days:
print(f"Products will reach 500K in approximately {days} days")
else:
print("Insufficient data or no growth detected")
```
## Related Documentation
- [Subscription & Billing](../features/subscription-billing.md) - Complete billing system
- [Capacity Planning](../architecture/capacity-planning.md) - Full sizing guide
- [Platform Health](platform-health.md) - Real-time health monitoring
- [Image Storage](image-storage.md) - Image system details