sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement metrics provider pattern for modular dashboard statistics

Browse Source 
This commit introduces a protocol-based metrics architecture that allows
each module to provide its own statistics for dashboards without creating
cross-module dependencies.

Key changes:
- Add MetricsProviderProtocol and MetricValue dataclass in contracts module
- Add StatsAggregatorService in core module that discovers and aggregates
  metrics from all enabled modules
- Implement metrics providers for all modules:
  - tenancy: vendor/user counts, team members, domains
  - customers: customer counts
  - cms: pages, media files
  - catalog: products
  - inventory: stock levels
  - orders: order counts, revenue
  - marketplace: import jobs, staging products
- Update dashboard routes to use StatsAggregator instead of direct imports
- Fix VendorPlatform junction table usage (Vendor.platform_id doesn't exist)
- Add comprehensive documentation for the pattern

This architecture ensures:
- Dashboards always work (aggregator in core)
- Each module owns its metrics (no cross-module coupling)
- Optional modules are truly optional (can be removed without breaking app)
- Multi-platform vendors are properly supported via VendorPlatform table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Samir Boulahtit
2026-02-03 21:11:29 +01:00
parent a76128e016
commit a8fae0fbc7
28 changed files with 3745 additions and 269 deletions
Download Patch File Download Diff File Expand all files Collapse all files

481
docs/architecture/metrics-provider-pattern.md Normal file
View File

@@ -0,0 +1,481 @@
# Metrics Provider Pattern
The metrics provider pattern enables modules to provide their own statistics for dashboards without creating cross-module dependencies. This is a key architectural pattern that ensures the platform remains modular and extensible.
## Overview
```
┌─────────────────────────────────────────────────────────────────────┐
│ Dashboard Request │
│ (Admin Dashboard or Vendor Dashboard) │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ StatsAggregatorService │
│ (app/modules/core/services/stats_aggregator.py) │
│ │
│ • Discovers MetricsProviders from all enabled modules │
│ • Calls get_vendor_metrics() or get_platform_metrics() │
│ • Returns categorized metrics dict │
└─────────────────────────────────────────────────────────────────────┘
│
┌───────────────────────┼───────────────────────┐
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│tenancy_metrics│ │ order_metrics │ │catalog_metrics│
│ (enabled) │ │ (enabled) │ │ (disabled) │
└───────┬───────┘ └───────┬───────┘ └───────────────┘
│ │ │
▼ ▼ × (skipped)
┌───────────────┐ ┌───────────────┐
│ vendor_count │ │ total_orders │
│ user_count │ │ total_revenue │
└───────────────┘ └───────────────┘
│ │
└───────────┬───────────┘
▼
┌─────────────────────────────────┐
│ Categorized Metrics │
│ {"tenancy": [...], "orders": [...]} │
└─────────────────────────────────┘
```
## Problem Solved
Before this pattern, dashboard routes had **hard imports** from optional modules:
```python
# BAD: Core module with hard dependency on optional module
from app.modules.analytics.services import stats_service # What if disabled?
from app.modules.marketplace.models import MarketplaceImportJob # What if disabled?
stats = stats_service.get_vendor_stats(db, vendor_id) # App crashes!
```
This violated the architecture rule: **Core modules cannot depend on optional modules.**
## Solution: Protocol-Based Metrics
Each module implements the `MetricsProviderProtocol` and registers it in its `definition.py`. The `StatsAggregatorService` in core discovers and aggregates metrics from all enabled modules.
## Key Components
### 1. MetricValue Dataclass
Standard structure for metric values:
```python
# app/modules/contracts/metrics.py
from dataclasses import dataclass
@dataclass
class MetricValue:
key: str # Unique identifier (e.g., "orders.total")
value: int | float | str # The metric value
label: str # Human-readable label
category: str # Grouping category (module name)
icon: str | None = None # Optional UI icon
description: str | None = None # Tooltip description
unit: str | None = None # Unit (%, EUR, items)
trend: str | None = None # "up", "down", "stable"
trend_value: float | None = None # Percentage change
```
### 2. MetricsContext Dataclass
Context for metric queries (date ranges, options):
```python
@dataclass
class MetricsContext:
date_from: datetime | None = None
date_to: datetime | None = None
include_trends: bool = False
period: str = "30d" # Default period for calculations
```
### 3. MetricsProviderProtocol
Protocol that modules implement:
```python
from typing import Protocol, runtime_checkable
@runtime_checkable
class MetricsProviderProtocol(Protocol):
@property
def metrics_category(self) -> str:
"""Category name for this provider's metrics (e.g., 'orders')."""
...
def get_vendor_metrics(
self,
db: Session,
vendor_id: int,
context: MetricsContext | None = None,
) -> list[MetricValue]:
"""Get metrics for a specific vendor (vendor dashboard)."""
...
def get_platform_metrics(
self,
db: Session,
platform_id: int,
context: MetricsContext | None = None,
) -> list[MetricValue]:
"""Get metrics aggregated for a platform (admin dashboard)."""
...
```
### 4. StatsAggregatorService
Central service in core that discovers and aggregates metrics:
```python
# app/modules/core/services/stats_aggregator.py
class StatsAggregatorService:
def get_vendor_dashboard_stats(
self,
db: Session,
vendor_id: int,
platform_id: int,
context: MetricsContext | None = None,
) -> dict[str, list[MetricValue]]:
"""Get all metrics for a vendor, grouped by category."""
providers = self._get_enabled_providers(db, platform_id)
return {
p.metrics_category: p.get_vendor_metrics(db, vendor_id, context)
for p in providers
}
def get_admin_dashboard_stats(
self,
db: Session,
platform_id: int,
context: MetricsContext | None = None,
) -> dict[str, list[MetricValue]]:
"""Get all metrics for admin dashboard, grouped by category."""
providers = self._get_enabled_providers(db, platform_id)
return {
p.metrics_category: p.get_platform_metrics(db, platform_id, context)
for p in providers
}
```
## Implementing a Metrics Provider
### Step 1: Create the Metrics Provider Class
```python
# app/modules/orders/services/order_metrics.py
import logging
from sqlalchemy import func
from sqlalchemy.orm import Session
from app.modules.contracts.metrics import (
MetricValue,
MetricsContext,
MetricsProviderProtocol,
)
logger = logging.getLogger(__name__)
class OrderMetricsProvider:
"""Metrics provider for orders module."""
@property
def metrics_category(self) -> str:
return "orders"
def get_vendor_metrics(
self,
db: Session,
vendor_id: int,
context: MetricsContext | None = None,
) -> list[MetricValue]:
"""Get order metrics for a specific vendor."""
from app.modules.orders.models import Order
try:
total_orders = (
db.query(Order)
.filter(Order.vendor_id == vendor_id)
.count()
)
total_revenue = (
db.query(func.sum(Order.total_amount))
.filter(Order.vendor_id == vendor_id)
.scalar() or 0
)
return [
MetricValue(
key="orders.total",
value=total_orders,
label="Total Orders",
category="orders",
icon="shopping-cart",
description="Total orders received",
),
MetricValue(
key="orders.total_revenue",
value=float(total_revenue),
label="Total Revenue",
category="orders",
icon="currency-euro",
unit="EUR",
description="Total revenue from orders",
),
]
except Exception as e:
logger.warning(f"Failed to get order vendor metrics: {e}")
return []
def get_platform_metrics(
self,
db: Session,
platform_id: int,
context: MetricsContext | None = None,
) -> list[MetricValue]:
"""Get order metrics aggregated for a platform."""
from app.modules.orders.models import Order
from app.modules.tenancy.models import VendorPlatform
try:
# IMPORTANT: Use VendorPlatform junction table for multi-platform support
vendor_ids = (
db.query(VendorPlatform.vendor_id)
.filter(
VendorPlatform.platform_id == platform_id,
VendorPlatform.is_active == True,
)
.subquery()
)
total_orders = (
db.query(Order)
.filter(Order.vendor_id.in_(vendor_ids))
.count()
)
return [
MetricValue(
key="orders.total",
value=total_orders,
label="Total Orders",
category="orders",
icon="shopping-cart",
description="Total orders across all vendors",
),
]
except Exception as e:
logger.warning(f"Failed to get order platform metrics: {e}")
return []
# Singleton instance
order_metrics_provider = OrderMetricsProvider()
```
### Step 2: Register in Module Definition
```python
# app/modules/orders/definition.py
from app.modules.base import ModuleDefinition
def _get_metrics_provider():
"""Lazy import to avoid circular imports."""
from app.modules.orders.services.order_metrics import order_metrics_provider
return order_metrics_provider
orders_module = ModuleDefinition(
code="orders",
name="Order Management",
# ... other config ...
# Register the metrics provider
metrics_provider=_get_metrics_provider,
)
```
### Step 3: Metrics Appear Automatically
When the module is enabled, its metrics automatically appear in dashboards.
## Multi-Platform Architecture
### VendorPlatform Junction Table
Vendors can belong to multiple platforms. When querying platform-level metrics, **always use the VendorPlatform junction table**:
```python
# CORRECT: Using VendorPlatform junction table
from app.modules.tenancy.models import VendorPlatform
vendor_ids = (
db.query(VendorPlatform.vendor_id)
.filter(
VendorPlatform.platform_id == platform_id,
VendorPlatform.is_active == True,
)
.subquery()
)
total_orders = (
db.query(Order)
.filter(Order.vendor_id.in_(vendor_ids))
.count()
)
# WRONG: Vendor.platform_id does not exist!
# vendor_ids = db.query(Vendor.id).filter(Vendor.platform_id == platform_id)
```
### Platform Context Flow
Platform context flows through middleware and JWT tokens:
```
┌─────────────────────────────────────────────────────────────────────┐
│ Request Flow │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ PlatformContextMiddleware │
│ Sets: request.state.platform (Platform object) │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ VendorContextMiddleware │
│ Sets: request.state.vendor (Vendor object) │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ Route Handler (Dashboard) │
│ │
│ # Get platform_id from middleware or JWT token │
│ platform = getattr(request.state, "platform", None) │
│ platform_id = platform.id if platform else 1 │
│ │
│ # Or from JWT for API routes │
│ platform_id = current_user.token_platform_id or 1 │
└─────────────────────────────────────────────────────────────────────┘
```
## Available Metrics Providers
| Module | Category | Metrics Provided |
|--------|----------|------------------|
| **tenancy** | `tenancy` | vendor counts, user counts, team members, domains |
| **customers** | `customers` | customer counts, new customers |
| **cms** | `cms` | pages, media files, themes |
| **catalog** | `catalog` | products, active products, featured |
| **inventory** | `inventory` | stock levels, low stock, out of stock |
| **orders** | `orders` | order counts, revenue, average order value |
| **marketplace** | `marketplace` | import jobs, staging products, success rate |
## Dashboard Routes
### Vendor Dashboard
```python
# app/modules/core/routes/api/vendor_dashboard.py
@router.get("/stats", response_model=VendorDashboardStatsResponse)
def get_vendor_dashboard_stats(
request: Request,
current_user: UserContext = Depends(get_current_vendor_api),
db: Session = Depends(get_db),
):
vendor_id = current_user.token_vendor_id
# Get platform from middleware
platform = getattr(request.state, "platform", None)
platform_id = platform.id if platform else 1
# Get aggregated metrics from all enabled modules
metrics = stats_aggregator.get_vendor_dashboard_stats(
db=db,
vendor_id=vendor_id,
platform_id=platform_id,
)
# Extract and return formatted response
...
```
### Admin Dashboard
```python
# app/modules/core/routes/api/admin_dashboard.py
@router.get("/stats", response_model=StatsResponse)
def get_comprehensive_stats(
request: Request,
db: Session = Depends(get_db),
current_admin: UserContext = Depends(get_current_admin_api),
):
# Get platform_id with fallback logic
platform_id = _get_platform_id(request, current_admin)
# Get aggregated metrics from all enabled modules
metrics = stats_aggregator.get_admin_dashboard_stats(
db=db,
platform_id=platform_id,
)
# Extract and return formatted response
...
```
## Benefits
| Aspect | Before | After |
|--------|--------|-------|
| Core depends on optional | Hard import (crashes) | Protocol-based (graceful) |
| Adding new metrics | Edit analytics module | Just add provider to your module |
| Module isolation | Coupled | Truly independent |
| Testing | Hard (need all modules) | Easy (mock protocol) |
| Disable module | App crashes | Dashboard shows partial data |
## Error Handling
Metrics providers are wrapped in try/except to prevent one failing module from breaking the entire dashboard:
```python
try:
metrics = provider.get_vendor_metrics(db, vendor_id, context)
except Exception as e:
logger.warning(f"Failed to get {provider.metrics_category} metrics: {e}")
metrics = [] # Continue with empty metrics for this module
```
## Best Practices
### Do
- Use lazy imports inside metric methods to avoid circular imports
- Always use `VendorPlatform` junction table for platform-level queries
- Return empty list on error, don't raise exceptions
- Log warnings for debugging but don't crash
- Include helpful descriptions and icons for UI
### Don't
- Import from optional modules at the top of core module files
- Assume `Vendor.platform_id` exists (it doesn't!)
- Let exceptions propagate from metric providers
- Create hard dependencies between core and optional modules
## Related Documentation
- [Module System Architecture](module-system.md) - Module structure and auto-discovery
- [Multi-Tenant Architecture](multi-tenant.md) - Platform/vendor/company hierarchy
- [Middleware](middleware.md) - Request context flow
- [User Context Pattern](user-context-pattern.md) - JWT token context

2
docs/architecture/module-system.md
View File

@@ -242,6 +242,7 @@ analytics_module = ModuleDefinition(
| `is_core` | `bool` | Cannot be disabled if True |
| `is_internal` | `bool` | Admin-only if True |
| `is_self_contained` | `bool` | Uses self-contained structure |
| `metrics_provider` | `Callable` | Factory function returning MetricsProviderProtocol (see [Metrics Provider Pattern](metrics-provider-pattern.md)) |
## Route Auto-Discovery
@@ -1260,6 +1261,7 @@ python scripts/validate_architecture.py
## Related Documentation
- [Creating Modules](../development/creating-modules.md) - Step-by-step guide
- [Metrics Provider Pattern](metrics-provider-pattern.md) - Dashboard statistics architecture
- [Menu Management](menu-management.md) - Sidebar configuration
- [Observability](observability.md) - Health checks integration
- [Feature Gating](../implementation/feature-gating-system.md) - Tier-based access