Files
orion/docs/architecture/module-system.md
Samir Boulahtit 2466dfd7ed feat: add module config and migrations auto-discovery infrastructure
Add self-contained configuration and migrations support for modules:

Config auto-discovery (app/modules/config.py):
- Modules can have config.py with Pydantic Settings
- Environment variables prefixed with MODULE_NAME_
- Auto-discovered via get_module_config()

Migrations auto-discovery:
- Each module has migrations/versions/ directory
- Alembic discovers module migrations automatically
- Naming convention: {module}_{seq}_{description}.py

New architecture rules (MOD-013 to MOD-015):
- MOD-013: config.py should export config/config_class
- MOD-014: Migrations must follow naming convention
- MOD-015: Migrations directory must have __init__.py

Created for all 11 self-contained modules:
- config.py placeholder files
- migrations/ directories with __init__.py files

Added core and tenancy module definitions for completeness.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 22:19:41 +01:00

19 KiB

Module System Architecture

The Wizamart platform uses a plug-and-play modular architecture where modules are fully self-contained and automatically discovered. Simply create a module directory with the required structure, and the framework handles registration, routing, and resource loading automatically.

Key Features

  • Auto-Discovery: Modules are automatically discovered from app/modules/*/definition.py
  • Zero Configuration: No changes to main.py, registry.py, or other framework files needed
  • Self-Contained: Each module owns its routes, services, models, templates, and translations
  • Hot-Pluggable: Add or remove modules by simply adding/removing directories

Overview

┌─────────────────────────────────────────────────────────────────────┐
│                        FRAMEWORK LAYER                               │
│  (Infrastructure that modules depend on - not modules themselves)    │
│                                                                      │
│  Config │ Database │ Auth │ Permissions │ Observability │ Celery    │
└─────────────────────────────────────────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────┐
│                    AUTO-DISCOVERED MODULE LAYER                      │
│                                                                      │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │              CORE MODULES (Always Enabled)                   │    │
│  │     core  │  tenancy  │  cms  │  customers                  │    │
│  └─────────────────────────────────────────────────────────────┘    │
│                                                                      │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │           OPTIONAL MODULES (Per-Platform)                    │    │
│  │  payments │ billing │ inventory │ orders │ marketplace │ ...│    │
│  └─────────────────────────────────────────────────────────────┘    │
│                                                                      │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │            INTERNAL MODULES (Admin Only)                     │    │
│  │              dev-tools  │  monitoring                        │    │
│  └─────────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────┘

Auto-Discovery System

All module components are automatically discovered by the framework:

Component Discovery Location Auto-Loaded By
Registry */definition.py app/modules/discovery.py
Configuration */config.py app/modules/config.py
API Routes */routes/api/*.py app/modules/routes.py
Page Routes */routes/pages/*.py app/modules/routes.py
Tasks */tasks/__init__.py app/modules/tasks.py
Templates */templates/ app/templates_config.py
Static Files */static/ main.py
Locales */locales/*.json app/utils/i18n.py
Migrations */migrations/versions/ app/modules/migrations.py

Creating a New Module (Zero Framework Changes)

# 1. Create module directory
mkdir -p app/modules/mymodule/{routes/{api,pages},services,models,schemas,templates/mymodule/vendor,static/vendor/js,locales,tasks}

# 2. Create required files
touch app/modules/mymodule/__init__.py
touch app/modules/mymodule/definition.py
touch app/modules/mymodule/exceptions.py

# 3. That's it! The framework auto-discovers and registers everything.

Three-Tier Classification

Core Modules (4)

Core modules are always enabled and cannot be disabled. They provide fundamental platform functionality.

Module Description Key Features
core Dashboard, settings, profile Basic platform operation
tenancy Platform, company, vendor, admin user management Multi-tenant infrastructure
cms Content pages, media library, themes Content management
customers Customer database, profiles, segmentation Customer data management

Optional Modules (7)

Optional modules can be enabled or disabled per platform. They provide additional functionality that may not be needed by all platforms.

Module Dependencies Description
payments - Payment gateway integrations (Stripe, PayPal, etc.)
billing payments Platform subscriptions, vendor invoices
inventory - Stock management, locations
orders payments Order management, customer checkout
marketplace inventory Letzshop integration
analytics - Reports, dashboards
messaging - Messages, notifications

Internal Modules (2)

Internal modules are admin-only tools not exposed to customers or vendors.

Module Description
dev-tools Component library, icon browser
monitoring Logs, background tasks, Flower, Grafana integration

Self-Contained Module Structure

Every module follows this standardized structure:

app/modules/analytics/
├── __init__.py              # Module package marker
├── definition.py            # ModuleDefinition (REQUIRED for auto-discovery)
├── config.py                # Environment config (auto-discovered)
├── exceptions.py            # Module-specific exceptions
├── routes/
│   ├── __init__.py
│   ├── api/                 # API endpoints (auto-discovered)
│   │   ├── __init__.py
│   │   ├── admin.py         # Must export: router = APIRouter()
│   │   └── vendor.py        # Must export: router = APIRouter()
│   └── pages/               # HTML page routes (auto-discovered)
│       ├── __init__.py
│       └── vendor.py        # Must export: router = APIRouter()
├── services/
│   ├── __init__.py
│   └── stats_service.py
├── models/
│   ├── __init__.py
│   └── report.py
├── schemas/
│   ├── __init__.py
│   └── stats.py
├── templates/               # Auto-discovered by Jinja2
│   └── analytics/
│       └── vendor/
│           └── analytics.html
├── static/                  # Auto-mounted at /static/modules/analytics/
│   └── vendor/
│       └── js/
│           └── analytics.js
├── locales/                 # Auto-loaded translations
│   ├── en.json
│   ├── de.json
│   ├── fr.json
│   └── lu.json
├── tasks/                   # Auto-discovered by Celery
│   ├── __init__.py          # REQUIRED for Celery discovery
│   └── reports.py
└── migrations/              # Auto-discovered by Alembic
    ├── __init__.py          # REQUIRED for discovery
    └── versions/
        ├── __init__.py      # REQUIRED for discovery
        └── analytics_001_create_reports.py

Module Definition

Each module must have a definition.py with a ModuleDefinition instance:

# app/modules/analytics/definition.py
from app.modules.base import ModuleDefinition
from models.database.admin_menu_config import FrontendType

analytics_module = ModuleDefinition(
    # Identity
    code="analytics",
    name="Analytics & Reporting",
    description="Dashboard analytics, custom reports, and data exports.",
    version="1.0.0",

    # Classification (determines tier)
    is_core=False,      # Set True for core modules
    is_internal=False,  # Set True for admin-only modules

    # Dependencies
    requires=[],  # List other module codes this depends on

    # Features (for tier-based gating)
    features=[
        "basic_reports",
        "analytics_dashboard",
        "custom_reports",
    ],

    # Menu items per frontend
    menu_items={
        FrontendType.ADMIN: [],  # Analytics uses dashboard
        FrontendType.VENDOR: ["analytics"],
    },

    # Self-contained module configuration
    is_self_contained=True,
    services_path="app.modules.analytics.services",
    models_path="app.modules.analytics.models",
    schemas_path="app.modules.analytics.schemas",
    exceptions_path="app.modules.analytics.exceptions",
    templates_path="templates",
    locales_path="locales",
)

ModuleDefinition Fields

Field Type Description
code str Unique identifier (e.g., "billing")
name str Display name
description str What the module provides
version str Semantic version (default: "1.0.0")
requires list[str] Module codes this depends on
features list[str] Feature codes for tier gating
menu_items dict Menu items per frontend type
is_core bool Cannot be disabled if True
is_internal bool Admin-only if True
is_self_contained bool Uses self-contained structure

Route Auto-Discovery

Routes in routes/api/ and routes/pages/ are automatically discovered and registered.

API Routes (routes/api/vendor.py)

# app/modules/analytics/routes/api/vendor.py
from fastapi import APIRouter, Depends
from app.api.deps import get_current_vendor_api, get_db

router = APIRouter()  # MUST be named 'router' for auto-discovery

@router.get("")
def get_analytics(
    current_user = Depends(get_current_vendor_api),
    db = Depends(get_db),
):
    """Get vendor analytics."""
    pass

Auto-registered at: /api/v1/vendor/analytics

Page Routes (routes/pages/vendor.py)

# app/modules/analytics/routes/pages/vendor.py
from fastapi import APIRouter, Depends, Request
from fastapi.responses import HTMLResponse

router = APIRouter()  # MUST be named 'router' for auto-discovery

@router.get("/{vendor_code}/analytics", response_class=HTMLResponse)
async def analytics_page(request: Request, vendor_code: str):
    """Render analytics page."""
    pass

Auto-registered at: /vendor/{vendor_code}/analytics

Framework Layer

The Framework Layer provides infrastructure that modules depend on. These are not modules - they're always available and cannot be disabled.

Component Location Purpose
Config app/core/config.py Settings management
Database app/core/database.py SQLAlchemy sessions
Logging app/core/logging.py Structured logging
Permissions app/core/permissions.py RBAC definitions
Feature Gate app/core/feature_gate.py Tier-based access
Celery app/core/celery_config.py Task queue
Observability app/core/observability.py Health checks, metrics, Sentry
Auth Middleware middleware/auth.py JWT authentication
Context Middleware middleware/platform_context.py Multi-tenancy
Dependencies app/api/deps.py FastAPI DI
Base Exceptions app/exceptions/base.py Exception hierarchy

Module Dependencies

Modules can depend on other modules. When enabling a module, its dependencies are automatically enabled.

                    payments
                   ↙        ↘
              billing        orders

              inventory
                  ↓
             marketplace

Dependency Rules:

  1. Core modules cannot depend on optional modules
  2. Enabling a module auto-enables its dependencies
  3. Disabling a module auto-disables modules that depend on it
  4. Circular dependencies are not allowed

Module Registry

The registry auto-discovers all modules:

from app.modules.registry import (
    MODULES,           # All modules (auto-discovered)
    CORE_MODULES,      # Core only
    OPTIONAL_MODULES,  # Optional only
    INTERNAL_MODULES,  # Internal only
    get_module,
    get_core_module_codes,
    get_module_tier,
)

# Get a specific module
billing = get_module("billing")

# Check module tier
tier = get_module_tier("billing")  # Returns "optional"

# Get all core module codes
core_codes = get_core_module_codes()  # {"core", "tenancy", "cms", "customers"}

Module Service

The ModuleService manages module enablement per platform:

from app.modules.service import module_service

# Check if module is enabled
if module_service.is_module_enabled(db, platform_id, "billing"):
    pass

# Get all enabled modules for a platform
modules = module_service.get_platform_modules(db, platform_id)

# Enable a module (auto-enables dependencies)
module_service.enable_module(db, platform_id, "billing", user_id=current_user.id)

# Disable a module (auto-disables dependents)
module_service.disable_module(db, platform_id, "billing", user_id=current_user.id)

Module Configuration

Modules can have environment-based configuration using Pydantic Settings. The config.py file is auto-discovered by app/modules/config.py.

# app/modules/marketplace/config.py
from pydantic import Field
from pydantic_settings import BaseSettings

class MarketplaceConfig(BaseSettings):
    """Configuration for marketplace module."""

    # Settings loaded from env vars with MARKETPLACE_ prefix
    api_timeout: int = Field(default=30, description="API timeout in seconds")
    batch_size: int = Field(default=100, description="Import batch size")
    max_retries: int = Field(default=3, description="Max retry attempts")

    model_config = {"env_prefix": "MARKETPLACE_"}

# Export for auto-discovery
config_class = MarketplaceConfig
config = MarketplaceConfig()

Usage:

# Direct import
from app.modules.marketplace.config import config
timeout = config.api_timeout

# Via discovery
from app.modules.config import get_module_config
config = get_module_config("marketplace")

Environment variables:

MARKETPLACE_API_TIMEOUT=60
MARKETPLACE_BATCH_SIZE=500

Module Migrations

Each module owns its database migrations in the migrations/versions/ directory. Alembic auto-discovers these via app/modules/migrations.py.

Migration Structure

app/modules/cms/migrations/
├── __init__.py           # REQUIRED for discovery
└── versions/
    ├── __init__.py       # REQUIRED for discovery
    ├── cms_001_create_content_pages.py
    ├── cms_002_add_sections.py
    └── cms_003_add_media_library.py

Naming Convention

{module_code}_{sequence}_{description}.py

Migration Template

# app/modules/cms/migrations/versions/cms_001_create_content_pages.py
"""Create content_pages table.

Revision ID: cms_001
Create Date: 2026-01-28
"""
from alembic import op
import sqlalchemy as sa

revision = "cms_001"
down_revision = None
branch_labels = ("cms",)  # Module-specific branch

def upgrade() -> None:
    op.create_table(
        "content_pages",
        sa.Column("id", sa.Integer(), primary_key=True),
        sa.Column("vendor_id", sa.Integer(), sa.ForeignKey("vendors.id")),
        sa.Column("slug", sa.String(100), nullable=False),
        sa.Column("title", sa.String(200), nullable=False),
    )

def downgrade() -> None:
    op.drop_table("content_pages")

Running Migrations

Module migrations are automatically discovered:

# Run all migrations (core + modules)
alembic upgrade head

# View migration history
alembic history

Current State

Currently, all migrations reside in central alembic/versions/. The module-specific directories are in place for:

  • New modules: Should create migrations in their own migrations/versions/
  • Future reorganization: Existing migrations will be moved to modules pre-production

Architecture Validation Rules

The architecture validator (scripts/validate_architecture.py) enforces module structure:

Rule Severity Description
MOD-001 ERROR Self-contained modules must have required directories
MOD-002 WARNING Services must contain actual code, not re-exports
MOD-003 WARNING Schemas must contain actual code, not re-exports
MOD-004 WARNING Routes must import from module, not legacy locations
MOD-005 WARNING Modules with UI must have templates and static
MOD-006 INFO Modules should have locales for i18n
MOD-007 ERROR Definition paths must match directory structure
MOD-008 WARNING Self-contained modules must have exceptions.py
MOD-009 ERROR Modules must have definition.py for auto-discovery
MOD-010 WARNING Route files must export router variable
MOD-011 WARNING Tasks directory must have __init__.py
MOD-012 INFO Locales should have all language files
MOD-013 INFO config.py should export config or config_class
MOD-014 WARNING Migrations must follow naming convention
MOD-015 WARNING Migrations directory must have __init__.py files

Run validation:

python scripts/validate_architecture.py

Best Practices

Do

  • Keep modules focused on a single domain
  • Use requires for hard dependencies
  • Provide health_check for critical modules
  • Use events for cross-module communication
  • Follow the standard directory structure
  • Export router variable in route files
  • Include all supported languages in locales

Don't

  • Create circular dependencies
  • Make core modules depend on optional modules
  • Put framework-level code in modules
  • Skip migration naming conventions
  • Forget __init__.py in tasks directory
  • Manually register modules in registry.py (use auto-discovery)