orion/docs/architecture/cross-module-import-rules.md

# Cross-Module Import Rules

This document defines the strict import rules that ensure the module system remains decoupled, testable, and resilient. These rules are critical for maintaining a truly modular architecture.

## Core Principles

### Principle 1: Core modules NEVER import from optional modules

This is the fundamental rule that enables optional modules to be truly optional. When a core module imports from an optional module:
- The app crashes if that module is disabled
- You can't test core functionality in isolation
- You create a hidden dependency that violates the architecture

### Principle 2: Services over models — NEVER import another module's models

**If module A needs data from module B, it MUST call module B's service methods.**

Modules must NEVER import and query another module's SQLAlchemy models directly. This applies to ALL cross-module interactions — core-to-core, optional-to-core, and optional-to-optional.

When a module imports another module's models, it:
- Couples to the internal schema (column names, relationships, table structure)
- Bypasses business logic, validation, and access control in the owning service
- Makes refactoring the model owner's schema a breaking change for all consumers
- Scatters query logic across multiple modules instead of centralizing it

**The owning module's service is the ONLY authorized gateway to its data.**

## Module Classification

### Core Modules (Always Enabled)
- `contracts` - Protocols and interfaces
- `core` - Dashboard, settings, profile
- `tenancy` - Platform, merchant, store, admin user management
- `cms` - Content pages, media library
- `customers` - Customer database
- `billing` - Subscriptions, tier limits
- `payments` - Payment gateway integrations
- `messaging` - Email, notifications

### Optional Modules (Per-Platform)
- `analytics` - Reports, dashboards
- `cart` - Shopping cart
- `catalog` - Product browsing
- `checkout` - Cart-to-order conversion
- `inventory` - Stock management
- `loyalty` - Loyalty programs
- `marketplace` - Letzshop integration
- `orders` - Order management

## Import Rules Matrix

| From \ To | Core Services | Core Models | Optional Services | Optional Models | Contracts |
|-----------|--------------|-------------|-------------------|-----------------|-----------|
| **Core** | :white_check_mark: | :x: Use services | :x: FORBIDDEN | :x: FORBIDDEN | :white_check_mark: |
| **Optional** | :white_check_mark: | :x: Use services | :warning: With care | :x: Use services | :white_check_mark: |
| **Contracts** | :x: | :x: | :x: | :x: | :white_check_mark: |

### Explanation

1. **Any → Any Services**: Allowed (with core→optional restriction). Import the service, call its methods.

2. **Any → Any Models**: **FORBIDDEN**. Never import another module's SQLAlchemy models. Use that module's service instead.

3. **Core → Optional**: **FORBIDDEN** (both services and models). Use provider protocols instead.

4. **Optional → Core Services**: Allowed. Optional modules can call core service methods.

5. **Optional → Optional Services**: Allowed with care. Declare dependency in `definition.py`.

6. **Any → Contracts**: Allowed. Contracts define shared protocols and data structures.

7. **Contracts → Anything**: Contracts should only depend on stdlib/typing/Protocol. No module imports.

## Anti-Patterns (DO NOT DO)

### Cross-Module Model Import (MOD-025)

```python
# app/modules/orders/services/order_service.py

# BAD: Importing and querying another module's models
from app.modules.catalog.models import Product

class OrderService:
    def get_order_with_products(self, db, order_id):
        order = db.query(Order).filter_by(id=order_id).first()
        # BAD: Direct query on catalog's model
        products = db.query(Product).filter(Product.id.in_(product_ids)).all()
        return order, products
```

```python
# GOOD: Call the owning module's service
from app.modules.catalog.services import product_service

class OrderService:
    def get_order_with_products(self, db, order_id):
        order = db.query(Order).filter_by(id=order_id).first()
        # GOOD: Catalog service owns Product data access
        products = product_service.get_products_by_ids(db, product_ids)
        return order, products
```

### Cross-Module Aggregation Query

```python
# BAD: Counting another module's models directly
from app.modules.orders.models import Order
count = db.query(func.count(Order.id)).filter_by(store_id=store_id).scalar()

# GOOD: Ask the owning service
from app.modules.orders.services import order_service
count = order_service.get_order_count(db, store_id=store_id)
```

### Direct Import from Optional Module

```python
# app/modules/core/routes/api/admin_dashboard.py

# BAD: Core importing from optional module
from app.modules.marketplace.services import marketplace_service
from app.modules.analytics.services import stats_service

@router.get("/dashboard")
def get_dashboard():
    # This crashes if marketplace is disabled!
    imports = marketplace_service.get_recent_imports()
```

### Conditional Import with ismodule_enabled Check

```python
# BAD: Still creates import-time dependency
from app.modules.service import module_service

if module_service.is_module_enabled(db, platform_id, "marketplace"):
    from app.modules.marketplace.services import marketplace_service  # Still loaded!
    imports = marketplace_service.get_recent_imports()
```

### Type Hints from Optional Modules

```python
# BAD: Type hint forces import
from app.modules.marketplace.models import MarketplaceImportJob

def process_import(job: MarketplaceImportJob) -> None:  # Crashes if disabled
    ...
```

## Approved Patterns

### 0. Cross-Module Service Calls (Primary Pattern)

The default way to access another module's data is through its service layer:

```python
# app/modules/inventory/services/inventory_service.py

# GOOD: Import the service, not the model
from app.modules.catalog.services import product_service

class InventoryService:
    def get_stock_for_product(self, db, product_id):
        # Verify product exists via catalog service
        product = product_service.get_product_by_id(db, product_id)
        if not product:
            raise InventoryError("Product not found")
        # Query own models
        return db.query(StockLevel).filter_by(product_id=product_id).first()
```

Each module should expose these standard service methods for external consumers:

| Method Pattern | Purpose |
|---------------|---------|
| `get_{entity}_by_id(db, id)` | Single entity lookup |
| `list_{entities}(db, **filters)` | Filtered list |
| `get_{entity}_count(db, **filters)` | Count query |
| `search_{entities}(db, query, **filters)` | Text search |
| `get_{entities}_by_ids(db, ids)` | Batch lookup |

### 1. Provider Protocol Pattern (Metrics & Widgets)

Use the provider protocol pattern for cross-module data:

```python
# app/modules/core/services/stats_aggregator.py

# GOOD: Import only protocols and contracts
from app.modules.contracts.metrics import MetricsProviderProtocol, MetricValue

class StatsAggregatorService:
    def _get_enabled_providers(self, db, platform_id):
        # Discover providers from MODULES registry
        from app.modules.registry import MODULES

        for module in MODULES.values():
            if module.has_metrics_provider():
                # Provider is called through protocol, not direct import
                provider = module.get_metrics_provider_instance()
                yield provider
```

### 2. Context Provider Pattern

Modules contribute context without direct imports:

```python
# app/modules/core/utils.py

def get_context_for_frontend(frontend_type, request, db, platform):
    # GOOD: Discover and call providers through registry
    for module in get_enabled_modules(db, platform.id):
        if module.has_context_provider(frontend_type):
            context.update(module.get_context_contribution(...))
```

### 3. Lazy Factory Functions

Use factory functions in module definitions:

```python
# app/modules/marketplace/definition.py

def _get_widget_provider():
    """Lazy import to avoid circular imports."""
    from app.modules.marketplace.services.marketplace_widgets import marketplace_widget_provider
    return marketplace_widget_provider

marketplace_module = ModuleDefinition(
    code="marketplace",
    widget_provider=_get_widget_provider,  # Called lazily when needed
)
```

### 4. Type Checking Only Imports

Use `TYPE_CHECKING` for type hints without runtime dependency:

```python
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from app.modules.marketplace.models import MarketplaceImportJob

def process_import(job: "MarketplaceImportJob") -> None:
    # Works at runtime even if marketplace is disabled
    ...
```

### 5. Registry-Based Discovery

Discover modules through the registry, not imports:

```python
# GOOD: Discovery through registry
from app.modules.registry import MODULES

def get_module_if_enabled(db, platform_id, module_code):
    module = MODULES.get(module_code)
    if module and module_service.is_module_enabled(db, platform_id, module_code):
        return module
    return None
```

## Architecture Validation

The architecture validator (`scripts/validate/validate_architecture.py`) includes rules to detect violations:

| Rule | Severity | Description |
|------|----------|-------------|
| IMPORT-001 | ERROR | Core module imports from optional module |
| IMPORT-002 | WARNING | Optional module imports from unrelated optional module |
| IMPORT-003 | INFO | Consider using protocol pattern instead of direct import |
| MOD-025 | ERROR | Module imports models from another module (use services) |
| MOD-026 | ERROR | Cross-module data access not going through service layer |

Run validation:
```bash
python scripts/validate/validate_architecture.py
```

## Provider Pattern Summary

### When to Use Each Pattern

| Need | Pattern | Location |
|------|---------|----------|
| Numeric statistics | MetricsProvider | `contracts/metrics.py` |
| Dashboard widgets | WidgetProvider | `contracts/widgets.py` |
| Page template context | Context Provider | `definition.py` |
| Service access | Protocol + DI | `contracts/{module}.py` |

### Provider Flow

```
┌─────────────────────────────────────────────────────────────────────┐
│                         CONTRACTS LAYER                              │
│                     (Shared protocol definitions)                    │
│                                                                      │
│   MetricsProviderProtocol    DashboardWidgetProviderProtocol        │
│   ContentServiceProtocol     ServiceProtocol                         │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
        ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
        │  CORE MODULE     │ │ OPTIONAL MODULE 1│ │ OPTIONAL MODULE 2│
        │                  │ │                  │ │                  │
        │ StatsAggregator  │ │ OrderMetrics     │ │ CatalogMetrics   │
        │ WidgetAggregator │ │ (implements)     │ │ (implements)     │
        │ (discovers)      │ │                  │ │                  │
        └──────────────────┘ └──────────────────┘ └──────────────────┘
                │                       │                    │
                │                       │                    │
                ▼                       ▼                    ▼
        ┌──────────────────────────────────────────────────────────────┐
        │                      MODULE REGISTRY                          │
        │                                                               │
        │  MODULES = {                                                  │
        │      "orders": ModuleDefinition(metrics_provider=...),        │
        │      "catalog": ModuleDefinition(metrics_provider=...),       │
        │  }                                                            │
        └──────────────────────────────────────────────────────────────┘
```

## Testing Without Dependencies

Because core doesn't depend on optional modules, you can test core in isolation:

```python
# tests/unit/core/test_stats_aggregator.py

from unittest.mock import MagicMock
from app.modules.contracts.metrics import MetricValue, MetricsProviderProtocol

class MockMetricsProvider:
    """Mock that implements the protocol without needing real modules."""
    @property
    def metrics_category(self) -> str:
        return "test"

    def get_store_metrics(self, db, store_id, context=None):
        return [MetricValue(key="test.value", value=42, label="Test", category="test")]

def test_stats_aggregator_with_mock_provider():
    # Test core without any optional modules
    mock_provider = MockMetricsProvider()
    ...
```

## Consequences of Violations

### If Core Imports Optional Module

1. **Import Error**: App fails to start if module code has syntax errors
2. **Runtime Crash**: App crashes when disabled module is accessed
3. **Hidden Dependency**: Module appears optional but isn't
4. **Testing Difficulty**: Can't test core without all modules
5. **Deployment Issues**: Can't deploy minimal configurations

### Detection

Add these checks to CI:

```bash
# Check for forbidden imports
grep -r "from app.modules.marketplace" app/modules/core/ && exit 1
grep -r "from app.modules.analytics" app/modules/core/ && exit 1
grep -r "from app.modules.orders" app/modules/core/ && exit 1
```

## Summary

| Rule | Enforcement |
|------|-------------|
| Core → Optional = FORBIDDEN | Architecture validation, CI checks |
| Cross-module model imports = FORBIDDEN | MOD-025 rule, code review |
| Use services for cross-module data | MOD-026 rule, code review |
| Use Protocol pattern for core→optional | Code review, documentation |
| Lazy factory functions | Required for definition.py |
| TYPE_CHECKING imports | Required for type hints across modules |
| Registry-based discovery | Required for all cross-module access |

Following these rules ensures:
- Modules can be truly enabled/disabled per platform
- Testing can be done in isolation
- New modules can be added without modifying core
- The app remains stable when modules fail
- Module schemas can evolve independently
- Data access logic is centralized in the owning service

## Related Documentation

- [Module System Architecture](module-system.md) - Module structure and auto-discovery
- [Metrics Provider Pattern](metrics-provider-pattern.md) - Numeric statistics architecture
- [Widget Provider Pattern](widget-provider-pattern.md) - Dashboard widgets architecture
- [Architecture Violations Status](architecture-violations-status.md) - Current violation tracking
- [Cross-Module Migration Plan](cross-module-migration-plan.md) - Migration plan for resolving all cross-module violations