# Cross-Module Import Rules

This document defines the strict import rules that ensure the module system remains decoupled, testable, and resilient. These rules are critical for maintaining a truly modular architecture.

## Core Principles

### Principle 1: Core modules NEVER import from optional modules

This is the fundamental rule that enables optional modules to be truly optional. When a core module imports from an optional module:
- The app crashes if that module is disabled
- You can't test core functionality in isolation
- You create a hidden dependency that violates the architecture

### Principle 2: Services over models — NEVER import another module's models

**If module A needs data from module B, it MUST call module B's service methods.**

Modules must NEVER import and query another module's SQLAlchemy models directly. This applies to ALL cross-module interactions — core-to-core, optional-to-core, and optional-to-optional.

When a module imports another module's models, it:
- Couples to the internal schema (column names, relationships, table structure)
- Bypasses business logic, validation, and access control in the owning service
- Makes refactoring the model owner's schema a breaking change for all consumers
- Scatters query logic across multiple modules instead of centralizing it

**The owning module's service is the ONLY authorized gateway to its data.**

## Module Classification

### Core Modules (Always Enabled)
- `contracts` - Protocols and interfaces
- `core` - Dashboard, settings, profile
- `tenancy` - Platform, merchant, store, admin user management
- `cms` - Content pages, media library
- `customers` - Customer database
- `billing` - Subscriptions, tier limits
- `payments` - Payment gateway integrations
- `messaging` - Email, notifications

### Optional Modules (Per-Platform)
- `analytics` - Reports, dashboards
- `cart` - Shopping cart
- `catalog` - Product browsing
- `checkout` - Cart-to-order conversion
- `inventory` - Stock management
- `loyalty` - Loyalty programs
- `marketplace` - Letzshop integration
- `orders` - Order management

## Import Rules Matrix

| From \ To | Core Services | Core Models | Optional Services | Optional Models | Contracts |
|-----------|--------------|-------------|-------------------|-----------------|-----------|
| **Core** | :white_check_mark: | :x: Use services | :x: FORBIDDEN | :x: FORBIDDEN | :white_check_mark: |
| **Optional** | :white_check_mark: | :x: Use services | :warning: With care | :x: Use services | :white_check_mark: |
| **Contracts** | :x: | :x: | :x: | :x: | :white_check_mark: |

### Explanation

1. **Any → Any Services**: Allowed (with core→optional restriction). Import the service, call its methods.

2. **Any → Any Models**: **FORBIDDEN**. Never import another module's SQLAlchemy models. Use that module's service instead.

3. **Core → Optional**: **FORBIDDEN** (both services and models). Use provider protocols instead.

4. **Optional → Core Services**: Allowed. Optional modules can call core service methods.

5. **Optional → Optional Services**: Allowed with care. Declare dependency in `definition.py`.

6. **Any → Contracts**: Allowed. Contracts define shared protocols and data structures.

7. **Contracts → Anything**: Contracts should only depend on stdlib/typing/Protocol. No module imports.

## Anti-Patterns (DO NOT DO)

### Cross-Module Model Import (MOD-025)

```python
# app/modules/orders/services/order_service.py

# BAD: Importing and querying another module's models
from app.modules.catalog.models import Product

class OrderService:
    def get_order_with_products(self, db, order_id):
        order = db.query(Order).filter_by(id=order_id).first()
        # BAD: Direct query on catalog's model
        products = db.query(Product).filter(Product.id.in_(product_ids)).all()
        return order, products
```

```python
# GOOD: Call the owning module's service
from app.modules.catalog.services import product_service

class OrderService:
    def get_order_with_products(self, db, order_id):
        order = db.query(Order).filter_by(id=order_id).first()
        # GOOD: Catalog service owns Product data access
        products = product_service.get_products_by_ids(db, product_ids)
        return order, products
```

### Cross-Module Aggregation Query

```python
# BAD: Counting another module's models directly
from app.modules.orders.models import Order
count = db.query(func.count(Order.id)).filter_by(store_id=store_id).scalar()

# GOOD: Ask the owning service
from app.modules.orders.services import order_service
count = order_service.get_order_count(db, store_id=store_id)
```

### Direct Import from Optional Module

```python
# app/modules/core/routes/api/admin_dashboard.py

# BAD: Core importing from optional module
from app.modules.marketplace.services import marketplace_service
from app.modules.analytics.services import stats_service

@router.get("/dashboard")
def get_dashboard():
    # This crashes if marketplace is disabled!
    imports = marketplace_service.get_recent_imports()
```

### Conditional Import with ismodule_enabled Check

```python
# BAD: Still creates import-time dependency
from app.modules.service import module_service

if module_service.is_module_enabled(db, platform_id, "marketplace"):
    from app.modules.marketplace.services import marketplace_service  # Still loaded!
    imports = marketplace_service.get_recent_imports()
```

### Type Hints from Optional Modules

```python
# BAD: Type hint forces import
from app.modules.marketplace.models import MarketplaceImportJob

def process_import(job: MarketplaceImportJob) -> None:  # Crashes if disabled
    ...
```

## Approved Patterns

### 0. Cross-Module Service Calls (Primary Pattern)

The default way to access another module's data is through its service layer:

```python
# app/modules/inventory/services/inventory_service.py

# GOOD: Import the service, not the model
from app.modules.catalog.services import product_service

class InventoryService:
    def get_stock_for_product(self, db, product_id):
        # Verify product exists via catalog service
        product = product_service.get_product_by_id(db, product_id)
        if not product:
            raise InventoryError("Product not found")
        # Query own models
        return db.query(StockLevel).filter_by(product_id=product_id).first()
```

Each module should expose these standard service methods for external consumers:

| Method Pattern | Purpose |
|---------------|---------|
| `get_{entity}_by_id(db, id)` | Single entity lookup |
| `list_{entities}(db, **filters)` | Filtered list |
| `get_{entity}_count(db, **filters)` | Count query |
| `search_{entities}(db, query, **filters)` | Text search |
| `get_{entities}_by_ids(db, ids)` | Batch lookup |

### 1. Provider Protocol Pattern (Metrics & Widgets)

Use the provider protocol pattern for cross-module data:

```python
# app/modules/core/services/stats_aggregator.py

# GOOD: Import only protocols and contracts
from app.modules.contracts.metrics import MetricsProviderProtocol, MetricValue

class StatsAggregatorService:
    def _get_enabled_providers(self, db, platform_id):
        # Discover providers from MODULES registry
        from app.modules.registry import MODULES

        for module in MODULES.values():
            if module.has_metrics_provider():
                # Provider is called through protocol, not direct import
                provider = module.get_metrics_provider_instance()
                yield provider
```

### 2. Context Provider Pattern

Modules contribute context without direct imports:

```python
# app/modules/core/utils.py

def get_context_for_frontend(frontend_type, request, db, platform):
    # GOOD: Discover and call providers through registry
    for module in get_enabled_modules(db, platform.id):
        if module.has_context_provider(frontend_type):
            context.update(module.get_context_contribution(...))
```

### 3. Lazy Factory Functions

Use factory functions in module definitions:

```python
# app/modules/marketplace/definition.py

def _get_widget_provider():
    """Lazy import to avoid circular imports."""
    from app.modules.marketplace.services.marketplace_widgets import marketplace_widget_provider
    return marketplace_widget_provider

marketplace_module = ModuleDefinition(
    code="marketplace",
    widget_provider=_get_widget_provider,  # Called lazily when needed
)
```

### 4. Type Checking Only Imports

Use `TYPE_CHECKING` for type hints without runtime dependency. Pair with `from __future__ import annotations` to avoid quoting type hints:

```python
from __future__ import annotations

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from app.modules.marketplace.models import MarketplaceImportJob

# With `from __future__ import annotations`, no string quoting needed
def process_import(job: MarketplaceImportJob | None = None) -> None:
    # Works at runtime even if marketplace is disabled
    ...
```

> **Note:** Without `from __future__ import annotations`, you must quote the type hint as `"MarketplaceImportJob"` to prevent a `NameError` at class/function definition time.

### 5. Registry-Based Discovery

Discover modules through the registry, not imports:

```python
# GOOD: Discovery through registry
from app.modules.registry import MODULES

def get_module_if_enabled(db, platform_id, module_code):
    module = MODULES.get(module_code)
    if module and module_service.is_module_enabled(db, platform_id, module_code):
        return module
    return None
```

### 6. Method-Body Deferred Imports

When a service needs to **query** another module's models directly (e.g., the model's owning service doesn't exist yet, or the service IS the gateway for a misplaced model), defer the import into the method body. This moves the import from module-load time to first-call time, breaking circular import chains and keeping the module boundary clean.

Choose the sub-pattern based on how many methods use the model:

#### 6a. Simple method-body import (1-2 methods)

When only one or two methods need the model, import directly inside the method:

```python
# app/modules/catalog/services/product_service.py

class ProductService:
    def create_product(self, db, data):
        # Deferred: only this method needs MarketplaceProduct
        from app.modules.marketplace.models import MarketplaceProduct

        mp = db.query(MarketplaceProduct).filter(
            MarketplaceProduct.id == data.marketplace_product_id
        ).first()
        ...
```

#### 6b. Module-level `_get_model()` helper (3+ methods, same model)

When many methods in one file use the same cross-module model, create a module-level deferred helper to avoid repeating the import:

```python
# app/modules/monitoring/services/log_service.py

def _get_application_log_model():
    """Deferred import for ApplicationLog (lives in tenancy, consumed by monitoring)."""
    from app.modules.tenancy.models import ApplicationLog
    return ApplicationLog


class LogService:
    def get_database_logs(self, db, filters):
        ApplicationLog = _get_application_log_model()
        return db.query(ApplicationLog).filter(...).all()

    def get_log_statistics(self, db, days=7):
        ApplicationLog = _get_application_log_model()
        return db.query(func.count(ApplicationLog.id)).scalar()

    def cleanup_old_logs(self, db, retention_days):
        ApplicationLog = _get_application_log_model()
        db.query(ApplicationLog).filter(...).delete()
```

#### 6c. Instance-cached models (pervasive usage across class)

When a model is used in nearly every method of a class, cache it on the instance at init time to avoid repetitive local assignments:

```python
# app/modules/marketplace/services/letzshop/order_service.py

def _get_order_models():
    """Deferred import for Order/OrderItem models."""
    from app.modules.orders.models import Order, OrderItem
    return Order, OrderItem


class LetzshopOrderService:
    def __init__(self, db):
        self.db = db
        self._Order, self._OrderItem = _get_order_models()

    def get_order(self, store_id, order_id):
        return self.db.query(self._Order).filter(
            self._Order.id == order_id,
            self._Order.store_id == store_id,
        ).first()

    def get_order_items(self, order):
        return self.db.query(self._OrderItem).filter(
            self._OrderItem.order_id == order.id
        ).all()
```

#### When to use each sub-pattern

| Model usage in file | Pattern | Example |
|---------------------|---------|---------|
| 1-2 methods | 6a: method-body import | `product_media_service.py` → `MediaFile` |
| 3+ methods, same model | 6b: `_get_model()` helper | `log_service.py` → `ApplicationLog` |
| Nearly every method | 6c: instance-cached | `letzshop/order_service.py` → `Order` |

> **When NOT to use these patterns:** If the owning module already has a service method that returns the data you need, call that service instead of querying the model. Deferred imports are for cases where the model's service doesn't expose the needed query, or the service IS the canonical gateway for a misplaced infrastructure model.

## Architecture Validation

The architecture validator (`scripts/validate/validate_architecture.py`) includes rules to detect violations:

| Rule | Severity | Description |
|------|----------|-------------|
| IMPORT-001 | ERROR | Core module imports from optional module |
| IMPORT-002 | WARNING | Optional module imports from unrelated optional module |
| IMPORT-003 | INFO | Consider using protocol pattern instead of direct import |
| MOD-025 | ERROR | Module imports models from another module (use services) |
| MOD-026 | ERROR | Cross-module data access not going through service layer |

Run validation:
```bash
python scripts/validate/validate_architecture.py
```

## Provider Pattern Summary

### When to Use Each Pattern

| Need | Pattern | Location |
|------|---------|----------|
| Numeric statistics | MetricsProvider | `contracts/metrics.py` |
| Dashboard widgets | WidgetProvider | `contracts/widgets.py` |
| Page template context | Context Provider | `definition.py` |
| Service access | Protocol + DI | `contracts/{module}.py` |

### Provider Flow

```
┌─────────────────────────────────────────────────────────────────────┐
│                         CONTRACTS LAYER                              │
│                     (Shared protocol definitions)                    │
│                                                                      │
│   MetricsProviderProtocol    DashboardWidgetProviderProtocol        │
│   ContentServiceProtocol     ServiceProtocol                         │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
        ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
        │  CORE MODULE     │ │ OPTIONAL MODULE 1│ │ OPTIONAL MODULE 2│
        │                  │ │                  │ │                  │
        │ StatsAggregator  │ │ OrderMetrics     │ │ CatalogMetrics   │
        │ WidgetAggregator │ │ (implements)     │ │ (implements)     │
        │ (discovers)      │ │                  │ │                  │
        └──────────────────┘ └──────────────────┘ └──────────────────┘
                │                       │                    │
                │                       │                    │
                ▼                       ▼                    ▼
        ┌──────────────────────────────────────────────────────────────┐
        │                      MODULE REGISTRY                          │
        │                                                               │
        │  MODULES = {                                                  │
        │      "orders": ModuleDefinition(metrics_provider=...),        │
        │      "catalog": ModuleDefinition(metrics_provider=...),       │
        │  }                                                            │
        └──────────────────────────────────────────────────────────────┘
```

## Testing Without Dependencies

Because core doesn't depend on optional modules, you can test core in isolation:

```python
# tests/unit/core/test_stats_aggregator.py

from unittest.mock import MagicMock
from app.modules.contracts.metrics import MetricValue, MetricsProviderProtocol

class MockMetricsProvider:
    """Mock that implements the protocol without needing real modules."""
    @property
    def metrics_category(self) -> str:
        return "test"

    def get_store_metrics(self, db, store_id, context=None):
        return [MetricValue(key="test.value", value=42, label="Test", category="test")]

def test_stats_aggregator_with_mock_provider():
    # Test core without any optional modules
    mock_provider = MockMetricsProvider()
    ...
```

## Consequences of Violations

### If Core Imports Optional Module

1. **Import Error**: App fails to start if module code has syntax errors
2. **Runtime Crash**: App crashes when disabled module is accessed
3. **Hidden Dependency**: Module appears optional but isn't
4. **Testing Difficulty**: Can't test core without all modules
5. **Deployment Issues**: Can't deploy minimal configurations

### Detection

Add these checks to CI:

```bash
# Check for forbidden imports
grep -r "from app.modules.marketplace" app/modules/core/ && exit 1
grep -r "from app.modules.analytics" app/modules/core/ && exit 1
grep -r "from app.modules.orders" app/modules/core/ && exit 1
```

## Summary

| Rule | Enforcement |
|------|-------------|
| Core → Optional = FORBIDDEN | Architecture validation, CI checks |
| Cross-module model imports = FORBIDDEN | MOD-025 rule, code review |
| Use services for cross-module data | MOD-026 rule, code review |
| Use Protocol pattern for core→optional | Code review, documentation |
| Lazy factory functions | Required for definition.py |
| TYPE_CHECKING imports | Required for type hints across modules |
| Registry-based discovery | Required for all cross-module access |

Following these rules ensures:
- Modules can be truly enabled/disabled per platform
- Testing can be done in isolation
- New modules can be added without modifying core
- The app remains stable when modules fail
- Module schemas can evolve independently
- Data access logic is centralized in the owning service

## Related Documentation

- [Module System Architecture](module-system.md) - Module structure and auto-discovery
- [Metrics Provider Pattern](metrics-provider-pattern.md) - Numeric statistics architecture
- [Widget Provider Pattern](widget-provider-pattern.md) - Dashboard widgets architecture
- [Architecture Violations Status](architecture-violations-status.md) - Current violation tracking
- [Cross-Module Migration Plan](cross-module-migration-plan.md) - Migration plan for resolving all cross-module violations