refactor: migrate remaining routes to modules and enforce auto-discovery

MIGRATION:
- Delete app/api/v1/vendor/analytics.py (duplicate - analytics module already auto-discovered)
- Move usage routes from app/api/v1/vendor/usage.py to billing module
- Move onboarding routes from app/api/v1/vendor/onboarding.py to marketplace module
- Move features routes to billing module (admin + vendor)
- Move inventory routes to inventory module (admin + vendor)
- Move marketplace/letzshop routes to marketplace module
- Move orders routes to orders module
- Delete legacy letzshop service files (moved to marketplace module)

DOCUMENTATION:
- Add docs/development/migration/module-autodiscovery-migration.md with full migration history
- Update docs/architecture/module-system.md with Entity Auto-Discovery Reference section
- Add detailed sections for each entity type: routes, services, models, schemas, tasks,
  exceptions, templates, static files, locales, configuration

ARCHITECTURE VALIDATION:
- Add MOD-016: Routes must be in modules, not app/api/v1/
- Add MOD-017: Services must be in modules, not app/services/
- Add MOD-018: Tasks must be in modules, not app/tasks/
- Add MOD-019: Schemas must be in modules, not models/schema/
- Update scripts/validate_architecture.py with _validate_legacy_locations method
- Update .architecture-rules/module.yaml with legacy location rules

These rules enforce that all entities must be in self-contained modules.
Legacy locations now trigger ERROR severity violations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-31 14:25:59 +01:00
parent e2cecff014
commit 401db56258
52 changed files with 1160 additions and 4968 deletions

View File

@@ -499,6 +499,381 @@ Currently, all migrations reside in central `alembic/versions/`. The module-spec
- **New modules**: Should create migrations in their own `migrations/versions/`
- **Future reorganization**: Existing migrations will be moved to modules pre-production
## Entity Auto-Discovery Reference
This section details the auto-discovery requirements for each entity type. **All entities must be in modules** - legacy locations are deprecated and will trigger architecture validation errors.
### Routes
Routes define API and page endpoints. They are auto-discovered from module directories.
| Type | Location | Discovery | Router Name |
|------|----------|-----------|-------------|
| Vendor API | `routes/api/vendor.py` | `app/modules/routes.py` | `vendor_router` |
| Admin API | `routes/api/admin.py` | `app/modules/routes.py` | `admin_router` |
| Shop API | `routes/api/shop.py` | `app/modules/routes.py` | `shop_router` |
| Vendor Pages | `routes/pages/vendor.py` | `app/modules/routes.py` | `vendor_router` |
| Admin Pages | `routes/pages/admin.py` | `app/modules/routes.py` | `admin_router` |
**Structure:**
```
app/modules/{module}/routes/
├── __init__.py
├── api/
│ ├── __init__.py
│ ├── vendor.py # Must export vendor_router
│ ├── admin.py # Must export admin_router
│ └── vendor_{feature}.py # Sub-routers aggregated in vendor.py
└── pages/
├── __init__.py
└── vendor.py # Must export vendor_router
```
**Example - Aggregating Sub-Routers:**
```python
# app/modules/billing/routes/api/vendor.py
from fastapi import APIRouter, Depends
from app.api.deps import require_module_access
vendor_router = APIRouter(
prefix="/billing",
dependencies=[Depends(require_module_access("billing"))],
)
# Aggregate sub-routers
from .vendor_checkout import vendor_checkout_router
from .vendor_usage import vendor_usage_router
vendor_router.include_router(vendor_checkout_router)
vendor_router.include_router(vendor_usage_router)
```
**Legacy Locations (DEPRECATED - will cause errors):**
- `app/api/v1/vendor/*.py` - Move to module `routes/api/vendor.py`
- `app/api/v1/admin/*.py` - Move to module `routes/api/admin.py`
---
### Services
Services contain business logic. They are not auto-discovered but should be in modules for organization.
| Location | Import Pattern |
|----------|----------------|
| `services/*.py` | `from app.modules.{module}.services import service_name` |
| `services/__init__.py` | Re-exports all public services |
**Structure:**
```
app/modules/{module}/services/
├── __init__.py # Re-exports: from .order_service import order_service
├── order_service.py # OrderService class + order_service singleton
└── fulfillment_service.py # Related services
```
**Example:**
```python
# app/modules/orders/services/order_service.py
from sqlalchemy.orm import Session
from app.modules.orders.models import Order
class OrderService:
def get_order(self, db: Session, order_id: int) -> Order:
return db.query(Order).filter(Order.id == order_id).first()
order_service = OrderService()
# app/modules/orders/services/__init__.py
from .order_service import order_service, OrderService
__all__ = ["order_service", "OrderService"]
```
**Legacy Locations (DEPRECATED - will cause errors):**
- `app/services/*.py` - Move to module `services/`
- `app/services/{module}/` - Move to `app/modules/{module}/services/`
---
### Models
Database models (SQLAlchemy). Currently in `models/database/`, migrating to modules.
| Location | Base Class | Discovery |
|----------|------------|-----------|
| `models/*.py` | `Base` from `models.base` | Alembic autogenerate |
**Structure:**
```
app/modules/{module}/models/
├── __init__.py # Re-exports: from .order import Order, OrderItem
├── order.py # Order model
└── order_item.py # Related models
```
**Example:**
```python
# app/modules/orders/models/order.py
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from models.base import Base, TimestampMixin
class Order(Base, TimestampMixin):
__tablename__ = "orders"
id = Column(Integer, primary_key=True)
vendor_id = Column(Integer, ForeignKey("vendors.id"), nullable=False)
status = Column(String(50), default="pending")
items = relationship("OrderItem", back_populates="order")
```
**Legacy Locations (being migrated):**
- `models/database/*.py` - Core models remain here, domain models move to modules
---
### Schemas
Pydantic schemas for request/response validation.
| Location | Base Class | Usage |
|----------|------------|-------|
| `schemas/*.py` | `BaseModel` from Pydantic | API routes, validation |
**Structure:**
```
app/modules/{module}/schemas/
├── __init__.py # Re-exports all schemas
├── order.py # Order request/response schemas
└── order_item.py # Related schemas
```
**Example:**
```python
# app/modules/orders/schemas/order.py
from pydantic import BaseModel
from datetime import datetime
class OrderResponse(BaseModel):
id: int
vendor_id: int
status: str
created_at: datetime
class Config:
from_attributes = True
class OrderCreateRequest(BaseModel):
customer_id: int
items: list[OrderItemRequest]
```
**Legacy Locations (DEPRECATED - will cause errors):**
- `models/schema/*.py` - Move to module `schemas/`
---
### Tasks (Celery)
Background tasks are auto-discovered by Celery from module `tasks/` directories.
| Location | Discovery | Registration |
|----------|-----------|--------------|
| `tasks/*.py` | `app/modules/tasks.py` | Celery autodiscover |
**Structure:**
```
app/modules/{module}/tasks/
├── __init__.py # REQUIRED - imports task functions
├── import_tasks.py # Task definitions
└── export_tasks.py # Related tasks
```
**Example:**
```python
# app/modules/marketplace/tasks/import_tasks.py
from celery import shared_task
from app.core.database import SessionLocal
@shared_task(bind=True)
def process_import(self, job_id: int, vendor_id: int):
db = SessionLocal()
try:
# Process import
pass
finally:
db.close()
# app/modules/marketplace/tasks/__init__.py
from .import_tasks import process_import
from .export_tasks import export_products
__all__ = ["process_import", "export_products"]
```
**Legacy Locations (DEPRECATED - will cause errors):**
- `app/tasks/*.py` - Move to module `tasks/`
---
### Exceptions
Module-specific exceptions inherit from `WizamartException`.
| Location | Base Class | Usage |
|----------|------------|-------|
| `exceptions.py` | `WizamartException` | Domain errors |
**Structure:**
```
app/modules/{module}/
└── exceptions.py # All module exceptions
```
**Example:**
```python
# app/modules/orders/exceptions.py
from app.exceptions import WizamartException
class OrderException(WizamartException):
"""Base exception for orders module."""
pass
class OrderNotFoundError(OrderException):
"""Order not found."""
def __init__(self, order_id: int):
super().__init__(f"Order {order_id} not found")
self.order_id = order_id
class OrderAlreadyFulfilledError(OrderException):
"""Order has already been fulfilled."""
pass
```
---
### Templates
Jinja2 templates are auto-discovered from module `templates/` directories.
| Location | URL Pattern | Discovery |
|----------|-------------|-----------|
| `templates/{module}/vendor/*.html` | `/vendor/{vendor}/...` | Jinja2 loader |
| `templates/{module}/admin/*.html` | `/admin/...` | Jinja2 loader |
**Structure:**
```
app/modules/{module}/templates/
└── {module}/
├── vendor/
│ ├── index.html
│ └── detail.html
└── admin/
└── list.html
```
**Template Reference:**
```python
# In route
return templates.TemplateResponse(
request=request,
name="{module}/vendor/index.html",
context={"items": items}
)
```
---
### Static Files
JavaScript, CSS, and images are auto-mounted from module `static/` directories.
| Location | URL | Discovery |
|----------|-----|-----------|
| `static/vendor/js/*.js` | `/static/modules/{module}/vendor/js/*.js` | `main.py` |
| `static/admin/js/*.js` | `/static/modules/{module}/admin/js/*.js` | `main.py` |
**Structure:**
```
app/modules/{module}/static/
├── vendor/js/
│ └── {module}.js
├── admin/js/
│ └── {module}.js
└── shared/js/
└── common.js
```
**Template Reference:**
```html
<script src="{{ url_for('{module}_static', path='vendor/js/{module}.js') }}"></script>
```
---
### Locales (i18n)
Translation files are auto-discovered from module `locales/` directories.
| Location | Format | Discovery |
|----------|--------|-----------|
| `locales/*.json` | JSON key-value | `app/utils/i18n.py` |
**Structure:**
```
app/modules/{module}/locales/
├── en.json
├── de.json
├── fr.json
└── lb.json
```
**Example:**
```json
{
"orders.title": "Orders",
"orders.status.pending": "Pending",
"orders.status.fulfilled": "Fulfilled"
}
```
**Usage:**
```python
from app.utils.i18n import t
message = t("orders.title", locale="en") # "Orders"
```
---
### Configuration
Module-specific environment configuration.
| Location | Base Class | Discovery |
|----------|------------|-----------|
| `config.py` | `BaseSettings` | `app/modules/config.py` |
**Example:**
```python
# app/modules/marketplace/config.py
from pydantic_settings import BaseSettings
class MarketplaceConfig(BaseSettings):
api_timeout: int = 30
batch_size: int = 100
model_config = {"env_prefix": "MARKETPLACE_"}
config = MarketplaceConfig()
```
**Environment Variables:**
```bash
MARKETPLACE_API_TIMEOUT=60
MARKETPLACE_BATCH_SIZE=500
```
## Architecture Validation Rules
The architecture validator (`scripts/validate_architecture.py`) enforces module structure:
@@ -520,6 +895,10 @@ The architecture validator (`scripts/validate_architecture.py`) enforces module
| MOD-013 | INFO | config.py should export `config` or `config_class` |
| MOD-014 | WARNING | Migrations must follow naming convention |
| MOD-015 | WARNING | Migrations directory must have `__init__.py` files |
| MOD-016 | ERROR | Routes must be in modules, not `app/api/v1/` |
| MOD-017 | ERROR | Services must be in modules, not `app/services/` |
| MOD-018 | ERROR | Tasks must be in modules, not `app/tasks/` |
| MOD-019 | ERROR | Schemas must be in modules, not `models/schema/` |
Run validation:
```bash

View File

@@ -0,0 +1,170 @@
# Module Auto-Discovery Migration History
This document tracks the migration of legacy code to the self-contained module architecture with auto-discovery.
## Overview
The Wizamart platform has been migrating from a monolithic structure with code in centralized locations (`app/api/v1/`, `app/services/`, `models/`) to a fully modular architecture where each module owns all its entities (routes, services, models, schemas, tasks).
## Migration Goals
1. **Self-Contained Modules**: Each module in `app/modules/{module}/` owns all its code
2. **Auto-Discovery**: All entities are automatically discovered - no manual registration
3. **No Legacy Dependencies**: Modules should not import from legacy locations
4. **Zero Framework Changes**: Adding/removing modules requires no changes to core framework
## Migration Timeline
### Phase 1: Foundation (2026-01-28)
#### Commit: `1ef5089` - Migrate schemas to canonical module locations
- Moved Pydantic schemas from `models/schema/` to `app/modules/{module}/schemas/`
- Established schema auto-discovery pattern
#### Commit: `b9f08b8` - Clean up legacy models and migrate remaining schemas
- Removed duplicate schema definitions
- Consolidated schema exports in module `__init__.py`
#### Commit: `0f9b80c` - Migrate Feature to billing module and split ProductMedia to catalog
- Moved `Feature` model to `app/modules/billing/models/`
- Moved `ProductMedia` to `app/modules/catalog/models/`
#### Commit: `d7de723` - Remove legacy feature.py re-export
- Removed `app/services/feature.py` re-export file
- Direct imports from module now required
### Phase 2: API Dependencies (2026-01-29)
#### Commit: `cad862f` - Introduce UserContext schema for API dependency injection
- Created `models/schema/auth.py` with `UserContext` schema
- Standardized vendor/admin API authentication pattern
- Enables consistent `token_vendor_id` access across routes
### Phase 3: Module Structure Enforcement (2026-01-29)
#### Commit: `434db15` - Add module exceptions, locales, and fix architecture warnings
- Added `exceptions.py` to all self-contained modules
- Created locale files for i18n support
- Fixed architecture validation warnings
#### Commit: `0b4291d` - Migrate JavaScript files to module directories
- Moved JS files from `static/vendor/js/` to `app/modules/{module}/static/vendor/js/`
- Module static files now auto-mounted at `/static/modules/{module}/`
### Phase 4: Customer Module (2026-01-30)
#### Commit: `e0b69f5` - Migrate customers routes to module with auto-discovery
- Created `app/modules/customers/routes/api/vendor.py`
- Moved customer management routes from legacy location
#### Commit: `0a82c84` - Remove legacy route files, fully self-contained
- Deleted `app/api/v1/vendor/customers.py`
- Customers module now fully self-contained
### Phase 5: Full Route Auto-Discovery (2026-01-31)
#### Commit: `db56b34` - Switch to full auto-discovery for module API routes
- Updated `app/modules/routes.py` with route auto-discovery
- Modules with `is_self_contained=True` have routes auto-registered
- No manual `include_router()` calls needed
#### Commit: `6f27813` - Migrate products and vendor_products to module auto-discovery
- Moved product routes to `app/modules/catalog/routes/api/`
- Moved vendor product routes to catalog module
- Deleted legacy `app/api/v1/vendor/products.py`
#### Commit: `e2cecff` - Migrate vendor billing, invoices, payments to module auto-discovery
- Created `app/modules/billing/routes/api/vendor_checkout.py`
- Created `app/modules/billing/routes/api/vendor_addons.py`
- Deleted legacy billing routes from `app/api/v1/vendor/`
### Phase 6: Remaining Vendor Routes (2026-01-31)
#### Current Changes - Migrate analytics, usage, onboarding
- **Deleted**: `app/api/v1/vendor/analytics.py` (duplicate - analytics module already auto-discovered)
- **Created**: `app/modules/billing/routes/api/vendor_usage.py` (usage limits/upgrades)
- **Created**: `app/modules/marketplace/routes/api/vendor_onboarding.py` (onboarding wizard)
- **Deleted**: `app/api/v1/vendor/usage.py` (migrated to billing)
- **Deleted**: `app/api/v1/vendor/onboarding.py` (migrated to marketplace)
## Current State
### Migrated to Modules (Auto-Discovered)
| Module | Routes | Services | Models | Schemas | Tasks |
|--------|--------|----------|--------|---------|-------|
| analytics | API | Stats | Report | Stats | - |
| billing | API | Billing, Subscription | Tier, Subscription, Invoice | Billing | Subscription |
| catalog | API | Product | Product, Category | Product | - |
| cart | API | Cart | Cart, CartItem | Cart | Cleanup |
| checkout | API | Checkout | - | Checkout | - |
| cms | API, Pages | ContentPage | ContentPage, Section | CMS | - |
| customers | API | Customer | Customer | Customer | - |
| inventory | API | Inventory | Stock, Location | Inventory | - |
| marketplace | API | Import, Export, Sync | ImportJob | Marketplace | Import, Export |
| messaging | API | Message | Message | Message | - |
| orders | API | Order | Order, OrderItem | Order | - |
| payments | API | Payment, Stripe | Payment | Payment | - |
### Still in Legacy Locations (Need Migration)
#### Vendor Routes (`app/api/v1/vendor/`)
- `auth.py` - Authentication (belongs in core/tenancy)
- `dashboard.py` - Dashboard (belongs in core)
- `email_settings.py` - Email settings (belongs in messaging)
- `email_templates.py` - Email templates (belongs in messaging)
- `info.py` - Vendor info (belongs in tenancy)
- `media.py` - Media library (belongs in cms)
- `messages.py` - Messages (belongs in messaging)
- `notifications.py` - Notifications (belongs in messaging)
- `profile.py` - Profile (belongs in core/tenancy)
- `settings.py` - Settings (belongs in core)
- `team.py` - Team management (belongs in tenancy)
#### Admin Routes (`app/api/v1/admin/`)
- Most files still in legacy location
- Target: Move to respective modules or tenancy module
#### Services (`app/services/`)
- 61 files still in legacy location
- Many are re-exports from modules
- Target: Move actual code to modules, delete re-exports
#### Tasks (`app/tasks/`)
- `letzshop_tasks.py` - Belongs in marketplace module
- `subscription_tasks.py` - Belongs in billing module
- Others need evaluation
## Architecture Rules
The following rules enforce the module-first architecture:
| Rule | Severity | Description |
|------|----------|-------------|
| MOD-016 | ERROR | Routes must be in modules, not `app/api/v1/{vendor,admin}/` |
| MOD-017 | ERROR | Services must be in modules, not `app/services/` |
| MOD-018 | ERROR | Tasks must be in modules, not `app/tasks/` |
| MOD-019 | WARNING | Schemas should be in modules, not `models/schema/` |
## Next Steps
1. **Migrate remaining vendor routes** to appropriate modules
2. **Migrate admin routes** to modules
3. **Move services** from `app/services/` to module `services/`
4. **Move tasks** from `app/tasks/` to module `tasks/`
5. **Clean up re-exports** once all code is in modules
## Verification
Run architecture validation to check compliance:
```bash
python scripts/validate_architecture.py
```
Check for legacy location violations:
```bash
python scripts/validate_architecture.py -d app/api/v1/vendor
python scripts/validate_architecture.py -d app/services
```