diff --git a/docs/architecture/models-structure.md b/docs/architecture/models-structure.md index d1e0056d..7c2e19ee 100644 --- a/docs/architecture/models-structure.md +++ b/docs/architecture/models-structure.md @@ -1,469 +1,327 @@ -# Models Structure +# Models & Schemas Architecture ## Overview -This project follows a **standardized models structure** at the root level, separating database models from Pydantic schemas. +This project uses a **hybrid architecture** for models and schemas: + +1. **CORE models/schemas** in `models/database/` and `models/schema/` - Framework-level entities used across modules +2. **Module models/schemas** in `app/modules//models/` and `app/modules//schemas/` - Domain-specific entities +3. **Inline schemas** defined directly in API route files - Endpoint-specific request/response models ## Directory Structure ``` -models/ -├── database/ # SQLAlchemy database models (ORM) -│ ├── __init__.py -│ ├── user.py -│ ├── vendor.py -│ ├── product.py -│ ├── order.py -│ ├── admin.py -│ ├── architecture_scan.py +models/ # CORE framework models +├── database/ # SQLAlchemy models (ORM) +│ ├── __init__.py # Exports all models + module discovery +│ ├── base.py # Base class, mixins +│ ├── user.py # User authentication +│ ├── vendor.py # Vendor, VendorUser, Role +│ ├── company.py # Company management +│ ├── platform.py # Multi-platform support +│ ├── media.py # MediaFile (cross-cutting) │ └── ... │ -└── schema/ # Pydantic schemas (API validation) +└── schema/ # Pydantic schemas (API validation) ├── __init__.py - ├── auth.py - ├── admin.py - ├── product.py - ├── order.py + ├── auth.py # Login, tokens, password reset + ├── vendor.py # Vendor CRUD schemas + ├── company.py # Company schemas └── ... -``` -## Important Rules - -### ✅ DO: Use Root-Level Models - -**ALL models must be in the root `models/` directory:** -- Database models → `models/database/` -- Pydantic schemas → `models/schema/` - -### ❌ DON'T: Create `app/models/` - -**NEVER create or use `app/models/` directory.** - -The application structure is: -``` -app/ # Application code (routes, services, core) -models/ # Models (database & schemas) -``` - -NOT: -``` -app/ - models/ # ❌ WRONG - Don't create this! -models/ # ✓ Correct location +app/modules// # Domain modules +├── models/ # Module-specific database models +│ ├── __init__.py # Canonical exports +│ └── *.py # Model definitions +│ +└── schemas/ # Module-specific schemas + ├── __init__.py # Canonical exports + └── *.py # Schema definitions ``` --- -## Database Models (`models/database/`) +## Architecture Principles -### Purpose -SQLAlchemy ORM models that represent database tables. +### 1. CORE vs Module Models -### Naming Convention -- Singular class names: `User`, `Product`, `Order` -- File names match class: `user.py`, `product.py`, `order.py` +**CORE models** (`models/database/`) are framework-level entities used across multiple modules: +- `User` - Authentication, used by all modules +- `Vendor` - Multi-tenant anchor, used by all vendor-scoped modules +- `Company` - Business entity management +- `Platform` - Multi-platform CMS support +- `MediaFile` - File storage, used by catalog, CMS, etc. -### Example Structure +**Module models** (`app/modules//models/`) are domain-specific: +- `billing/models/` - Feature, SubscriptionTier, VendorSubscription +- `catalog/models/` - Product, ProductTranslation, ProductMedia +- `orders/models/` - Order, OrderItem, Invoice +- `inventory/models/` - Inventory, InventoryTransaction -**File:** `models/database/product.py` -```python -"""Product database model""" +### 2. Schema Patterns -from sqlalchemy import Column, Integer, String, Float, ForeignKey -from sqlalchemy.orm import relationship +The codebase uses **three valid patterns** for schemas: -from .base import Base - - -class Product(Base): - """Product database model""" - - __tablename__ = "products" - - id = Column(Integer, primary_key=True, index=True) - name = Column(String(255), nullable=False) - price = Column(Float, nullable=False) - vendor_id = Column(Integer, ForeignKey("vendors.id")) - - # Relationships - vendor = relationship("Vendor", back_populates="products") -``` - -### Exporting Models - -All database models must be exported in `models/database/__init__.py`: +#### Pattern A: Dedicated Schema Files (Reusable) +For schemas used across multiple endpoints or modules. ```python -# models/database/__init__.py -from .user import User -from .vendor import Vendor -from .product import Product -from .order import Order, OrderItem +# app/modules/inventory/schemas/inventory.py +class InventoryCreate(BaseModel): + product_id: int + location: str + quantity: int -__all__ = [ - "User", - "Vendor", - "Product", - "Order", - "OrderItem", -] -``` - -### Importing Database Models - -```python -# ✅ CORRECT - Import from models.database -from models.database import User, Product -from models.database.vendor import Vendor - -# ❌ WRONG - Don't import from app.models -from app.models.user import User # This path doesn't exist! -``` - ---- - -## Pydantic Schemas (`models/schema/`) - -### Purpose -Pydantic models for API request/response validation and serialization. - -### Naming Convention -- Use descriptive suffixes: `Create`, `Update`, `Response`, `InDB` -- Group related schemas in same file -- File names match domain: `auth.py`, `product.py`, `order.py` - -### Example Structure - -**File:** `models/schema/product.py` -```python -"""Product Pydantic schemas""" - -from typing import Optional -from pydantic import BaseModel, Field - - -class ProductBase(BaseModel): - """Base product schema""" - name: str = Field(..., min_length=1, max_length=255) - description: Optional[str] = None - price: float = Field(..., gt=0) - - -class ProductCreate(ProductBase): - """Schema for creating a product""" - vendor_id: int - - -class ProductUpdate(BaseModel): - """Schema for updating a product""" - name: Optional[str] = Field(None, min_length=1, max_length=255) - description: Optional[str] = None - price: Optional[float] = Field(None, gt=0) - - -class ProductResponse(ProductBase): - """Schema for product API response""" +class InventoryResponse(BaseModel): + model_config = ConfigDict(from_attributes=True) id: int + product_id: int vendor_id: int - - class Config: - from_attributes = True # Pydantic v2 - # orm_mode = True # Pydantic v1 + quantity: int ``` -### Exporting Schemas - -Export schemas in `models/schema/__init__.py`: +#### Pattern B: Inline Schemas (Endpoint-Specific) +For schemas only used by a single endpoint or closely related endpoints. ```python -# models/schema/__init__.py -from .auth import LoginRequest, TokenResponse -from .product import ProductCreate, ProductUpdate, ProductResponse - -__all__ = [ - "LoginRequest", - "TokenResponse", - "ProductCreate", - "ProductUpdate", - "ProductResponse", -] -``` - -### Importing Schemas - -```python -# ✅ CORRECT -from models.schema import ProductCreate, ProductResponse -from models.schema.auth import LoginRequest - -# ❌ WRONG -from app.models.schema.product import ProductCreate -``` - ---- - -## Common Patterns - -### Pattern 1: Database Model with Schema - -**Database Model:** `models/database/vendor.py` -```python -from sqlalchemy import Column, Integer, String, Boolean -from .base import Base - -class Vendor(Base): - __tablename__ = "vendors" - - id = Column(Integer, primary_key=True) - name = Column(String(255), nullable=False) - code = Column(String(50), unique=True, nullable=False) - is_active = Column(Boolean, default=True) -``` - -**Pydantic Schema:** `models/schema/vendor.py` -```python -from pydantic import BaseModel - -class VendorBase(BaseModel): - name: str +# app/api/v1/admin/platforms.py +class PlatformResponse(BaseModel): + """Platform response schema - only used in this file.""" + id: int code: str + name: str + # ... -class VendorCreate(VendorBase): - pass +class PlatformUpdateRequest(BaseModel): + """Request schema - only used in this file.""" + name: str | None = None + description: str | None = None -class VendorResponse(VendorBase): - id: int - is_active: bool - - class Config: - from_attributes = True +@router.get("/platforms/{code}", response_model=PlatformResponse) +def get_platform(...): + ... ``` -**Usage in API:** -```python -from fastapi import APIRouter -from sqlalchemy.orm import Session - -from models.database import Vendor -from models.schema import VendorCreate, VendorResponse - -router = APIRouter() - -@router.post("/vendors", response_model=VendorResponse) -def create_vendor(vendor_data: VendorCreate, db: Session): - # VendorCreate validates input - db_vendor = Vendor(**vendor_data.dict()) - db.add(db_vendor) - db.commit() - db.refresh(db_vendor) - # VendorResponse serializes output - return db_vendor -``` - ---- - -### Pattern 2: Complex Schemas - -For complex domains, organize schemas by purpose: +#### Pattern C: No Schema (Internal Service Use) +For models only accessed internally by services, not exposed via API. ```python -# models/schema/order.py -class OrderBase(BaseModel): - """Base order fields""" - pass +# Internal association table - no API exposure +class VendorPlatform(Base): + """Links vendors to platforms - internal use only.""" + __tablename__ = "vendor_platforms" + vendor_id = Column(Integer, ForeignKey("vendors.id")) + platform_id = Column(Integer, ForeignKey("platforms.id")) +``` -class OrderCreate(OrderBase): - """Create order from customer""" - items: List[OrderItemCreate] +### 3. When to Use Each Pattern -class OrderUpdate(BaseModel): - """Admin order update""" - status: Optional[OrderStatus] +| Scenario | Pattern | Location | +|----------|---------|----------| +| Schema used by multiple endpoints | Dedicated file | `app/modules//schemas/` | +| Schema used by single endpoint | Inline | In the route file | +| Admin-only endpoint schemas | Inline | In the admin route file | +| Model not exposed via API | No schema | N/A | +| Cross-module utility schemas | Dedicated file | `models/schema/` | -class OrderResponse(OrderBase): - """Order API response""" - id: int - items: List[OrderItemResponse] +--- -class OrderAdminResponse(OrderResponse): - """Extended response for admin""" - internal_notes: Optional[str] +## Data Flow Architecture + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ API Layer │ +│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ +│ │ Request Schema │───▶│ Endpoint │───▶│ Response Schema │ │ +│ │ (Pydantic) │ │ │ │ (Pydantic) │ │ +│ └─────────────────┘ └────────┬────────┘ └─────────────────┘ │ +└──────────────────────────────────┼──────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Service Layer │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ Business Logic │ │ +│ │ • Validation • Transformations • Business Rules │ │ +│ └─────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────┐ │ +│ │ Database Models (SQLAlchemy) │ │ +│ │ • Direct access • Queries • Transactions │ │ +│ └─────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Database Layer │ +│ PostgreSQL / SQLite │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +**Key principles:** +- **Services access database models directly** - No schema layer between service and database +- **APIs use schemas for validation** - Request/response validation at API boundary +- **Schemas are API contracts** - Define what clients send/receive + +--- + +## Import Guidelines + +### Canonical Imports (Preferred) + +```python +# CORE models - from models.database +from models.database import User, Vendor, MediaFile + +# Module models - from app.modules..models +from app.modules.billing.models import Feature, SubscriptionTier +from app.modules.catalog.models import Product, ProductMedia +from app.modules.orders.models import Order, OrderItem + +# CORE schemas - from models.schema +from models.schema.auth import LoginRequest, TokenResponse + +# Module schemas - from app.modules..schemas +from app.modules.inventory.schemas import InventoryCreate, InventoryResponse +from app.modules.orders.schemas import OrderResponse +``` + +### Legacy Re-exports (Backwards Compatibility) + +`models/database/__init__.py` re-exports module models for backwards compatibility: + +```python +# These work but prefer canonical imports +from models.database import Product # Re-exported from catalog module +from models.database import Order # Re-exported from orders module ``` --- -## Migration Guide +## Gap Analysis: Models vs Schemas -If you accidentally created models in the wrong location: +Not every database model needs a dedicated schema file. Here's the current alignment: -### Moving Database Models +### CORE Framework -```bash -# If you created app/models/my_model.py (WRONG) -# Move to correct location: -mv app/models/my_model.py models/database/my_model.py +| Database Model | Schema | Notes | +|----------------|--------|-------| +| `user.py` | `auth.py` | Auth schemas cover user operations | +| `vendor.py` | `vendor.py` | Full CRUD schemas | +| `company.py` | `company.py` | Full CRUD schemas | +| `media.py` | `media.py` | Upload/response schemas | +| `platform.py` | Inline | Admin-only, in `platforms.py` route | +| `platform_module.py` | Inline | Admin-only, in `modules.py` route | +| `admin_menu_config.py` | Inline | Admin-only, in `menu_config.py` route | +| `vendor_email_settings.py` | Inline | In `email_settings.py` route | +| `vendor_email_template.py` | None | Internal email service use | +| `vendor_platform.py` | None | Internal association table | -# Update imports in all files -# FROM: from app.models.my_model import MyModel -# TO: from models.database.my_model import MyModel +### Modules -# Add to models/database/__init__.py -# Remove app/models/ directory -rm -rf app/models/ -``` - -### Moving Pydantic Schemas - -```bash -# If you created app/schemas/my_schema.py (WRONG) -# Move to correct location: -mv app/schemas/my_schema.py models/schema/my_schema.py - -# Update imports -# FROM: from app.schemas.my_schema import MySchema -# TO: from models.schema.my_schema import MySchema - -# Add to models/schema/__init__.py -# Remove app/schemas/ directory -rm -rf app/schemas/ -``` +| Module | Models | Schemas | Alignment | +|--------|--------|---------|-----------| +| billing | feature, subscription | billing, subscription | ✅ | +| cart | cart | cart | ✅ | +| catalog | product, product_media, product_translation | catalog, product, vendor_product | ✅ | +| checkout | - | checkout | Schema-only (orchestration) | +| cms | content_page | content_page, homepage_sections | ✅ | +| customers | customer, password_reset_token | customer, context | ✅ (password reset uses auth schemas) | +| dev_tools | architecture_scan, test_run | Inline | Admin-only, inline in route files | +| inventory | inventory, inventory_transaction | inventory | ✅ (transaction included in inventory.py) | +| loyalty | 5 models | 5 schemas | ✅ | +| marketplace | 5 models | 4 schemas | ✅ (translation in product schema) | +| messaging | admin_notification, message | message, notification | ✅ | +| orders | order, order_item, invoice | order, invoice, order_item_exception | ✅ | +| payments | - | payment | Schema-only (external APIs) | +| analytics | - | stats | Schema-only (aggregation views) | --- -## Why This Structure? - -### ✅ Benefits - -1. **Clear Separation** - - Database layer separate from application layer - - Easy to understand where models live - -2. **Import Consistency** - - `from models.database import ...` - - `from models.schema import ...` - - No confusion about import paths - -3. **Testing** - - Easy to mock database models - - Easy to test schema validation - -4. **Scalability** - - Models can be used by multiple apps - - Clean separation of concerns - -5. **Tool Compatibility** - - Alembic migrations find models easily - - IDE autocomplete works better - - Linters understand structure - -### ❌ Problems with `app/models/` - -1. **Confusion**: Is it database or schema? -2. **Import Issues**: Circular dependencies -3. **Migration Problems**: Alembic can't find models -4. **Inconsistency**: Different parts of codebase use different paths - ---- - -## Verification Checklist - -Use this checklist when adding new models: +## Creating New Models ### Database Model Checklist -- [ ] File in `models/database/{name}.py` -- [ ] Inherits from `Base` -- [ ] Has `__tablename__` defined -- [ ] Exported in `models/database/__init__.py` -- [ ] Imported using `from models.database import ...` -- [ ] NO file in `app/models/` -### Pydantic Schema Checklist -- [ ] File in `models/schema/{name}.py` -- [ ] Inherits from `BaseModel` -- [ ] Has descriptive suffix (`Create`, `Update`, `Response`) -- [ ] Exported in `models/schema/__init__.py` -- [ ] Imported using `from models.schema import ...` -- [ ] NO file in `app/schemas/` +1. **Determine location:** + - Cross-module use → `models/database/` + - Module-specific → `app/modules//models/` ---- +2. **Create the model file:** + ```python + # app/modules/mymodule/models/my_entity.py + from sqlalchemy import Column, Integer, String, ForeignKey + from sqlalchemy.orm import relationship + from app.core.database import Base + from models.database.base import TimestampMixin -## Project Structure + class MyEntity(Base, TimestampMixin): + __tablename__ = "my_entities" -``` -project/ -├── app/ -│ ├── api/ # API routes -│ ├── core/ # Core functionality (config, database, auth) -│ ├── services/ # Business logic -│ ├── templates/ # Jinja2 templates -│ └── routes/ # Page routes -│ -├── models/ # ✓ Models live here! -│ ├── database/ # ✓ SQLAlchemy models -│ └── schema/ # ✓ Pydantic schemas -│ -├── static/ # Frontend assets -├── docs/ # Documentation -├── tests/ # Tests -└── scripts/ # Utility scripts -``` + id = Column(Integer, primary_key=True, index=True) + vendor_id = Column(Integer, ForeignKey("vendors.id"), nullable=False) + name = Column(String(255), nullable=False) -**NOT:** -``` -app/ - models/ # ❌ Don't create this - schemas/ # ❌ Don't create this -``` + vendor = relationship("Vendor") + ``` ---- +3. **Export in `__init__.py`:** + ```python + # app/modules/mymodule/models/__init__.py + from app.modules.mymodule.models.my_entity import MyEntity -## Examples from the Codebase + __all__ = ["MyEntity"] + ``` -### ✅ Correct Examples +### Schema Checklist -**Database Model:** -```python -# models/database/architecture_scan.py -from sqlalchemy import Column, Integer, String -from .base import Base +1. **Decide if schema is needed:** + - Exposed via API? → Yes, create schema + - Multiple endpoints use it? → Dedicated schema file + - Single endpoint? → Consider inline schema + - Internal only? → No schema needed -class ArchitectureScan(Base): - __tablename__ = "architecture_scans" - id = Column(Integer, primary_key=True) -``` +2. **Create schema (if needed):** + ```python + # app/modules/mymodule/schemas/my_entity.py + from pydantic import BaseModel, ConfigDict, Field -**Import in Service:** -```python -# app/services/code_quality_service.py -from app.modules.dev_tools.models import ArchitectureScan -``` + class MyEntityCreate(BaseModel): + name: str = Field(..., min_length=1, max_length=255) -**Pydantic Schema:** -```python -# models/schema/admin.py -from pydantic import BaseModel + class MyEntityResponse(BaseModel): + model_config = ConfigDict(from_attributes=True) -class AdminDashboardStats(BaseModel): - total_vendors: int - total_users: int -``` + id: int + vendor_id: int + name: str + ``` -**Import in API:** -```python -# app/api/v1/admin/dashboard.py -from models.schema.admin import AdminDashboardStats -``` +3. **Or use inline schema:** + ```python + # app/api/v1/vendor/my_entity.py + from pydantic import BaseModel + + class MyEntityResponse(BaseModel): + """Response schema - only used in this file.""" + id: int + name: str + + @router.get("/my-entities/{id}", response_model=MyEntityResponse) + def get_entity(...): + ... + ``` --- ## Summary -**Golden Rule:** All models in `models/`, never in `app/models/` or `app/schemas/`. - -**Quick Reference:** -- Database models → `models/database/` -- Pydantic schemas → `models/schema/` -- Import pattern → `from models.{type} import ...` -- No models in `app/` directory - -This standard ensures consistency, clarity, and maintainability across the entire project. +| Principle | Implementation | +|-----------|----------------| +| Services access DB directly | `from app.modules.X.models import Model` | +| APIs validate with schemas | Request/response Pydantic models | +| Reusable schemas | Dedicated files in `schemas/` | +| Endpoint-specific schemas | Inline in route files | +| Internal models | No schema needed | +| CORE models | `models/database/` | +| Module models | `app/modules//models/` |