docs: update models-structure with hybrid architecture patterns

Document the current architecture for models and schemas:
- CORE models (models/database/) vs module models (app/modules/*/models/)
- Three schema patterns: dedicated files, inline, and no-schema
- Data flow diagram showing services access DB directly
- Gap analysis table showing model-schema alignment
- Import guidelines with canonical vs legacy re-exports
- Checklists for creating new models and schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-30 20:09:16 +01:00
parent d7de7238ff
commit 1ad30bd77e

View File

@@ -1,469 +1,327 @@
# Models Structure
# Models & Schemas Architecture
## Overview
This project follows a **standardized models structure** at the root level, separating database models from Pydantic schemas.
This project uses a **hybrid architecture** for models and schemas:
1. **CORE models/schemas** in `models/database/` and `models/schema/` - Framework-level entities used across modules
2. **Module models/schemas** in `app/modules/<module>/models/` and `app/modules/<module>/schemas/` - Domain-specific entities
3. **Inline schemas** defined directly in API route files - Endpoint-specific request/response models
## Directory Structure
```
models/
├── database/ # SQLAlchemy database models (ORM)
│ ├── __init__.py
│ ├── user.py
│ ├── vendor.py
│ ├── product.py
│ ├── order.py
│ ├── admin.py
│ ├── architecture_scan.py
models/ # CORE framework models
├── database/ # SQLAlchemy models (ORM)
│ ├── __init__.py # Exports all models + module discovery
│ ├── base.py # Base class, mixins
│ ├── user.py # User authentication
│ ├── vendor.py # Vendor, VendorUser, Role
│ ├── company.py # Company management
│ ├── platform.py # Multi-platform support
│ ├── media.py # MediaFile (cross-cutting)
│ └── ...
└── schema/ # Pydantic schemas (API validation)
└── schema/ # Pydantic schemas (API validation)
├── __init__.py
├── auth.py
├── admin.py
├── product.py
├── order.py
├── auth.py # Login, tokens, password reset
├── vendor.py # Vendor CRUD schemas
├── company.py # Company schemas
└── ...
```
## Important Rules
### ✅ DO: Use Root-Level Models
**ALL models must be in the root `models/` directory:**
- Database models → `models/database/`
- Pydantic schemas → `models/schema/`
### ❌ DON'T: Create `app/models/`
**NEVER create or use `app/models/` directory.**
The application structure is:
```
app/ # Application code (routes, services, core)
models/ # Models (database & schemas)
```
NOT:
```
app/
models/ # ❌ WRONG - Don't create this!
models/ # ✓ Correct location
app/modules/<module>/ # Domain modules
├── models/ # Module-specific database models
│ ├── __init__.py # Canonical exports
│ └── *.py # Model definitions
└── schemas/ # Module-specific schemas
├── __init__.py # Canonical exports
└── *.py # Schema definitions
```
---
## Database Models (`models/database/`)
## Architecture Principles
### Purpose
SQLAlchemy ORM models that represent database tables.
### 1. CORE vs Module Models
### Naming Convention
- Singular class names: `User`, `Product`, `Order`
- File names match class: `user.py`, `product.py`, `order.py`
**CORE models** (`models/database/`) are framework-level entities used across multiple modules:
- `User` - Authentication, used by all modules
- `Vendor` - Multi-tenant anchor, used by all vendor-scoped modules
- `Company` - Business entity management
- `Platform` - Multi-platform CMS support
- `MediaFile` - File storage, used by catalog, CMS, etc.
### Example Structure
**Module models** (`app/modules/<module>/models/`) are domain-specific:
- `billing/models/` - Feature, SubscriptionTier, VendorSubscription
- `catalog/models/` - Product, ProductTranslation, ProductMedia
- `orders/models/` - Order, OrderItem, Invoice
- `inventory/models/` - Inventory, InventoryTransaction
**File:** `models/database/product.py`
```python
"""Product database model"""
### 2. Schema Patterns
from sqlalchemy import Column, Integer, String, Float, ForeignKey
from sqlalchemy.orm import relationship
The codebase uses **three valid patterns** for schemas:
from .base import Base
class Product(Base):
"""Product database model"""
__tablename__ = "products"
id = Column(Integer, primary_key=True, index=True)
name = Column(String(255), nullable=False)
price = Column(Float, nullable=False)
vendor_id = Column(Integer, ForeignKey("vendors.id"))
# Relationships
vendor = relationship("Vendor", back_populates="products")
```
### Exporting Models
All database models must be exported in `models/database/__init__.py`:
#### Pattern A: Dedicated Schema Files (Reusable)
For schemas used across multiple endpoints or modules.
```python
# models/database/__init__.py
from .user import User
from .vendor import Vendor
from .product import Product
from .order import Order, OrderItem
# app/modules/inventory/schemas/inventory.py
class InventoryCreate(BaseModel):
product_id: int
location: str
quantity: int
__all__ = [
"User",
"Vendor",
"Product",
"Order",
"OrderItem",
]
```
### Importing Database Models
```python
# ✅ CORRECT - Import from models.database
from models.database import User, Product
from models.database.vendor import Vendor
# ❌ WRONG - Don't import from app.models
from app.models.user import User # This path doesn't exist!
```
---
## Pydantic Schemas (`models/schema/`)
### Purpose
Pydantic models for API request/response validation and serialization.
### Naming Convention
- Use descriptive suffixes: `Create`, `Update`, `Response`, `InDB`
- Group related schemas in same file
- File names match domain: `auth.py`, `product.py`, `order.py`
### Example Structure
**File:** `models/schema/product.py`
```python
"""Product Pydantic schemas"""
from typing import Optional
from pydantic import BaseModel, Field
class ProductBase(BaseModel):
"""Base product schema"""
name: str = Field(..., min_length=1, max_length=255)
description: Optional[str] = None
price: float = Field(..., gt=0)
class ProductCreate(ProductBase):
"""Schema for creating a product"""
vendor_id: int
class ProductUpdate(BaseModel):
"""Schema for updating a product"""
name: Optional[str] = Field(None, min_length=1, max_length=255)
description: Optional[str] = None
price: Optional[float] = Field(None, gt=0)
class ProductResponse(ProductBase):
"""Schema for product API response"""
class InventoryResponse(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
product_id: int
vendor_id: int
class Config:
from_attributes = True # Pydantic v2
# orm_mode = True # Pydantic v1
quantity: int
```
### Exporting Schemas
Export schemas in `models/schema/__init__.py`:
#### Pattern B: Inline Schemas (Endpoint-Specific)
For schemas only used by a single endpoint or closely related endpoints.
```python
# models/schema/__init__.py
from .auth import LoginRequest, TokenResponse
from .product import ProductCreate, ProductUpdate, ProductResponse
__all__ = [
"LoginRequest",
"TokenResponse",
"ProductCreate",
"ProductUpdate",
"ProductResponse",
]
```
### Importing Schemas
```python
# ✅ CORRECT
from models.schema import ProductCreate, ProductResponse
from models.schema.auth import LoginRequest
# ❌ WRONG
from app.models.schema.product import ProductCreate
```
---
## Common Patterns
### Pattern 1: Database Model with Schema
**Database Model:** `models/database/vendor.py`
```python
from sqlalchemy import Column, Integer, String, Boolean
from .base import Base
class Vendor(Base):
__tablename__ = "vendors"
id = Column(Integer, primary_key=True)
name = Column(String(255), nullable=False)
code = Column(String(50), unique=True, nullable=False)
is_active = Column(Boolean, default=True)
```
**Pydantic Schema:** `models/schema/vendor.py`
```python
from pydantic import BaseModel
class VendorBase(BaseModel):
name: str
# app/api/v1/admin/platforms.py
class PlatformResponse(BaseModel):
"""Platform response schema - only used in this file."""
id: int
code: str
name: str
# ...
class VendorCreate(VendorBase):
pass
class PlatformUpdateRequest(BaseModel):
"""Request schema - only used in this file."""
name: str | None = None
description: str | None = None
class VendorResponse(VendorBase):
id: int
is_active: bool
class Config:
from_attributes = True
@router.get("/platforms/{code}", response_model=PlatformResponse)
def get_platform(...):
...
```
**Usage in API:**
```python
from fastapi import APIRouter
from sqlalchemy.orm import Session
from models.database import Vendor
from models.schema import VendorCreate, VendorResponse
router = APIRouter()
@router.post("/vendors", response_model=VendorResponse)
def create_vendor(vendor_data: VendorCreate, db: Session):
# VendorCreate validates input
db_vendor = Vendor(**vendor_data.dict())
db.add(db_vendor)
db.commit()
db.refresh(db_vendor)
# VendorResponse serializes output
return db_vendor
```
---
### Pattern 2: Complex Schemas
For complex domains, organize schemas by purpose:
#### Pattern C: No Schema (Internal Service Use)
For models only accessed internally by services, not exposed via API.
```python
# models/schema/order.py
class OrderBase(BaseModel):
"""Base order fields"""
pass
# Internal association table - no API exposure
class VendorPlatform(Base):
"""Links vendors to platforms - internal use only."""
__tablename__ = "vendor_platforms"
vendor_id = Column(Integer, ForeignKey("vendors.id"))
platform_id = Column(Integer, ForeignKey("platforms.id"))
```
class OrderCreate(OrderBase):
"""Create order from customer"""
items: List[OrderItemCreate]
### 3. When to Use Each Pattern
class OrderUpdate(BaseModel):
"""Admin order update"""
status: Optional[OrderStatus]
| Scenario | Pattern | Location |
|----------|---------|----------|
| Schema used by multiple endpoints | Dedicated file | `app/modules/<module>/schemas/` |
| Schema used by single endpoint | Inline | In the route file |
| Admin-only endpoint schemas | Inline | In the admin route file |
| Model not exposed via API | No schema | N/A |
| Cross-module utility schemas | Dedicated file | `models/schema/` |
class OrderResponse(OrderBase):
"""Order API response"""
id: int
items: List[OrderItemResponse]
---
class OrderAdminResponse(OrderResponse):
"""Extended response for admin"""
internal_notes: Optional[str]
## Data Flow Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ API Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Request Schema │───▶│ Endpoint │───▶│ Response Schema │ │
│ │ (Pydantic) │ │ │ │ (Pydantic) │ │
│ └─────────────────┘ └────────┬────────┘ └─────────────────┘ │
└──────────────────────────────────┼──────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Service Layer │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Business Logic │ │
│ │ • Validation • Transformations • Business Rules │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Database Models (SQLAlchemy) │ │
│ │ • Direct access • Queries • Transactions │ │
│ └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Database Layer │
│ PostgreSQL / SQLite │
└─────────────────────────────────────────────────────────────────────┘
```
**Key principles:**
- **Services access database models directly** - No schema layer between service and database
- **APIs use schemas for validation** - Request/response validation at API boundary
- **Schemas are API contracts** - Define what clients send/receive
---
## Import Guidelines
### Canonical Imports (Preferred)
```python
# CORE models - from models.database
from models.database import User, Vendor, MediaFile
# Module models - from app.modules.<module>.models
from app.modules.billing.models import Feature, SubscriptionTier
from app.modules.catalog.models import Product, ProductMedia
from app.modules.orders.models import Order, OrderItem
# CORE schemas - from models.schema
from models.schema.auth import LoginRequest, TokenResponse
# Module schemas - from app.modules.<module>.schemas
from app.modules.inventory.schemas import InventoryCreate, InventoryResponse
from app.modules.orders.schemas import OrderResponse
```
### Legacy Re-exports (Backwards Compatibility)
`models/database/__init__.py` re-exports module models for backwards compatibility:
```python
# These work but prefer canonical imports
from models.database import Product # Re-exported from catalog module
from models.database import Order # Re-exported from orders module
```
---
## Migration Guide
## Gap Analysis: Models vs Schemas
If you accidentally created models in the wrong location:
Not every database model needs a dedicated schema file. Here's the current alignment:
### Moving Database Models
### CORE Framework
```bash
# If you created app/models/my_model.py (WRONG)
# Move to correct location:
mv app/models/my_model.py models/database/my_model.py
| Database Model | Schema | Notes |
|----------------|--------|-------|
| `user.py` | `auth.py` | Auth schemas cover user operations |
| `vendor.py` | `vendor.py` | Full CRUD schemas |
| `company.py` | `company.py` | Full CRUD schemas |
| `media.py` | `media.py` | Upload/response schemas |
| `platform.py` | Inline | Admin-only, in `platforms.py` route |
| `platform_module.py` | Inline | Admin-only, in `modules.py` route |
| `admin_menu_config.py` | Inline | Admin-only, in `menu_config.py` route |
| `vendor_email_settings.py` | Inline | In `email_settings.py` route |
| `vendor_email_template.py` | None | Internal email service use |
| `vendor_platform.py` | None | Internal association table |
# Update imports in all files
# FROM: from app.models.my_model import MyModel
# TO: from models.database.my_model import MyModel
### Modules
# Add to models/database/__init__.py
# Remove app/models/ directory
rm -rf app/models/
```
### Moving Pydantic Schemas
```bash
# If you created app/schemas/my_schema.py (WRONG)
# Move to correct location:
mv app/schemas/my_schema.py models/schema/my_schema.py
# Update imports
# FROM: from app.schemas.my_schema import MySchema
# TO: from models.schema.my_schema import MySchema
# Add to models/schema/__init__.py
# Remove app/schemas/ directory
rm -rf app/schemas/
```
| Module | Models | Schemas | Alignment |
|--------|--------|---------|-----------|
| billing | feature, subscription | billing, subscription | ✅ |
| cart | cart | cart | ✅ |
| catalog | product, product_media, product_translation | catalog, product, vendor_product | ✅ |
| checkout | - | checkout | Schema-only (orchestration) |
| cms | content_page | content_page, homepage_sections | ✅ |
| customers | customer, password_reset_token | customer, context | ✅ (password reset uses auth schemas) |
| dev_tools | architecture_scan, test_run | Inline | Admin-only, inline in route files |
| inventory | inventory, inventory_transaction | inventory | ✅ (transaction included in inventory.py) |
| loyalty | 5 models | 5 schemas | ✅ |
| marketplace | 5 models | 4 schemas | ✅ (translation in product schema) |
| messaging | admin_notification, message | message, notification | ✅ |
| orders | order, order_item, invoice | order, invoice, order_item_exception | ✅ |
| payments | - | payment | Schema-only (external APIs) |
| analytics | - | stats | Schema-only (aggregation views) |
---
## Why This Structure?
### ✅ Benefits
1. **Clear Separation**
- Database layer separate from application layer
- Easy to understand where models live
2. **Import Consistency**
- `from models.database import ...`
- `from models.schema import ...`
- No confusion about import paths
3. **Testing**
- Easy to mock database models
- Easy to test schema validation
4. **Scalability**
- Models can be used by multiple apps
- Clean separation of concerns
5. **Tool Compatibility**
- Alembic migrations find models easily
- IDE autocomplete works better
- Linters understand structure
### ❌ Problems with `app/models/`
1. **Confusion**: Is it database or schema?
2. **Import Issues**: Circular dependencies
3. **Migration Problems**: Alembic can't find models
4. **Inconsistency**: Different parts of codebase use different paths
---
## Verification Checklist
Use this checklist when adding new models:
## Creating New Models
### Database Model Checklist
- [ ] File in `models/database/{name}.py`
- [ ] Inherits from `Base`
- [ ] Has `__tablename__` defined
- [ ] Exported in `models/database/__init__.py`
- [ ] Imported using `from models.database import ...`
- [ ] NO file in `app/models/`
### Pydantic Schema Checklist
- [ ] File in `models/schema/{name}.py`
- [ ] Inherits from `BaseModel`
- [ ] Has descriptive suffix (`Create`, `Update`, `Response`)
- [ ] Exported in `models/schema/__init__.py`
- [ ] Imported using `from models.schema import ...`
- [ ] NO file in `app/schemas/`
1. **Determine location:**
- Cross-module use → `models/database/`
- Module-specific → `app/modules/<module>/models/`
---
2. **Create the model file:**
```python
# app/modules/mymodule/models/my_entity.py
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from app.core.database import Base
from models.database.base import TimestampMixin
## Project Structure
class MyEntity(Base, TimestampMixin):
__tablename__ = "my_entities"
```
project/
├── app/
│ ├── api/ # API routes
│ ├── core/ # Core functionality (config, database, auth)
│ ├── services/ # Business logic
│ ├── templates/ # Jinja2 templates
│ └── routes/ # Page routes
├── models/ # ✓ Models live here!
│ ├── database/ # ✓ SQLAlchemy models
│ └── schema/ # ✓ Pydantic schemas
├── static/ # Frontend assets
├── docs/ # Documentation
├── tests/ # Tests
└── scripts/ # Utility scripts
```
id = Column(Integer, primary_key=True, index=True)
vendor_id = Column(Integer, ForeignKey("vendors.id"), nullable=False)
name = Column(String(255), nullable=False)
**NOT:**
```
app/
models/ # ❌ Don't create this
schemas/ # ❌ Don't create this
```
vendor = relationship("Vendor")
```
---
3. **Export in `__init__.py`:**
```python
# app/modules/mymodule/models/__init__.py
from app.modules.mymodule.models.my_entity import MyEntity
## Examples from the Codebase
__all__ = ["MyEntity"]
```
### ✅ Correct Examples
### Schema Checklist
**Database Model:**
```python
# models/database/architecture_scan.py
from sqlalchemy import Column, Integer, String
from .base import Base
1. **Decide if schema is needed:**
- Exposed via API? → Yes, create schema
- Multiple endpoints use it? → Dedicated schema file
- Single endpoint? → Consider inline schema
- Internal only? → No schema needed
class ArchitectureScan(Base):
__tablename__ = "architecture_scans"
id = Column(Integer, primary_key=True)
```
2. **Create schema (if needed):**
```python
# app/modules/mymodule/schemas/my_entity.py
from pydantic import BaseModel, ConfigDict, Field
**Import in Service:**
```python
# app/services/code_quality_service.py
from app.modules.dev_tools.models import ArchitectureScan
```
class MyEntityCreate(BaseModel):
name: str = Field(..., min_length=1, max_length=255)
**Pydantic Schema:**
```python
# models/schema/admin.py
from pydantic import BaseModel
class MyEntityResponse(BaseModel):
model_config = ConfigDict(from_attributes=True)
class AdminDashboardStats(BaseModel):
total_vendors: int
total_users: int
```
id: int
vendor_id: int
name: str
```
**Import in API:**
```python
# app/api/v1/admin/dashboard.py
from models.schema.admin import AdminDashboardStats
```
3. **Or use inline schema:**
```python
# app/api/v1/vendor/my_entity.py
from pydantic import BaseModel
class MyEntityResponse(BaseModel):
"""Response schema - only used in this file."""
id: int
name: str
@router.get("/my-entities/{id}", response_model=MyEntityResponse)
def get_entity(...):
...
```
---
## Summary
**Golden Rule:** All models in `models/`, never in `app/models/` or `app/schemas/`.
**Quick Reference:**
- Database models → `models/database/`
- Pydantic schemas → `models/schema/`
- Import pattern → `from models.{type} import ...`
- No models in `app/` directory
This standard ensures consistency, clarity, and maintainability across the entire project.
| Principle | Implementation |
|-----------|----------------|
| Services access DB directly | `from app.modules.X.models import Model` |
| APIs validate with schemas | Request/response Pydantic models |
| Reusable schemas | Dedicated files in `schemas/` |
| Endpoint-specific schemas | Inline in route files |
| Internal models | No schema needed |
| CORE models | `models/database/` |
| Module models | `app/modules/<module>/models/` |