docs: update models-structure with hybrid architecture patterns

Document the current architecture for models and schemas:
- CORE models (models/database/) vs module models (app/modules/*/models/)
- Three schema patterns: dedicated files, inline, and no-schema
- Data flow diagram showing services access DB directly
- Gap analysis table showing model-schema alignment
- Import guidelines with canonical vs legacy re-exports
- Checklists for creating new models and schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-30 20:09:16 +01:00
parent d7de7238ff
commit 1ad30bd77e

View File

@@ -1,469 +1,327 @@
# Models Structure # Models & Schemas Architecture
## Overview ## Overview
This project follows a **standardized models structure** at the root level, separating database models from Pydantic schemas. This project uses a **hybrid architecture** for models and schemas:
1. **CORE models/schemas** in `models/database/` and `models/schema/` - Framework-level entities used across modules
2. **Module models/schemas** in `app/modules/<module>/models/` and `app/modules/<module>/schemas/` - Domain-specific entities
3. **Inline schemas** defined directly in API route files - Endpoint-specific request/response models
## Directory Structure ## Directory Structure
``` ```
models/ models/ # CORE framework models
├── database/ # SQLAlchemy database models (ORM) ├── database/ # SQLAlchemy models (ORM)
│ ├── __init__.py │ ├── __init__.py # Exports all models + module discovery
│ ├── user.py │ ├── base.py # Base class, mixins
│ ├── vendor.py │ ├── user.py # User authentication
│ ├── product.py │ ├── vendor.py # Vendor, VendorUser, Role
│ ├── order.py │ ├── company.py # Company management
│ ├── admin.py │ ├── platform.py # Multi-platform support
│ ├── architecture_scan.py │ ├── media.py # MediaFile (cross-cutting)
│ └── ... │ └── ...
└── schema/ # Pydantic schemas (API validation) └── schema/ # Pydantic schemas (API validation)
├── __init__.py ├── __init__.py
├── auth.py ├── auth.py # Login, tokens, password reset
├── admin.py ├── vendor.py # Vendor CRUD schemas
├── product.py ├── company.py # Company schemas
├── order.py
└── ... └── ...
```
## Important Rules app/modules/<module>/ # Domain modules
├── models/ # Module-specific database models
### ✅ DO: Use Root-Level Models │ ├── __init__.py # Canonical exports
│ └── *.py # Model definitions
**ALL models must be in the root `models/` directory:**
- Database models → `models/database/` └── schemas/ # Module-specific schemas
- Pydantic schemas → `models/schema/` ├── __init__.py # Canonical exports
└── *.py # Schema definitions
### ❌ DON'T: Create `app/models/`
**NEVER create or use `app/models/` directory.**
The application structure is:
```
app/ # Application code (routes, services, core)
models/ # Models (database & schemas)
```
NOT:
```
app/
models/ # ❌ WRONG - Don't create this!
models/ # ✓ Correct location
``` ```
--- ---
## Database Models (`models/database/`) ## Architecture Principles
### Purpose ### 1. CORE vs Module Models
SQLAlchemy ORM models that represent database tables.
### Naming Convention **CORE models** (`models/database/`) are framework-level entities used across multiple modules:
- Singular class names: `User`, `Product`, `Order` - `User` - Authentication, used by all modules
- File names match class: `user.py`, `product.py`, `order.py` - `Vendor` - Multi-tenant anchor, used by all vendor-scoped modules
- `Company` - Business entity management
- `Platform` - Multi-platform CMS support
- `MediaFile` - File storage, used by catalog, CMS, etc.
### Example Structure **Module models** (`app/modules/<module>/models/`) are domain-specific:
- `billing/models/` - Feature, SubscriptionTier, VendorSubscription
- `catalog/models/` - Product, ProductTranslation, ProductMedia
- `orders/models/` - Order, OrderItem, Invoice
- `inventory/models/` - Inventory, InventoryTransaction
**File:** `models/database/product.py` ### 2. Schema Patterns
```python
"""Product database model"""
from sqlalchemy import Column, Integer, String, Float, ForeignKey The codebase uses **three valid patterns** for schemas:
from sqlalchemy.orm import relationship
from .base import Base #### Pattern A: Dedicated Schema Files (Reusable)
For schemas used across multiple endpoints or modules.
class Product(Base):
"""Product database model"""
__tablename__ = "products"
id = Column(Integer, primary_key=True, index=True)
name = Column(String(255), nullable=False)
price = Column(Float, nullable=False)
vendor_id = Column(Integer, ForeignKey("vendors.id"))
# Relationships
vendor = relationship("Vendor", back_populates="products")
```
### Exporting Models
All database models must be exported in `models/database/__init__.py`:
```python ```python
# models/database/__init__.py # app/modules/inventory/schemas/inventory.py
from .user import User class InventoryCreate(BaseModel):
from .vendor import Vendor product_id: int
from .product import Product location: str
from .order import Order, OrderItem quantity: int
__all__ = [ class InventoryResponse(BaseModel):
"User", model_config = ConfigDict(from_attributes=True)
"Vendor",
"Product",
"Order",
"OrderItem",
]
```
### Importing Database Models
```python
# ✅ CORRECT - Import from models.database
from models.database import User, Product
from models.database.vendor import Vendor
# ❌ WRONG - Don't import from app.models
from app.models.user import User # This path doesn't exist!
```
---
## Pydantic Schemas (`models/schema/`)
### Purpose
Pydantic models for API request/response validation and serialization.
### Naming Convention
- Use descriptive suffixes: `Create`, `Update`, `Response`, `InDB`
- Group related schemas in same file
- File names match domain: `auth.py`, `product.py`, `order.py`
### Example Structure
**File:** `models/schema/product.py`
```python
"""Product Pydantic schemas"""
from typing import Optional
from pydantic import BaseModel, Field
class ProductBase(BaseModel):
"""Base product schema"""
name: str = Field(..., min_length=1, max_length=255)
description: Optional[str] = None
price: float = Field(..., gt=0)
class ProductCreate(ProductBase):
"""Schema for creating a product"""
vendor_id: int
class ProductUpdate(BaseModel):
"""Schema for updating a product"""
name: Optional[str] = Field(None, min_length=1, max_length=255)
description: Optional[str] = None
price: Optional[float] = Field(None, gt=0)
class ProductResponse(ProductBase):
"""Schema for product API response"""
id: int id: int
product_id: int
vendor_id: int vendor_id: int
quantity: int
class Config:
from_attributes = True # Pydantic v2
# orm_mode = True # Pydantic v1
``` ```
### Exporting Schemas #### Pattern B: Inline Schemas (Endpoint-Specific)
For schemas only used by a single endpoint or closely related endpoints.
Export schemas in `models/schema/__init__.py`:
```python ```python
# models/schema/__init__.py # app/api/v1/admin/platforms.py
from .auth import LoginRequest, TokenResponse class PlatformResponse(BaseModel):
from .product import ProductCreate, ProductUpdate, ProductResponse """Platform response schema - only used in this file."""
id: int
__all__ = [
"LoginRequest",
"TokenResponse",
"ProductCreate",
"ProductUpdate",
"ProductResponse",
]
```
### Importing Schemas
```python
# ✅ CORRECT
from models.schema import ProductCreate, ProductResponse
from models.schema.auth import LoginRequest
# ❌ WRONG
from app.models.schema.product import ProductCreate
```
---
## Common Patterns
### Pattern 1: Database Model with Schema
**Database Model:** `models/database/vendor.py`
```python
from sqlalchemy import Column, Integer, String, Boolean
from .base import Base
class Vendor(Base):
__tablename__ = "vendors"
id = Column(Integer, primary_key=True)
name = Column(String(255), nullable=False)
code = Column(String(50), unique=True, nullable=False)
is_active = Column(Boolean, default=True)
```
**Pydantic Schema:** `models/schema/vendor.py`
```python
from pydantic import BaseModel
class VendorBase(BaseModel):
name: str
code: str code: str
name: str
# ...
class VendorCreate(VendorBase): class PlatformUpdateRequest(BaseModel):
pass """Request schema - only used in this file."""
name: str | None = None
description: str | None = None
class VendorResponse(VendorBase): @router.get("/platforms/{code}", response_model=PlatformResponse)
id: int def get_platform(...):
is_active: bool ...
class Config:
from_attributes = True
``` ```
**Usage in API:** #### Pattern C: No Schema (Internal Service Use)
```python For models only accessed internally by services, not exposed via API.
from fastapi import APIRouter
from sqlalchemy.orm import Session
from models.database import Vendor
from models.schema import VendorCreate, VendorResponse
router = APIRouter()
@router.post("/vendors", response_model=VendorResponse)
def create_vendor(vendor_data: VendorCreate, db: Session):
# VendorCreate validates input
db_vendor = Vendor(**vendor_data.dict())
db.add(db_vendor)
db.commit()
db.refresh(db_vendor)
# VendorResponse serializes output
return db_vendor
```
---
### Pattern 2: Complex Schemas
For complex domains, organize schemas by purpose:
```python ```python
# models/schema/order.py # Internal association table - no API exposure
class OrderBase(BaseModel): class VendorPlatform(Base):
"""Base order fields""" """Links vendors to platforms - internal use only."""
pass __tablename__ = "vendor_platforms"
vendor_id = Column(Integer, ForeignKey("vendors.id"))
platform_id = Column(Integer, ForeignKey("platforms.id"))
```
class OrderCreate(OrderBase): ### 3. When to Use Each Pattern
"""Create order from customer"""
items: List[OrderItemCreate]
class OrderUpdate(BaseModel): | Scenario | Pattern | Location |
"""Admin order update""" |----------|---------|----------|
status: Optional[OrderStatus] | Schema used by multiple endpoints | Dedicated file | `app/modules/<module>/schemas/` |
| Schema used by single endpoint | Inline | In the route file |
| Admin-only endpoint schemas | Inline | In the admin route file |
| Model not exposed via API | No schema | N/A |
| Cross-module utility schemas | Dedicated file | `models/schema/` |
class OrderResponse(OrderBase): ---
"""Order API response"""
id: int
items: List[OrderItemResponse]
class OrderAdminResponse(OrderResponse): ## Data Flow Architecture
"""Extended response for admin"""
internal_notes: Optional[str] ```
┌─────────────────────────────────────────────────────────────────────┐
│ API Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Request Schema │───▶│ Endpoint │───▶│ Response Schema │ │
│ │ (Pydantic) │ │ │ │ (Pydantic) │ │
│ └─────────────────┘ └────────┬────────┘ └─────────────────┘ │
└──────────────────────────────────┼──────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Service Layer │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Business Logic │ │
│ │ • Validation • Transformations • Business Rules │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Database Models (SQLAlchemy) │ │
│ │ • Direct access • Queries • Transactions │ │
│ └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Database Layer │
│ PostgreSQL / SQLite │
└─────────────────────────────────────────────────────────────────────┘
```
**Key principles:**
- **Services access database models directly** - No schema layer between service and database
- **APIs use schemas for validation** - Request/response validation at API boundary
- **Schemas are API contracts** - Define what clients send/receive
---
## Import Guidelines
### Canonical Imports (Preferred)
```python
# CORE models - from models.database
from models.database import User, Vendor, MediaFile
# Module models - from app.modules.<module>.models
from app.modules.billing.models import Feature, SubscriptionTier
from app.modules.catalog.models import Product, ProductMedia
from app.modules.orders.models import Order, OrderItem
# CORE schemas - from models.schema
from models.schema.auth import LoginRequest, TokenResponse
# Module schemas - from app.modules.<module>.schemas
from app.modules.inventory.schemas import InventoryCreate, InventoryResponse
from app.modules.orders.schemas import OrderResponse
```
### Legacy Re-exports (Backwards Compatibility)
`models/database/__init__.py` re-exports module models for backwards compatibility:
```python
# These work but prefer canonical imports
from models.database import Product # Re-exported from catalog module
from models.database import Order # Re-exported from orders module
``` ```
--- ---
## Migration Guide ## Gap Analysis: Models vs Schemas
If you accidentally created models in the wrong location: Not every database model needs a dedicated schema file. Here's the current alignment:
### Moving Database Models ### CORE Framework
```bash | Database Model | Schema | Notes |
# If you created app/models/my_model.py (WRONG) |----------------|--------|-------|
# Move to correct location: | `user.py` | `auth.py` | Auth schemas cover user operations |
mv app/models/my_model.py models/database/my_model.py | `vendor.py` | `vendor.py` | Full CRUD schemas |
| `company.py` | `company.py` | Full CRUD schemas |
| `media.py` | `media.py` | Upload/response schemas |
| `platform.py` | Inline | Admin-only, in `platforms.py` route |
| `platform_module.py` | Inline | Admin-only, in `modules.py` route |
| `admin_menu_config.py` | Inline | Admin-only, in `menu_config.py` route |
| `vendor_email_settings.py` | Inline | In `email_settings.py` route |
| `vendor_email_template.py` | None | Internal email service use |
| `vendor_platform.py` | None | Internal association table |
# Update imports in all files ### Modules
# FROM: from app.models.my_model import MyModel
# TO: from models.database.my_model import MyModel
# Add to models/database/__init__.py | Module | Models | Schemas | Alignment |
# Remove app/models/ directory |--------|--------|---------|-----------|
rm -rf app/models/ | billing | feature, subscription | billing, subscription | ✅ |
``` | cart | cart | cart | ✅ |
| catalog | product, product_media, product_translation | catalog, product, vendor_product | ✅ |
### Moving Pydantic Schemas | checkout | - | checkout | Schema-only (orchestration) |
| cms | content_page | content_page, homepage_sections | ✅ |
```bash | customers | customer, password_reset_token | customer, context | ✅ (password reset uses auth schemas) |
# If you created app/schemas/my_schema.py (WRONG) | dev_tools | architecture_scan, test_run | Inline | Admin-only, inline in route files |
# Move to correct location: | inventory | inventory, inventory_transaction | inventory | ✅ (transaction included in inventory.py) |
mv app/schemas/my_schema.py models/schema/my_schema.py | loyalty | 5 models | 5 schemas | ✅ |
| marketplace | 5 models | 4 schemas | ✅ (translation in product schema) |
# Update imports | messaging | admin_notification, message | message, notification | ✅ |
# FROM: from app.schemas.my_schema import MySchema | orders | order, order_item, invoice | order, invoice, order_item_exception | ✅ |
# TO: from models.schema.my_schema import MySchema | payments | - | payment | Schema-only (external APIs) |
| analytics | - | stats | Schema-only (aggregation views) |
# Add to models/schema/__init__.py
# Remove app/schemas/ directory
rm -rf app/schemas/
```
--- ---
## Why This Structure? ## Creating New Models
### ✅ Benefits
1. **Clear Separation**
- Database layer separate from application layer
- Easy to understand where models live
2. **Import Consistency**
- `from models.database import ...`
- `from models.schema import ...`
- No confusion about import paths
3. **Testing**
- Easy to mock database models
- Easy to test schema validation
4. **Scalability**
- Models can be used by multiple apps
- Clean separation of concerns
5. **Tool Compatibility**
- Alembic migrations find models easily
- IDE autocomplete works better
- Linters understand structure
### ❌ Problems with `app/models/`
1. **Confusion**: Is it database or schema?
2. **Import Issues**: Circular dependencies
3. **Migration Problems**: Alembic can't find models
4. **Inconsistency**: Different parts of codebase use different paths
---
## Verification Checklist
Use this checklist when adding new models:
### Database Model Checklist ### Database Model Checklist
- [ ] File in `models/database/{name}.py`
- [ ] Inherits from `Base`
- [ ] Has `__tablename__` defined
- [ ] Exported in `models/database/__init__.py`
- [ ] Imported using `from models.database import ...`
- [ ] NO file in `app/models/`
### Pydantic Schema Checklist 1. **Determine location:**
- [ ] File in `models/schema/{name}.py` - Cross-module use → `models/database/`
- [ ] Inherits from `BaseModel` - Module-specific → `app/modules/<module>/models/`
- [ ] Has descriptive suffix (`Create`, `Update`, `Response`)
- [ ] Exported in `models/schema/__init__.py`
- [ ] Imported using `from models.schema import ...`
- [ ] NO file in `app/schemas/`
--- 2. **Create the model file:**
## Project Structure
```
project/
├── app/
│ ├── api/ # API routes
│ ├── core/ # Core functionality (config, database, auth)
│ ├── services/ # Business logic
│ ├── templates/ # Jinja2 templates
│ └── routes/ # Page routes
├── models/ # ✓ Models live here!
│ ├── database/ # ✓ SQLAlchemy models
│ └── schema/ # ✓ Pydantic schemas
├── static/ # Frontend assets
├── docs/ # Documentation
├── tests/ # Tests
└── scripts/ # Utility scripts
```
**NOT:**
```
app/
models/ # ❌ Don't create this
schemas/ # ❌ Don't create this
```
---
## Examples from the Codebase
### ✅ Correct Examples
**Database Model:**
```python ```python
# models/database/architecture_scan.py # app/modules/mymodule/models/my_entity.py
from sqlalchemy import Column, Integer, String from sqlalchemy import Column, Integer, String, ForeignKey
from .base import Base from sqlalchemy.orm import relationship
from app.core.database import Base
from models.database.base import TimestampMixin
class ArchitectureScan(Base): class MyEntity(Base, TimestampMixin):
__tablename__ = "architecture_scans" __tablename__ = "my_entities"
id = Column(Integer, primary_key=True)
id = Column(Integer, primary_key=True, index=True)
vendor_id = Column(Integer, ForeignKey("vendors.id"), nullable=False)
name = Column(String(255), nullable=False)
vendor = relationship("Vendor")
``` ```
**Import in Service:** 3. **Export in `__init__.py`:**
```python ```python
# app/services/code_quality_service.py # app/modules/mymodule/models/__init__.py
from app.modules.dev_tools.models import ArchitectureScan from app.modules.mymodule.models.my_entity import MyEntity
__all__ = ["MyEntity"]
``` ```
**Pydantic Schema:** ### Schema Checklist
1. **Decide if schema is needed:**
- Exposed via API? → Yes, create schema
- Multiple endpoints use it? → Dedicated schema file
- Single endpoint? → Consider inline schema
- Internal only? → No schema needed
2. **Create schema (if needed):**
```python ```python
# models/schema/admin.py # app/modules/mymodule/schemas/my_entity.py
from pydantic import BaseModel, ConfigDict, Field
class MyEntityCreate(BaseModel):
name: str = Field(..., min_length=1, max_length=255)
class MyEntityResponse(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
vendor_id: int
name: str
```
3. **Or use inline schema:**
```python
# app/api/v1/vendor/my_entity.py
from pydantic import BaseModel from pydantic import BaseModel
class AdminDashboardStats(BaseModel): class MyEntityResponse(BaseModel):
total_vendors: int """Response schema - only used in this file."""
total_users: int id: int
``` name: str
**Import in API:** @router.get("/my-entities/{id}", response_model=MyEntityResponse)
```python def get_entity(...):
# app/api/v1/admin/dashboard.py ...
from models.schema.admin import AdminDashboardStats
``` ```
--- ---
## Summary ## Summary
**Golden Rule:** All models in `models/`, never in `app/models/` or `app/schemas/`. | Principle | Implementation |
|-----------|----------------|
**Quick Reference:** | Services access DB directly | `from app.modules.X.models import Model` |
- Database models → `models/database/` | APIs validate with schemas | Request/response Pydantic models |
- Pydantic schemas → `models/schema/` | Reusable schemas | Dedicated files in `schemas/` |
- Import pattern → `from models.{type} import ...` | Endpoint-specific schemas | Inline in route files |
- No models in `app/` directory | Internal models | No schema needed |
| CORE models | `models/database/` |
This standard ensures consistency, clarity, and maintainability across the entire project. | Module models | `app/modules/<module>/models/` |