Files
orion/docs/architecture/models-structure.md
Samir Boulahtit cc74970223 feat: add logging, marketplace, and admin enhancements
Database & Migrations:
- Add application_logs table migration for hybrid cloud logging
- Add companies table migration and restructure vendor relationships

Logging System:
- Implement hybrid logging system (database + file)
- Add log_service for centralized log management
- Create admin logs page with filtering and viewing capabilities
- Add init_log_settings.py script for log configuration
- Enhance core logging with database integration

Marketplace Integration:
- Add marketplace admin page with product management
- Create marketplace vendor page with product listings
- Implement marketplace.js for both admin and vendor interfaces
- Add marketplace integration documentation

Admin Enhancements:
- Add imports management page and functionality
- Create settings page for admin configuration
- Add vendor themes management page
- Enhance vendor detail and edit pages
- Improve code quality dashboard and violation details
- Add logs viewing and management
- Update icons guide and shared icon system

Architecture & Documentation:
- Document frontend structure and component architecture
- Document models structure and relationships
- Add vendor-in-token architecture documentation
- Add vendor RBAC (role-based access control) documentation
- Document marketplace integration patterns
- Update architecture patterns documentation

Infrastructure:
- Add platform static files structure (css, img, js)
- Move architecture_scan.py to proper models location
- Update model imports and registrations
- Enhance exception handling
- Update dependency injection patterns

UI/UX:
- Improve vendor edit interface
- Update admin user interface
- Enhance page templates documentation
- Add vendor marketplace interface
2025-12-01 21:51:07 +01:00

470 lines
10 KiB
Markdown

# Models Structure
## Overview
This project follows a **standardized models structure** at the root level, separating database models from Pydantic schemas.
## Directory Structure
```
models/
├── database/ # SQLAlchemy database models (ORM)
│ ├── __init__.py
│ ├── user.py
│ ├── vendor.py
│ ├── product.py
│ ├── order.py
│ ├── admin.py
│ ├── architecture_scan.py
│ └── ...
└── schema/ # Pydantic schemas (API validation)
├── __init__.py
├── auth.py
├── admin.py
├── product.py
├── order.py
└── ...
```
## Important Rules
### ✅ DO: Use Root-Level Models
**ALL models must be in the root `models/` directory:**
- Database models → `models/database/`
- Pydantic schemas → `models/schema/`
### ❌ DON'T: Create `app/models/`
**NEVER create or use `app/models/` directory.**
The application structure is:
```
app/ # Application code (routes, services, core)
models/ # Models (database & schemas)
```
NOT:
```
app/
models/ # ❌ WRONG - Don't create this!
models/ # ✓ Correct location
```
---
## Database Models (`models/database/`)
### Purpose
SQLAlchemy ORM models that represent database tables.
### Naming Convention
- Singular class names: `User`, `Product`, `Order`
- File names match class: `user.py`, `product.py`, `order.py`
### Example Structure
**File:** `models/database/product.py`
```python
"""Product database model"""
from sqlalchemy import Column, Integer, String, Float, ForeignKey
from sqlalchemy.orm import relationship
from .base import Base
class Product(Base):
"""Product database model"""
__tablename__ = "products"
id = Column(Integer, primary_key=True, index=True)
name = Column(String(255), nullable=False)
price = Column(Float, nullable=False)
vendor_id = Column(Integer, ForeignKey("vendors.id"))
# Relationships
vendor = relationship("Vendor", back_populates="products")
```
### Exporting Models
All database models must be exported in `models/database/__init__.py`:
```python
# models/database/__init__.py
from .user import User
from .vendor import Vendor
from .product import Product
from .order import Order, OrderItem
__all__ = [
"User",
"Vendor",
"Product",
"Order",
"OrderItem",
]
```
### Importing Database Models
```python
# ✅ CORRECT - Import from models.database
from models.database import User, Product
from models.database.vendor import Vendor
# ❌ WRONG - Don't import from app.models
from app.models.user import User # This path doesn't exist!
```
---
## Pydantic Schemas (`models/schema/`)
### Purpose
Pydantic models for API request/response validation and serialization.
### Naming Convention
- Use descriptive suffixes: `Create`, `Update`, `Response`, `InDB`
- Group related schemas in same file
- File names match domain: `auth.py`, `product.py`, `order.py`
### Example Structure
**File:** `models/schema/product.py`
```python
"""Product Pydantic schemas"""
from typing import Optional
from pydantic import BaseModel, Field
class ProductBase(BaseModel):
"""Base product schema"""
name: str = Field(..., min_length=1, max_length=255)
description: Optional[str] = None
price: float = Field(..., gt=0)
class ProductCreate(ProductBase):
"""Schema for creating a product"""
vendor_id: int
class ProductUpdate(BaseModel):
"""Schema for updating a product"""
name: Optional[str] = Field(None, min_length=1, max_length=255)
description: Optional[str] = None
price: Optional[float] = Field(None, gt=0)
class ProductResponse(ProductBase):
"""Schema for product API response"""
id: int
vendor_id: int
class Config:
from_attributes = True # Pydantic v2
# orm_mode = True # Pydantic v1
```
### Exporting Schemas
Export schemas in `models/schema/__init__.py`:
```python
# models/schema/__init__.py
from .auth import LoginRequest, TokenResponse
from .product import ProductCreate, ProductUpdate, ProductResponse
__all__ = [
"LoginRequest",
"TokenResponse",
"ProductCreate",
"ProductUpdate",
"ProductResponse",
]
```
### Importing Schemas
```python
# ✅ CORRECT
from models.schema import ProductCreate, ProductResponse
from models.schema.auth import LoginRequest
# ❌ WRONG
from app.models.schema.product import ProductCreate
```
---
## Common Patterns
### Pattern 1: Database Model with Schema
**Database Model:** `models/database/vendor.py`
```python
from sqlalchemy import Column, Integer, String, Boolean
from .base import Base
class Vendor(Base):
__tablename__ = "vendors"
id = Column(Integer, primary_key=True)
name = Column(String(255), nullable=False)
code = Column(String(50), unique=True, nullable=False)
is_active = Column(Boolean, default=True)
```
**Pydantic Schema:** `models/schema/vendor.py`
```python
from pydantic import BaseModel
class VendorBase(BaseModel):
name: str
code: str
class VendorCreate(VendorBase):
pass
class VendorResponse(VendorBase):
id: int
is_active: bool
class Config:
from_attributes = True
```
**Usage in API:**
```python
from fastapi import APIRouter
from sqlalchemy.orm import Session
from models.database import Vendor
from models.schema import VendorCreate, VendorResponse
router = APIRouter()
@router.post("/vendors", response_model=VendorResponse)
def create_vendor(vendor_data: VendorCreate, db: Session):
# VendorCreate validates input
db_vendor = Vendor(**vendor_data.dict())
db.add(db_vendor)
db.commit()
db.refresh(db_vendor)
# VendorResponse serializes output
return db_vendor
```
---
### Pattern 2: Complex Schemas
For complex domains, organize schemas by purpose:
```python
# models/schema/order.py
class OrderBase(BaseModel):
"""Base order fields"""
pass
class OrderCreate(OrderBase):
"""Create order from customer"""
items: List[OrderItemCreate]
class OrderUpdate(BaseModel):
"""Admin order update"""
status: Optional[OrderStatus]
class OrderResponse(OrderBase):
"""Order API response"""
id: int
items: List[OrderItemResponse]
class OrderAdminResponse(OrderResponse):
"""Extended response for admin"""
internal_notes: Optional[str]
```
---
## Migration Guide
If you accidentally created models in the wrong location:
### Moving Database Models
```bash
# If you created app/models/my_model.py (WRONG)
# Move to correct location:
mv app/models/my_model.py models/database/my_model.py
# Update imports in all files
# FROM: from app.models.my_model import MyModel
# TO: from models.database.my_model import MyModel
# Add to models/database/__init__.py
# Remove app/models/ directory
rm -rf app/models/
```
### Moving Pydantic Schemas
```bash
# If you created app/schemas/my_schema.py (WRONG)
# Move to correct location:
mv app/schemas/my_schema.py models/schema/my_schema.py
# Update imports
# FROM: from app.schemas.my_schema import MySchema
# TO: from models.schema.my_schema import MySchema
# Add to models/schema/__init__.py
# Remove app/schemas/ directory
rm -rf app/schemas/
```
---
## Why This Structure?
### ✅ Benefits
1. **Clear Separation**
- Database layer separate from application layer
- Easy to understand where models live
2. **Import Consistency**
- `from models.database import ...`
- `from models.schema import ...`
- No confusion about import paths
3. **Testing**
- Easy to mock database models
- Easy to test schema validation
4. **Scalability**
- Models can be used by multiple apps
- Clean separation of concerns
5. **Tool Compatibility**
- Alembic migrations find models easily
- IDE autocomplete works better
- Linters understand structure
### ❌ Problems with `app/models/`
1. **Confusion**: Is it database or schema?
2. **Import Issues**: Circular dependencies
3. **Migration Problems**: Alembic can't find models
4. **Inconsistency**: Different parts of codebase use different paths
---
## Verification Checklist
Use this checklist when adding new models:
### Database Model Checklist
- [ ] File in `models/database/{name}.py`
- [ ] Inherits from `Base`
- [ ] Has `__tablename__` defined
- [ ] Exported in `models/database/__init__.py`
- [ ] Imported using `from models.database import ...`
- [ ] NO file in `app/models/`
### Pydantic Schema Checklist
- [ ] File in `models/schema/{name}.py`
- [ ] Inherits from `BaseModel`
- [ ] Has descriptive suffix (`Create`, `Update`, `Response`)
- [ ] Exported in `models/schema/__init__.py`
- [ ] Imported using `from models.schema import ...`
- [ ] NO file in `app/schemas/`
---
## Project Structure
```
project/
├── app/
│ ├── api/ # API routes
│ ├── core/ # Core functionality (config, database, auth)
│ ├── services/ # Business logic
│ ├── templates/ # Jinja2 templates
│ └── routes/ # Page routes
├── models/ # ✓ Models live here!
│ ├── database/ # ✓ SQLAlchemy models
│ └── schema/ # ✓ Pydantic schemas
├── static/ # Frontend assets
├── docs/ # Documentation
├── tests/ # Tests
└── scripts/ # Utility scripts
```
**NOT:**
```
app/
models/ # ❌ Don't create this
schemas/ # ❌ Don't create this
```
---
## Examples from the Codebase
### ✅ Correct Examples
**Database Model:**
```python
# models/database/architecture_scan.py
from sqlalchemy import Column, Integer, String
from .base import Base
class ArchitectureScan(Base):
__tablename__ = "architecture_scans"
id = Column(Integer, primary_key=True)
```
**Import in Service:**
```python
# app/services/code_quality_service.py
from models.database.architecture_scan import ArchitectureScan
```
**Pydantic Schema:**
```python
# models/schema/admin.py
from pydantic import BaseModel
class AdminDashboardStats(BaseModel):
total_vendors: int
total_users: int
```
**Import in API:**
```python
# app/api/v1/admin/dashboard.py
from models.schema.admin import AdminDashboardStats
```
---
## Summary
**Golden Rule:** All models in `models/`, never in `app/models/` or `app/schemas/`.
**Quick Reference:**
- Database models → `models/database/`
- Pydantic schemas → `models/schema/`
- Import pattern → `from models.{type} import ...`
- No models in `app/` directory
This standard ensures consistency, clarity, and maintainability across the entire project.