Database & Migrations: - Add application_logs table migration for hybrid cloud logging - Add companies table migration and restructure vendor relationships Logging System: - Implement hybrid logging system (database + file) - Add log_service for centralized log management - Create admin logs page with filtering and viewing capabilities - Add init_log_settings.py script for log configuration - Enhance core logging with database integration Marketplace Integration: - Add marketplace admin page with product management - Create marketplace vendor page with product listings - Implement marketplace.js for both admin and vendor interfaces - Add marketplace integration documentation Admin Enhancements: - Add imports management page and functionality - Create settings page for admin configuration - Add vendor themes management page - Enhance vendor detail and edit pages - Improve code quality dashboard and violation details - Add logs viewing and management - Update icons guide and shared icon system Architecture & Documentation: - Document frontend structure and component architecture - Document models structure and relationships - Add vendor-in-token architecture documentation - Add vendor RBAC (role-based access control) documentation - Document marketplace integration patterns - Update architecture patterns documentation Infrastructure: - Add platform static files structure (css, img, js) - Move architecture_scan.py to proper models location - Update model imports and registrations - Enhance exception handling - Update dependency injection patterns UI/UX: - Improve vendor edit interface - Update admin user interface - Enhance page templates documentation - Add vendor marketplace interface
10 KiB
Models Structure
Overview
This project follows a standardized models structure at the root level, separating database models from Pydantic schemas.
Directory Structure
models/
├── database/ # SQLAlchemy database models (ORM)
│ ├── __init__.py
│ ├── user.py
│ ├── vendor.py
│ ├── product.py
│ ├── order.py
│ ├── admin.py
│ ├── architecture_scan.py
│ └── ...
│
└── schema/ # Pydantic schemas (API validation)
├── __init__.py
├── auth.py
├── admin.py
├── product.py
├── order.py
└── ...
Important Rules
✅ DO: Use Root-Level Models
ALL models must be in the root models/ directory:
- Database models →
models/database/ - Pydantic schemas →
models/schema/
❌ DON'T: Create app/models/
NEVER create or use app/models/ directory.
The application structure is:
app/ # Application code (routes, services, core)
models/ # Models (database & schemas)
NOT:
app/
models/ # ❌ WRONG - Don't create this!
models/ # ✓ Correct location
Database Models (models/database/)
Purpose
SQLAlchemy ORM models that represent database tables.
Naming Convention
- Singular class names:
User,Product,Order - File names match class:
user.py,product.py,order.py
Example Structure
File: models/database/product.py
"""Product database model"""
from sqlalchemy import Column, Integer, String, Float, ForeignKey
from sqlalchemy.orm import relationship
from .base import Base
class Product(Base):
"""Product database model"""
__tablename__ = "products"
id = Column(Integer, primary_key=True, index=True)
name = Column(String(255), nullable=False)
price = Column(Float, nullable=False)
vendor_id = Column(Integer, ForeignKey("vendors.id"))
# Relationships
vendor = relationship("Vendor", back_populates="products")
Exporting Models
All database models must be exported in models/database/__init__.py:
# models/database/__init__.py
from .user import User
from .vendor import Vendor
from .product import Product
from .order import Order, OrderItem
__all__ = [
"User",
"Vendor",
"Product",
"Order",
"OrderItem",
]
Importing Database Models
# ✅ CORRECT - Import from models.database
from models.database import User, Product
from models.database.vendor import Vendor
# ❌ WRONG - Don't import from app.models
from app.models.user import User # This path doesn't exist!
Pydantic Schemas (models/schema/)
Purpose
Pydantic models for API request/response validation and serialization.
Naming Convention
- Use descriptive suffixes:
Create,Update,Response,InDB - Group related schemas in same file
- File names match domain:
auth.py,product.py,order.py
Example Structure
File: models/schema/product.py
"""Product Pydantic schemas"""
from typing import Optional
from pydantic import BaseModel, Field
class ProductBase(BaseModel):
"""Base product schema"""
name: str = Field(..., min_length=1, max_length=255)
description: Optional[str] = None
price: float = Field(..., gt=0)
class ProductCreate(ProductBase):
"""Schema for creating a product"""
vendor_id: int
class ProductUpdate(BaseModel):
"""Schema for updating a product"""
name: Optional[str] = Field(None, min_length=1, max_length=255)
description: Optional[str] = None
price: Optional[float] = Field(None, gt=0)
class ProductResponse(ProductBase):
"""Schema for product API response"""
id: int
vendor_id: int
class Config:
from_attributes = True # Pydantic v2
# orm_mode = True # Pydantic v1
Exporting Schemas
Export schemas in models/schema/__init__.py:
# models/schema/__init__.py
from .auth import LoginRequest, TokenResponse
from .product import ProductCreate, ProductUpdate, ProductResponse
__all__ = [
"LoginRequest",
"TokenResponse",
"ProductCreate",
"ProductUpdate",
"ProductResponse",
]
Importing Schemas
# ✅ CORRECT
from models.schema import ProductCreate, ProductResponse
from models.schema.auth import LoginRequest
# ❌ WRONG
from app.models.schema.product import ProductCreate
Common Patterns
Pattern 1: Database Model with Schema
Database Model: models/database/vendor.py
from sqlalchemy import Column, Integer, String, Boolean
from .base import Base
class Vendor(Base):
__tablename__ = "vendors"
id = Column(Integer, primary_key=True)
name = Column(String(255), nullable=False)
code = Column(String(50), unique=True, nullable=False)
is_active = Column(Boolean, default=True)
Pydantic Schema: models/schema/vendor.py
from pydantic import BaseModel
class VendorBase(BaseModel):
name: str
code: str
class VendorCreate(VendorBase):
pass
class VendorResponse(VendorBase):
id: int
is_active: bool
class Config:
from_attributes = True
Usage in API:
from fastapi import APIRouter
from sqlalchemy.orm import Session
from models.database import Vendor
from models.schema import VendorCreate, VendorResponse
router = APIRouter()
@router.post("/vendors", response_model=VendorResponse)
def create_vendor(vendor_data: VendorCreate, db: Session):
# VendorCreate validates input
db_vendor = Vendor(**vendor_data.dict())
db.add(db_vendor)
db.commit()
db.refresh(db_vendor)
# VendorResponse serializes output
return db_vendor
Pattern 2: Complex Schemas
For complex domains, organize schemas by purpose:
# models/schema/order.py
class OrderBase(BaseModel):
"""Base order fields"""
pass
class OrderCreate(OrderBase):
"""Create order from customer"""
items: List[OrderItemCreate]
class OrderUpdate(BaseModel):
"""Admin order update"""
status: Optional[OrderStatus]
class OrderResponse(OrderBase):
"""Order API response"""
id: int
items: List[OrderItemResponse]
class OrderAdminResponse(OrderResponse):
"""Extended response for admin"""
internal_notes: Optional[str]
Migration Guide
If you accidentally created models in the wrong location:
Moving Database Models
# If you created app/models/my_model.py (WRONG)
# Move to correct location:
mv app/models/my_model.py models/database/my_model.py
# Update imports in all files
# FROM: from app.models.my_model import MyModel
# TO: from models.database.my_model import MyModel
# Add to models/database/__init__.py
# Remove app/models/ directory
rm -rf app/models/
Moving Pydantic Schemas
# If you created app/schemas/my_schema.py (WRONG)
# Move to correct location:
mv app/schemas/my_schema.py models/schema/my_schema.py
# Update imports
# FROM: from app.schemas.my_schema import MySchema
# TO: from models.schema.my_schema import MySchema
# Add to models/schema/__init__.py
# Remove app/schemas/ directory
rm -rf app/schemas/
Why This Structure?
✅ Benefits
-
Clear Separation
- Database layer separate from application layer
- Easy to understand where models live
-
Import Consistency
from models.database import ...from models.schema import ...- No confusion about import paths
-
Testing
- Easy to mock database models
- Easy to test schema validation
-
Scalability
- Models can be used by multiple apps
- Clean separation of concerns
-
Tool Compatibility
- Alembic migrations find models easily
- IDE autocomplete works better
- Linters understand structure
❌ Problems with app/models/
- Confusion: Is it database or schema?
- Import Issues: Circular dependencies
- Migration Problems: Alembic can't find models
- Inconsistency: Different parts of codebase use different paths
Verification Checklist
Use this checklist when adding new models:
Database Model Checklist
- File in
models/database/{name}.py - Inherits from
Base - Has
__tablename__defined - Exported in
models/database/__init__.py - Imported using
from models.database import ... - NO file in
app/models/
Pydantic Schema Checklist
- File in
models/schema/{name}.py - Inherits from
BaseModel - Has descriptive suffix (
Create,Update,Response) - Exported in
models/schema/__init__.py - Imported using
from models.schema import ... - NO file in
app/schemas/
Project Structure
project/
├── app/
│ ├── api/ # API routes
│ ├── core/ # Core functionality (config, database, auth)
│ ├── services/ # Business logic
│ ├── templates/ # Jinja2 templates
│ └── routes/ # Page routes
│
├── models/ # ✓ Models live here!
│ ├── database/ # ✓ SQLAlchemy models
│ └── schema/ # ✓ Pydantic schemas
│
├── static/ # Frontend assets
├── docs/ # Documentation
├── tests/ # Tests
└── scripts/ # Utility scripts
NOT:
app/
models/ # ❌ Don't create this
schemas/ # ❌ Don't create this
Examples from the Codebase
✅ Correct Examples
Database Model:
# models/database/architecture_scan.py
from sqlalchemy import Column, Integer, String
from .base import Base
class ArchitectureScan(Base):
__tablename__ = "architecture_scans"
id = Column(Integer, primary_key=True)
Import in Service:
# app/services/code_quality_service.py
from models.database.architecture_scan import ArchitectureScan
Pydantic Schema:
# models/schema/admin.py
from pydantic import BaseModel
class AdminDashboardStats(BaseModel):
total_vendors: int
total_users: int
Import in API:
# app/api/v1/admin/dashboard.py
from models.schema.admin import AdminDashboardStats
Summary
Golden Rule: All models in models/, never in app/models/ or app/schemas/.
Quick Reference:
- Database models →
models/database/ - Pydantic schemas →
models/schema/ - Import pattern →
from models.{type} import ... - No models in
app/directory
This standard ensures consistency, clarity, and maintainability across the entire project.