Database & Migrations: - Add application_logs table migration for hybrid cloud logging - Add companies table migration and restructure vendor relationships Logging System: - Implement hybrid logging system (database + file) - Add log_service for centralized log management - Create admin logs page with filtering and viewing capabilities - Add init_log_settings.py script for log configuration - Enhance core logging with database integration Marketplace Integration: - Add marketplace admin page with product management - Create marketplace vendor page with product listings - Implement marketplace.js for both admin and vendor interfaces - Add marketplace integration documentation Admin Enhancements: - Add imports management page and functionality - Create settings page for admin configuration - Add vendor themes management page - Enhance vendor detail and edit pages - Improve code quality dashboard and violation details - Add logs viewing and management - Update icons guide and shared icon system Architecture & Documentation: - Document frontend structure and component architecture - Document models structure and relationships - Add vendor-in-token architecture documentation - Add vendor RBAC (role-based access control) documentation - Document marketplace integration patterns - Update architecture patterns documentation Infrastructure: - Add platform static files structure (css, img, js) - Move architecture_scan.py to proper models location - Update model imports and registrations - Enhance exception handling - Update dependency injection patterns UI/UX: - Improve vendor edit interface - Update admin user interface - Enhance page templates documentation - Add vendor marketplace interface
470 lines
10 KiB
Markdown
470 lines
10 KiB
Markdown
# Models Structure
|
|
|
|
## Overview
|
|
|
|
This project follows a **standardized models structure** at the root level, separating database models from Pydantic schemas.
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
models/
|
|
├── database/ # SQLAlchemy database models (ORM)
|
|
│ ├── __init__.py
|
|
│ ├── user.py
|
|
│ ├── vendor.py
|
|
│ ├── product.py
|
|
│ ├── order.py
|
|
│ ├── admin.py
|
|
│ ├── architecture_scan.py
|
|
│ └── ...
|
|
│
|
|
└── schema/ # Pydantic schemas (API validation)
|
|
├── __init__.py
|
|
├── auth.py
|
|
├── admin.py
|
|
├── product.py
|
|
├── order.py
|
|
└── ...
|
|
```
|
|
|
|
## Important Rules
|
|
|
|
### ✅ DO: Use Root-Level Models
|
|
|
|
**ALL models must be in the root `models/` directory:**
|
|
- Database models → `models/database/`
|
|
- Pydantic schemas → `models/schema/`
|
|
|
|
### ❌ DON'T: Create `app/models/`
|
|
|
|
**NEVER create or use `app/models/` directory.**
|
|
|
|
The application structure is:
|
|
```
|
|
app/ # Application code (routes, services, core)
|
|
models/ # Models (database & schemas)
|
|
```
|
|
|
|
NOT:
|
|
```
|
|
app/
|
|
models/ # ❌ WRONG - Don't create this!
|
|
models/ # ✓ Correct location
|
|
```
|
|
|
|
---
|
|
|
|
## Database Models (`models/database/`)
|
|
|
|
### Purpose
|
|
SQLAlchemy ORM models that represent database tables.
|
|
|
|
### Naming Convention
|
|
- Singular class names: `User`, `Product`, `Order`
|
|
- File names match class: `user.py`, `product.py`, `order.py`
|
|
|
|
### Example Structure
|
|
|
|
**File:** `models/database/product.py`
|
|
```python
|
|
"""Product database model"""
|
|
|
|
from sqlalchemy import Column, Integer, String, Float, ForeignKey
|
|
from sqlalchemy.orm import relationship
|
|
|
|
from .base import Base
|
|
|
|
|
|
class Product(Base):
|
|
"""Product database model"""
|
|
|
|
__tablename__ = "products"
|
|
|
|
id = Column(Integer, primary_key=True, index=True)
|
|
name = Column(String(255), nullable=False)
|
|
price = Column(Float, nullable=False)
|
|
vendor_id = Column(Integer, ForeignKey("vendors.id"))
|
|
|
|
# Relationships
|
|
vendor = relationship("Vendor", back_populates="products")
|
|
```
|
|
|
|
### Exporting Models
|
|
|
|
All database models must be exported in `models/database/__init__.py`:
|
|
|
|
```python
|
|
# models/database/__init__.py
|
|
from .user import User
|
|
from .vendor import Vendor
|
|
from .product import Product
|
|
from .order import Order, OrderItem
|
|
|
|
__all__ = [
|
|
"User",
|
|
"Vendor",
|
|
"Product",
|
|
"Order",
|
|
"OrderItem",
|
|
]
|
|
```
|
|
|
|
### Importing Database Models
|
|
|
|
```python
|
|
# ✅ CORRECT - Import from models.database
|
|
from models.database import User, Product
|
|
from models.database.vendor import Vendor
|
|
|
|
# ❌ WRONG - Don't import from app.models
|
|
from app.models.user import User # This path doesn't exist!
|
|
```
|
|
|
|
---
|
|
|
|
## Pydantic Schemas (`models/schema/`)
|
|
|
|
### Purpose
|
|
Pydantic models for API request/response validation and serialization.
|
|
|
|
### Naming Convention
|
|
- Use descriptive suffixes: `Create`, `Update`, `Response`, `InDB`
|
|
- Group related schemas in same file
|
|
- File names match domain: `auth.py`, `product.py`, `order.py`
|
|
|
|
### Example Structure
|
|
|
|
**File:** `models/schema/product.py`
|
|
```python
|
|
"""Product Pydantic schemas"""
|
|
|
|
from typing import Optional
|
|
from pydantic import BaseModel, Field
|
|
|
|
|
|
class ProductBase(BaseModel):
|
|
"""Base product schema"""
|
|
name: str = Field(..., min_length=1, max_length=255)
|
|
description: Optional[str] = None
|
|
price: float = Field(..., gt=0)
|
|
|
|
|
|
class ProductCreate(ProductBase):
|
|
"""Schema for creating a product"""
|
|
vendor_id: int
|
|
|
|
|
|
class ProductUpdate(BaseModel):
|
|
"""Schema for updating a product"""
|
|
name: Optional[str] = Field(None, min_length=1, max_length=255)
|
|
description: Optional[str] = None
|
|
price: Optional[float] = Field(None, gt=0)
|
|
|
|
|
|
class ProductResponse(ProductBase):
|
|
"""Schema for product API response"""
|
|
id: int
|
|
vendor_id: int
|
|
|
|
class Config:
|
|
from_attributes = True # Pydantic v2
|
|
# orm_mode = True # Pydantic v1
|
|
```
|
|
|
|
### Exporting Schemas
|
|
|
|
Export schemas in `models/schema/__init__.py`:
|
|
|
|
```python
|
|
# models/schema/__init__.py
|
|
from .auth import LoginRequest, TokenResponse
|
|
from .product import ProductCreate, ProductUpdate, ProductResponse
|
|
|
|
__all__ = [
|
|
"LoginRequest",
|
|
"TokenResponse",
|
|
"ProductCreate",
|
|
"ProductUpdate",
|
|
"ProductResponse",
|
|
]
|
|
```
|
|
|
|
### Importing Schemas
|
|
|
|
```python
|
|
# ✅ CORRECT
|
|
from models.schema import ProductCreate, ProductResponse
|
|
from models.schema.auth import LoginRequest
|
|
|
|
# ❌ WRONG
|
|
from app.models.schema.product import ProductCreate
|
|
```
|
|
|
|
---
|
|
|
|
## Common Patterns
|
|
|
|
### Pattern 1: Database Model with Schema
|
|
|
|
**Database Model:** `models/database/vendor.py`
|
|
```python
|
|
from sqlalchemy import Column, Integer, String, Boolean
|
|
from .base import Base
|
|
|
|
class Vendor(Base):
|
|
__tablename__ = "vendors"
|
|
|
|
id = Column(Integer, primary_key=True)
|
|
name = Column(String(255), nullable=False)
|
|
code = Column(String(50), unique=True, nullable=False)
|
|
is_active = Column(Boolean, default=True)
|
|
```
|
|
|
|
**Pydantic Schema:** `models/schema/vendor.py`
|
|
```python
|
|
from pydantic import BaseModel
|
|
|
|
class VendorBase(BaseModel):
|
|
name: str
|
|
code: str
|
|
|
|
class VendorCreate(VendorBase):
|
|
pass
|
|
|
|
class VendorResponse(VendorBase):
|
|
id: int
|
|
is_active: bool
|
|
|
|
class Config:
|
|
from_attributes = True
|
|
```
|
|
|
|
**Usage in API:**
|
|
```python
|
|
from fastapi import APIRouter
|
|
from sqlalchemy.orm import Session
|
|
|
|
from models.database import Vendor
|
|
from models.schema import VendorCreate, VendorResponse
|
|
|
|
router = APIRouter()
|
|
|
|
@router.post("/vendors", response_model=VendorResponse)
|
|
def create_vendor(vendor_data: VendorCreate, db: Session):
|
|
# VendorCreate validates input
|
|
db_vendor = Vendor(**vendor_data.dict())
|
|
db.add(db_vendor)
|
|
db.commit()
|
|
db.refresh(db_vendor)
|
|
# VendorResponse serializes output
|
|
return db_vendor
|
|
```
|
|
|
|
---
|
|
|
|
### Pattern 2: Complex Schemas
|
|
|
|
For complex domains, organize schemas by purpose:
|
|
|
|
```python
|
|
# models/schema/order.py
|
|
class OrderBase(BaseModel):
|
|
"""Base order fields"""
|
|
pass
|
|
|
|
class OrderCreate(OrderBase):
|
|
"""Create order from customer"""
|
|
items: List[OrderItemCreate]
|
|
|
|
class OrderUpdate(BaseModel):
|
|
"""Admin order update"""
|
|
status: Optional[OrderStatus]
|
|
|
|
class OrderResponse(OrderBase):
|
|
"""Order API response"""
|
|
id: int
|
|
items: List[OrderItemResponse]
|
|
|
|
class OrderAdminResponse(OrderResponse):
|
|
"""Extended response for admin"""
|
|
internal_notes: Optional[str]
|
|
```
|
|
|
|
---
|
|
|
|
## Migration Guide
|
|
|
|
If you accidentally created models in the wrong location:
|
|
|
|
### Moving Database Models
|
|
|
|
```bash
|
|
# If you created app/models/my_model.py (WRONG)
|
|
# Move to correct location:
|
|
mv app/models/my_model.py models/database/my_model.py
|
|
|
|
# Update imports in all files
|
|
# FROM: from app.models.my_model import MyModel
|
|
# TO: from models.database.my_model import MyModel
|
|
|
|
# Add to models/database/__init__.py
|
|
# Remove app/models/ directory
|
|
rm -rf app/models/
|
|
```
|
|
|
|
### Moving Pydantic Schemas
|
|
|
|
```bash
|
|
# If you created app/schemas/my_schema.py (WRONG)
|
|
# Move to correct location:
|
|
mv app/schemas/my_schema.py models/schema/my_schema.py
|
|
|
|
# Update imports
|
|
# FROM: from app.schemas.my_schema import MySchema
|
|
# TO: from models.schema.my_schema import MySchema
|
|
|
|
# Add to models/schema/__init__.py
|
|
# Remove app/schemas/ directory
|
|
rm -rf app/schemas/
|
|
```
|
|
|
|
---
|
|
|
|
## Why This Structure?
|
|
|
|
### ✅ Benefits
|
|
|
|
1. **Clear Separation**
|
|
- Database layer separate from application layer
|
|
- Easy to understand where models live
|
|
|
|
2. **Import Consistency**
|
|
- `from models.database import ...`
|
|
- `from models.schema import ...`
|
|
- No confusion about import paths
|
|
|
|
3. **Testing**
|
|
- Easy to mock database models
|
|
- Easy to test schema validation
|
|
|
|
4. **Scalability**
|
|
- Models can be used by multiple apps
|
|
- Clean separation of concerns
|
|
|
|
5. **Tool Compatibility**
|
|
- Alembic migrations find models easily
|
|
- IDE autocomplete works better
|
|
- Linters understand structure
|
|
|
|
### ❌ Problems with `app/models/`
|
|
|
|
1. **Confusion**: Is it database or schema?
|
|
2. **Import Issues**: Circular dependencies
|
|
3. **Migration Problems**: Alembic can't find models
|
|
4. **Inconsistency**: Different parts of codebase use different paths
|
|
|
|
---
|
|
|
|
## Verification Checklist
|
|
|
|
Use this checklist when adding new models:
|
|
|
|
### Database Model Checklist
|
|
- [ ] File in `models/database/{name}.py`
|
|
- [ ] Inherits from `Base`
|
|
- [ ] Has `__tablename__` defined
|
|
- [ ] Exported in `models/database/__init__.py`
|
|
- [ ] Imported using `from models.database import ...`
|
|
- [ ] NO file in `app/models/`
|
|
|
|
### Pydantic Schema Checklist
|
|
- [ ] File in `models/schema/{name}.py`
|
|
- [ ] Inherits from `BaseModel`
|
|
- [ ] Has descriptive suffix (`Create`, `Update`, `Response`)
|
|
- [ ] Exported in `models/schema/__init__.py`
|
|
- [ ] Imported using `from models.schema import ...`
|
|
- [ ] NO file in `app/schemas/`
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
project/
|
|
├── app/
|
|
│ ├── api/ # API routes
|
|
│ ├── core/ # Core functionality (config, database, auth)
|
|
│ ├── services/ # Business logic
|
|
│ ├── templates/ # Jinja2 templates
|
|
│ └── routes/ # Page routes
|
|
│
|
|
├── models/ # ✓ Models live here!
|
|
│ ├── database/ # ✓ SQLAlchemy models
|
|
│ └── schema/ # ✓ Pydantic schemas
|
|
│
|
|
├── static/ # Frontend assets
|
|
├── docs/ # Documentation
|
|
├── tests/ # Tests
|
|
└── scripts/ # Utility scripts
|
|
```
|
|
|
|
**NOT:**
|
|
```
|
|
app/
|
|
models/ # ❌ Don't create this
|
|
schemas/ # ❌ Don't create this
|
|
```
|
|
|
|
---
|
|
|
|
## Examples from the Codebase
|
|
|
|
### ✅ Correct Examples
|
|
|
|
**Database Model:**
|
|
```python
|
|
# models/database/architecture_scan.py
|
|
from sqlalchemy import Column, Integer, String
|
|
from .base import Base
|
|
|
|
class ArchitectureScan(Base):
|
|
__tablename__ = "architecture_scans"
|
|
id = Column(Integer, primary_key=True)
|
|
```
|
|
|
|
**Import in Service:**
|
|
```python
|
|
# app/services/code_quality_service.py
|
|
from models.database.architecture_scan import ArchitectureScan
|
|
```
|
|
|
|
**Pydantic Schema:**
|
|
```python
|
|
# models/schema/admin.py
|
|
from pydantic import BaseModel
|
|
|
|
class AdminDashboardStats(BaseModel):
|
|
total_vendors: int
|
|
total_users: int
|
|
```
|
|
|
|
**Import in API:**
|
|
```python
|
|
# app/api/v1/admin/dashboard.py
|
|
from models.schema.admin import AdminDashboardStats
|
|
```
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
**Golden Rule:** All models in `models/`, never in `app/models/` or `app/schemas/`.
|
|
|
|
**Quick Reference:**
|
|
- Database models → `models/database/`
|
|
- Pydantic schemas → `models/schema/`
|
|
- Import pattern → `from models.{type} import ...`
|
|
- No models in `app/` directory
|
|
|
|
This standard ensures consistency, clarity, and maintainability across the entire project.
|