orion/tests_restructure.md

Tests Folder Restructure Plan

Current vs Recommended Structure

Before (Single Folder)

tests/
├── test_auth.py
├── test_products.py
├── test_stock.py
├── test_shops.py
├── test_marketplace.py
├── test_admin.py
├── test_stats.py
├── test_database.py
├── test_utils.py
├── conftest.py
└── ...more files

After (Organized Structure)

tests/
├── conftest.py                    # Global test configuration and fixtures
├── pytest.ini                    # Pytest configuration
├── __init__.py
├── fixtures/                     # Shared test fixtures
│   ├── __init__.py
│   ├── auth_fixtures.py          # Auth-related fixtures
│   ├── product_fixtures.py       # Product fixtures
│   ├── shop_fixtures.py          # Shop fixtures
│   └── database_fixtures.py      # Database setup fixtures
├── unit/                         # Unit tests (isolated, fast)
│   ├── __init__.py
│   ├── conftest.py              # Unit test specific fixtures
│   ├── models/                  # Test database and API models
│   │   ├── __init__.py
│   │   ├── test_user_model.py
│   │   ├── test_product_model.py
│   │   ├── test_shop_model.py
│   │   └── test_stock_model.py
│   ├── utils/                   # Test utility functions
│   │   ├── __init__.py
│   │   ├── test_data_processing.py
│   │   ├── test_csv_processor.py
│   │   └── test_database_utils.py
│   ├── services/                # Test business logic
│   │   ├── __init__.py
│   │   ├── test_auth_service.py
│   │   ├── test_product_service.py
│   │   └── test_marketplace_service.py
│   └── middleware/              # Test middleware components
│       ├── __init__.py
│       ├── test_auth_middleware.py
│       ├── test_rate_limiter.py
│       └── test_error_handler.py
├── integration/                 # Integration tests (multiple components)
│   ├── __init__.py
│   ├── conftest.py             # Integration test fixtures
│   ├── api/                    # API endpoint tests
│   │   ├── __init__.py
│   │   ├── conftest.py         # API test fixtures (test client, etc.)
│   │   ├── v1/
│   │   │   ├── __init__.py
│   │   │   ├── test_auth_endpoints.py
│   │   │   ├── test_product_endpoints.py
│   │   │   ├── test_shop_endpoints.py
│   │   │   ├── test_stock_endpoints.py
│   │   │   ├── test_marketplace_endpoints.py
│   │   │   ├── test_admin_endpoints.py
│   │   │   └── test_stats_endpoints.py
│   │   └── test_api_main.py    # Test API router setup
│   ├── database/               # Database integration tests
│   │   ├── __init__.py
│   │   ├── test_crud_operations.py
│   │   ├── test_relationships.py
│   │   └── test_migrations.py
│   └── workflows/              # End-to-end workflow tests
│       ├── __init__.py
│       ├── test_product_import_workflow.py
│       ├── test_shop_setup_workflow.py
│       └── test_stock_management_workflow.py
├── e2e/                        # End-to-end tests (full application)
│   ├── __init__.py
│   ├── conftest.py
│   ├── test_user_registration_flow.py
│   ├── test_marketplace_import_flow.py
│   └── test_complete_shop_setup.py
├── performance/                # Performance and load tests
│   ├── __init__.py
│   ├── test_api_performance.py
│   ├── test_database_performance.py
│   └── test_import_performance.py
└── test_data/                  # Test data files
    ├── csv/
    │   ├── valid_products.csv
    │   ├── invalid_products.csv
    │   └── large_dataset.csv
    ├── json/
    │   ├── sample_product.json
    │   └── marketplace_response.json
    └── fixtures/
        ├── test_users.json
        └── test_products.json

File Organization Principles

1. Test Types Separation

• Unit Tests: Fast, isolated tests for individual functions/classes
• Integration Tests: Tests that involve multiple components working together
• E2E Tests: Full application flow tests
• Performance Tests: Load and performance testing

2. Fixture Organization

# tests/fixtures/auth_fixtures.py
import pytest
from models.database import User
from utils.auth import create_access_token

@pytest.fixture
def test_user_data():
    return {
        "email": "test@example.com",
        "username": "testuser",
        "password": "testpass123"
    }

@pytest.fixture
def authenticated_user(db_session, test_user_data):
    user = User(**test_user_data)
    db_session.add(user)
    db_session.commit()
    return user

@pytest.fixture
def auth_headers(authenticated_user):
    token = create_access_token(data={"sub": authenticated_user.username})
    return {"Authorization": f"Bearer {token}"}

3. Conftest.py Structure

# tests/conftest.py (Global fixtures)
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

from main import app
from app.core.database import get_db, Base

# Database fixtures
@pytest.fixture(scope="session")
def test_engine():
    engine = create_engine("sqlite:///test.db")
    Base.metadata.create_all(bind=engine)
    yield engine
    Base.metadata.drop_all(bind=engine)

@pytest.fixture
def db_session(test_engine):
    TestingSessionLocal = sessionmaker(bind=test_engine)
    session = TestingSessionLocal()
    yield session
    session.close()

@pytest.fixture
def client(db_session):
    def override_get_db():
        yield db_session
    
    app.dependency_overrides[get_db] = override_get_db
    yield TestClient(app)
    app.dependency_overrides.clear()

# tests/integration/api/conftest.py (API-specific fixtures)
import pytest

@pytest.fixture
def api_client(client):
    """Pre-configured API client for integration tests"""
    return client

@pytest.fixture
def admin_client(client, admin_auth_headers):
    """API client with admin authentication"""
    client.headers.update(admin_auth_headers)
    return client

Test Naming Conventions

File Naming

• test_*.py for all test files
• Mirror your app structure: test_product_endpoints.py for api/v1/products.py
• Use descriptive names: test_marketplace_import_workflow.py

Test Function Naming

# Good test naming patterns
def test_create_product_with_valid_data_returns_201():
    pass

def test_create_product_without_title_returns_422():
    pass

def test_get_product_by_id_returns_product_data():
    pass

def test_get_nonexistent_product_returns_404():
    pass

def test_update_product_stock_updates_quantity():
    pass

def test_marketplace_import_with_invalid_csv_fails_gracefully():
    pass

Running Tests by Category

Pytest Configuration (pytest.ini)

[tool:pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts = 
    -v
    --tb=short
    --strict-markers
markers =
    unit: Unit tests
    integration: Integration tests  
    e2e: End-to-end tests
    performance: Performance tests
    slow: Slow running tests
    database: Tests that require database
    external: Tests that require external services

Running Specific Test Categories

# Run only unit tests (fast)
pytest tests/unit -m unit

# Run only integration tests
pytest tests/integration -m integration

# Run all tests except slow ones
pytest -m "not slow"

# Run only database tests
pytest -m database

# Run tests for specific domain
pytest tests/unit/models/ tests/integration/api/v1/test_product_endpoints.py

# Run with coverage
pytest --cov=app --cov-report=html

Benefits of This Structure

1. Faster Development: Developers can run relevant test subsets
2. Clear Separation: Easy to understand what each test covers
3. Parallel Execution: Can run different test types in parallel
4. Maintenance: Easier to maintain and update tests
5. CI/CD Pipeline: Can have different pipeline stages for different test types

Migration Strategy

Phase 1: Create Structure

1. Create new directory structure
2. Move global fixtures to fixtures/ directory
3. Update conftest.py files

Phase 2: Move Unit Tests

1. Start with utility and model tests
2. Move to tests/unit/ with appropriate subdirectories
3. Update imports and fixtures

Phase 3: Move Integration Tests

1. Move API endpoint tests to tests/integration/api/
2. Create database integration tests
3. Add workflow tests

Phase 4: Add Missing Coverage

1. Add performance tests if needed
2. Create E2E tests for critical flows
3. Add proper test markers

Example Test File Structure

Unit Test Example

# tests/unit/models/test_product_model.py
import pytest
from models.database import Product

class TestProductModel:
    def test_product_creation_with_valid_data(self, db_session):
        product = Product(
            product_id="TEST123",
            title="Test Product",
            price="99.99"
        )
        db_session.add(product)
        db_session.commit()
        
        assert product.id is not None
        assert product.product_id == "TEST123"
        assert product.title == "Test Product"

    def test_product_gtin_relationship_with_stock(self, db_session, test_product, test_stock):
        # Test the GTIN-based relationship
        assert test_product.gtin in [stock.gtin for stock in test_product.stock_entries]

Integration Test Example

# tests/integration/api/v1/test_product_endpoints.py
import pytest

class TestProductEndpoints:
    def test_create_product_endpoint(self, client, auth_headers, valid_product_data):
        response = client.post(
            "/api/v1/products/",
            json=valid_product_data,
            headers=auth_headers
        )
        assert response.status_code == 201
        assert response.json()["product_id"] == valid_product_data["product_id"]

    def test_get_products_with_pagination(self, client, auth_headers, multiple_products):
        response = client.get(
            "/api/v1/products/?skip=0&limit=10",
            headers=auth_headers
        )
        assert response.status_code == 200
        data = response.json()
        assert "products" in data
        assert "total" in data
        assert len(data["products"]) <= 10

This structure will scale beautifully as your application grows and makes it much easier for your team to maintain and extend the test suite!