orion/docs/testing/test-structure.md

Test Structure

Overview

The integration test suite is organized to mirror the API code structure, providing clear mapping between code and tests for better maintainability, discoverability, and team collaboration.

Directory Structure

tests/integration/api/v1/
├── admin/                           # Admin API tests
│   ├── __init__.py
│   └── README.md
├── vendor/                          # Vendor API tests
│   ├── __init__.py
│   ├── README.md
│   ├── test_authentication.py      # Authentication tests
│   └── test_dashboard.py           # Dashboard tests
├── public/                          # Public API tests
│   ├── __init__.py
│   └── README.md
└── shared/                          # Shared/common tests
    ├── __init__.py
    └── README.md

Structure Mapping

The test structure directly mirrors the API code structure:

app/api/v1/admin/     →  tests/integration/api/v1/admin/
app/api/v1/vendor/    →  tests/integration/api/v1/vendor/
app/api/v1/platform/    →  tests/integration/api/v1/platform/
app/api/v1/shared/    →  tests/integration/api/v1/shared/

Benefits

1. Easy Discoverability

Finding tests is straightforward:

# Looking for vendor product tests?
ls tests/integration/api/v1/vendor/test_products.py

# Looking for admin user tests?
ls tests/integration/api/v1/admin/test_users.py

2. Clear Code-to-Test Mapping

Code change:    app/api/v1/vendor/products.py
Test location:  tests/integration/api/v1/vendor/test_products.py
                ↑ 1:1 mapping!

3. Faster Testing

Run tests for specific areas without complex filters:

# Test only vendor endpoints
pytest tests/integration/api/v1/vendor/ -v

# Test only admin endpoints
pytest tests/integration/api/v1/admin/ -v

4. Better Code Reviews

Changes are grouped logically:

Pull Request:
  Changed files:
    app/api/v1/vendor/products.py                      # Code
    tests/integration/api/v1/vendor/test_products.py  # Test
                                                       ↑ Easy to verify

5. Team Collaboration

Different teams can work in parallel with fewer conflicts:

Admin Team:   works in tests/integration/api/v1/admin/
Vendor Team:  works in tests/integration/api/v1/vendor/
Public Team:  works in tests/integration/api/v1/platform/

Running Tests

By Area

# All vendor tests
pytest tests/integration/api/v1/vendor/ -v

# All admin tests
pytest tests/integration/api/v1/admin/ -v

# All public tests
pytest tests/integration/api/v1/platform/ -v

# All shared tests
pytest tests/integration/api/v1/shared/ -v

Specific Test Files

# Run specific test file
pytest tests/integration/api/v1/vendor/test_authentication.py -v

# Run specific test class
pytest tests/integration/api/v1/vendor/test_authentication.py::TestVendorAPIAuthentication -v

# Run specific test
pytest tests/integration/api/v1/vendor/test_authentication.py::TestVendorAPIAuthentication::test_vendor_auth_me_success -v

With Coverage

# Vendor API coverage
pytest tests/integration/api/v1/vendor/ \
  --cov=app/api/v1/vendor \
  --cov-report=html

# View coverage report
open htmlcov/index.html

Using Markers

All tests use pytest markers for flexible filtering:

# All vendor tests
pytest -m vendor -v

# Vendor API tests only
pytest -m "vendor and api" -v

# Vendor authentication tests
pytest -m "vendor and auth" -v

Naming Conventions

Test Files

• Pattern: test_<endpoint_name>.py
• Examples:
  • test_products.py - Tests for products endpoints
  • test_authentication.py - Tests for authentication
  • test_dashboard.py - Tests for dashboard endpoints

Test Classes

• Pattern: Test<Area><Feature>API
• Examples:

  class TestVendorProductAPI:      # Vendor product endpoints
  class TestAdminUserAPI:          # Admin user endpoints
  class TestPublicCatalogAPI:      # Public catalog endpoints
  

Test Functions

• Pattern: test_<action>_<scenario>
• Examples:

  def test_list_products_success(self):
  def test_create_product_without_auth(self):
  def test_update_product_invalid_data(self):
  def test_delete_product_not_found(self):
  

Best Practices

1. Mirror API Structure

Always create test files that mirror the API code structure:

# API Code
app/api/v1/vendor/products.py

# Test File
tests/integration/api/v1/vendor/test_products.py

2. One Test File Per API Module

Keep tests focused:

• Each API module gets its own test file
• Easy to find and maintain
• Clear responsibility

3. Descriptive Test Names

Use clear, descriptive names:

# Good ✅
class TestVendorProductAPI:
    def test_list_products_success(self):
    def test_create_product_without_auth(self):
    def test_update_product_with_invalid_data(self):

# Avoid ❌
class TestProducts:
    def test_api(self):
    def test_endpoint(self):

4. Appropriate Markers

Mark tests appropriately:

@pytest.mark.integration
@pytest.mark.api
@pytest.mark.vendor
class TestVendorProductAPI:
    pass

5. Test Both Success and Failure

Cover all scenarios:

class TestVendorProductAPI:
    # Success cases
    def test_list_products_success(self):
    def test_create_product_success(self):

    # Failure cases
    def test_list_products_without_auth(self):
    def test_create_product_invalid_data(self):
    def test_create_product_as_admin_user(self):

Test Areas

Admin Tests (admin/)

Tests for admin-only endpoints at /api/v1/admin/*:

• User management
• System configuration
• Analytics and reporting
• Vendor approval workflows

Requirements:

• Admin user authentication
• Admin-specific permissions
• Cross-vendor data access

Vendor Tests (vendor/)

Tests for vendor endpoints at /api/v1/vendor/*:

• Product management
• Order processing
• Customer management
• Analytics and stats
• Team management

Requirements:

• Vendor user authentication
• Vendor context isolation
• VendorUser associations

See Vendor API Testing Guide for details.

Public Tests (public/)

Tests for public endpoints at /api/v1/platform/*:

• Product catalog browsing
• Public vendor profiles
• Search functionality

Requirements:

• No authentication required
• Rate limiting tests
• Public data only

Shared Tests (shared/)

Tests for shared/common functionality:

• Authentication endpoints
• Pagination utilities
• Filtering utilities
• Common error handling

Requirements:

• Work across all user types
• Consistent behavior
• Proper error handling

Migration Status

Completed ✅

Area	Files	Tests	Status
Vendor Authentication	vendor/test_authentication.py	30+	✅ Complete
Vendor Dashboard	vendor/test_dashboard.py	12	✅ Complete

Pending ⚠️

Legacy File	New Location	Priority
test_vendor_endpoints.py	Split into vendor/test_*.py	High
test_admin_endpoints.py	Split into admin/test_*.py	High
test_auth_endpoints.py	shared/test_auth.py	Medium
test_marketplace_*.py	vendor/test_marketplace_*.py	Medium
test_inventory_endpoints.py	vendor/test_inventory.py	Medium

Adding New Tests

When adding new integration tests:

1. Identify the API area (admin/vendor/public/shared)
2. Create test file in appropriate folder
3. Follow naming conventions
4. Add appropriate markers
5. Test both success and failure cases
6. Update coverage reports

Example: Adding Vendor Products Tests

# 1. Create test file
touch tests/integration/api/v1/vendor/test_products.py

# 2. Write tests following conventions
import pytest

@pytest.mark.integration
@pytest.mark.api
@pytest.mark.vendor
class TestVendorProductAPI:
    """Tests for vendor product management endpoints"""

    def test_list_products_success(self, client, vendor_user_headers):
        """Test listing products with authentication"""
        response = client.get(
            "/api/v1/vendor/products",
            headers=vendor_user_headers
        )
        assert response.status_code == 200
        data = response.json()
        assert "products" in data
        assert "total" in data

    def test_list_products_without_auth(self, client):
        """Test listing products requires authentication"""
        response = client.get("/api/v1/vendor/products")
        assert response.status_code == 401

    def test_create_product_success(self, client, vendor_user_headers):
        """Test creating product with valid data"""
        response = client.post(
            "/api/v1/vendor/products",
            headers=vendor_user_headers,
            json={
                "name": "Test Product",
                "price": 29.99,
                "description": "Test description"
            }
        )
        assert response.status_code == 201

# 3. Run tests
pytest tests/integration/api/v1/vendor/test_products.py -v

# 4. Check coverage
pytest tests/integration/api/v1/vendor/test_products.py \
  --cov=app/api/v1/vendor/products \
  --cov-report=html

Troubleshooting

Tests Not Discovered

Problem: pytest doesn't find tests

Solutions:

• Ensure __init__.py exists in test directory
• Check file naming: test_*.py or *_test.py
• Verify test functions start with test_
• Check pytest collection with: pytest --collect-only

Import Errors

Problem: Fixtures not found

Solutions:

• Check pytest_plugins in tests/conftest.py
• Use absolute imports: from tests.fixtures.auth_fixtures import ...
• Verify fixture files are in correct location

Coverage Not Working

Problem: Coverage report shows 0%

Solutions:

• Specify correct source path: --cov=app/api/v1/vendor
• Check test actually exercises the code
• Verify coverage config in pyproject.toml or .coveragerc

See Also

• Vendor API Testing Guide - Comprehensive vendor testing documentation
• Testing Guide - Overall testing strategy
• Test Maintenance - Maintaining test quality