orion/docs/testing/running_tests.md

# Running Tests

This guide covers everything you need to know about running tests in the Letzshop Import project.

## Prerequisites

Ensure you have the test dependencies installed:

```bash
pip install -r tests/requirements-test.txt
```

Required packages:
- `pytest>=7.4.0` - Test framework
- `pytest-cov>=4.1.0` - Coverage reporting  
- `pytest-asyncio>=0.21.0` - Async test support
- `pytest-mock>=3.11.0` - Mocking utilities
- `httpx>=0.24.0` - HTTP client for API tests
- `faker>=19.0.0` - Test data generation

## Basic Test Commands

### Run All Tests
```bash
# Run the entire test suite
pytest

# Run with verbose output
pytest -v

# Run with very verbose output (show individual test results)
pytest -vv
```

### Run by Test Type
```bash
# Run only unit tests (fast)
pytest -m unit

# Run only integration tests  
pytest -m integration

# Run unit and integration tests
pytest -m "unit or integration"

# Run everything except slow tests
pytest -m "not slow"

# Run performance tests only
pytest -m performance
```

### Run by Directory
```bash
# Run all unit tests
pytest tests/unit/

# Run all integration tests
pytest tests/integration/

# Run API endpoint tests
pytest tests/integration/api/

# Run service layer tests
pytest tests/unit/services/
```

### Run Specific Files
```bash
# Run specific test file
pytest tests/unit/services/test_product_service.py

# Run multiple specific files
pytest tests/unit/services/test_product_service.py tests/unit/utils/test_data_processing.py
```

### Run Specific Tests
```bash
# Run specific test class
pytest tests/unit/services/test_product_service.py::TestProductService

# Run specific test method
pytest tests/unit/services/test_product_service.py::TestProductService::test_create_product_success

# Run tests matching pattern
pytest -k "product and create"
pytest -k "test_create_product"
```

## Advanced Test Options

### Coverage Reporting
```bash
# Run with coverage report
pytest --cov=app

# Coverage with missing lines
pytest --cov=app --cov-report=term-missing

# Generate HTML coverage report
pytest --cov=app --cov-report=html

# Coverage for specific modules
pytest --cov=app.services --cov=app.api

# Fail if coverage below threshold
pytest --cov=app --cov-fail-under=80
```

### Output and Debugging
```bash
# Show local variables on failure
pytest --tb=short --showlocals

# Stop on first failure
pytest -x

# Stop after N failures
pytest --maxfail=3

# Show print statements
pytest -s

# Show warnings
pytest -W error

# Capture output (default)
pytest --capture=sys
```

### Test Selection and Filtering
```bash
# Run only failed tests from last run
pytest --lf

# Run failed tests first, then continue
pytest --ff  

# Run tests that match keyword expression
pytest -k "user and not admin"

# Run tests modified since last commit
pytest --testmon

# Collect tests without running
pytest --collect-only
```

### Performance and Parallel Execution
```bash
# Show 10 slowest tests
pytest --durations=10

# Show all test durations
pytest --durations=0

# Run tests in parallel (requires pytest-xdist)
pytest -n auto
pytest -n 4  # Use 4 workers
```

## Test Environment Setup

### Database Tests
```bash
# Run database tests (uses test database)
pytest -m database

# Run with test database reset
pytest --tb=short tests/integration/database/
```

### API Tests
```bash
# Run API endpoint tests
pytest -m api

# Run with test client
pytest tests/integration/api/ -v
```

### Authentication Tests
```bash
# Run auth-related tests
pytest -m auth

# Run with user fixtures
pytest tests/integration/security/ -v
```

## Configuration Options

### pytest.ini Settings
Our `pytest.ini` is configured with:

```ini
[pytest]
# Test discovery
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*

# Default options
addopts = 
    -v
    --tb=short
    --strict-markers
    --color=yes
    --durations=10
    --cov=app
    --cov-report=term-missing
    --cov-report=html:htmlcov
    --cov-fail-under=80

# Custom markers
markers =
    unit: Unit tests
    integration: Integration tests
    # ... other markers
```

### Environment Variables
```bash
# Set test environment
export TESTING=true

# Database URL for tests (uses in-memory SQLite by default)
export TEST_DATABASE_URL="sqlite:///:memory:"

# Disable external API calls in tests
export MOCK_EXTERNAL_APIS=true
```

## Common Test Scenarios

### Development Workflow
```bash
# Quick smoke test (unit tests only)
pytest -m unit --maxfail=1

# Full test run before commit
pytest -m "unit or integration"

# Pre-push comprehensive test
pytest --cov=app --cov-fail-under=80
```

### Debugging Failed Tests
```bash
# Run failed test with detailed output
pytest --lf -vv --tb=long --showlocals

# Drop into debugger on failure (requires ipdb)
pytest --pdb

# Run specific failing test in isolation
pytest tests/path/to/test.py::TestClass::test_method -vv -s
```

### Performance Testing
```bash
# Run performance tests only
pytest -m performance --durations=0

# Run with performance profiling
pytest -m performance --profile

# Load testing specific endpoints
pytest tests/performance/test_api_performance.py -v
```

### CI/CD Pipeline Tests
```bash
# Minimal test run (fast feedback)
pytest -m "unit and not slow" --maxfail=5

# Full CI test run
pytest --cov=app --cov-report=xml --junitxml=test-results.xml

# Security and integration tests
pytest -m "security or integration" --tb=short
```

## Test Reports and Output

### Coverage Reports
```bash
# Terminal coverage report
pytest --cov=app --cov-report=term

# HTML coverage report (opens in browser)
pytest --cov=app --cov-report=html
open htmlcov/index.html

# XML coverage for CI
pytest --cov=app --cov-report=xml
```

### JUnit XML Reports
```bash
# Generate JUnit XML (for CI integration)
pytest --junitxml=test-results.xml

# With coverage and JUnit
pytest --cov=app --cov-report=xml --junitxml=test-results.xml
```

## Troubleshooting

### Common Issues

#### Import Errors
```bash
# If getting import errors, ensure PYTHONPATH is set
PYTHONPATH=. pytest

# Or install in development mode
pip install -e .
```

#### Database Connection Issues
```bash
# Check if test database is accessible
python -c "from tests.conftest import engine; engine.connect()"

# Run with in-memory database
pytest --override-db-url="sqlite:///:memory:"
```

#### Fixture Not Found
```bash
# Ensure conftest.py is in the right location
ls tests/conftest.py

# Check fixture imports
pytest --fixtures tests/unit/
```

#### Permission Issues
```bash
# Fix test file permissions
chmod +x tests/**/*.py

# Clear pytest cache
rm -rf .pytest_cache/
```

### Performance Issues
```bash
# Identify slow tests
pytest --durations=10 --durations-min=1.0

# Profile test execution
pytest --profile-svg

# Run subset of tests for faster feedback
pytest -m "unit and not slow"
```

## Integration with IDEs

### VS Code
Add to `.vscode/settings.json`:
```json
{
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "tests",
        "-v"
    ],
    "python.testing.cwd": "${workspaceFolder}"
}
```

### PyCharm
1. Go to Settings → Tools → Python Integrated Tools
2. Set Testing → Default test runner to "pytest"
3. Set pytest options: `-v --tb=short`

## Best Practices

### During Development
1. **Run unit tests frequently** - Fast feedback loop
2. **Run integration tests before commits** - Catch interaction issues  
3. **Check coverage regularly** - Ensure good test coverage
4. **Use descriptive test names** - Easy to understand failures

### Before Code Review
1. **Run full test suite** - `pytest`
2. **Check coverage meets threshold** - `pytest --cov-fail-under=80`
3. **Ensure no warnings** - `pytest -W error`
4. **Test with fresh environment** - New terminal/clean cache

### In CI/CD
1. **Fail fast on unit tests** - `pytest -m unit --maxfail=1`
2. **Generate reports** - Coverage and JUnit XML
3. **Run performance tests on schedule** - Not every commit
4. **Archive test results** - For debugging and trends

---

Need help with a specific testing scenario? Check our [Testing FAQ](testing-faq.md) or open a GitHub issue!