orion/docs/development/performance-rules.md

# Performance Rules Reference

This document provides a comprehensive reference for all performance rules enforced by the `scripts/validate/validate_performance.py` validator.

## Overview

The performance validator identifies potential performance issues and enforces best practices for efficient code across the codebase. Rules are organized by category and severity level.

**Version:** 1.0
**Total Rules:** 70
**Configuration Directory:** `.performance-rules/`

## Running the Validator

### Using Python Directly

```bash
# Check all files
python scripts/validate/validate_performance.py

# Verbose output
python scripts/validate/validate_performance.py -v

# Errors only
python scripts/validate/validate_performance.py --errors-only

# JSON output (for CI/CD)
python scripts/validate/validate_performance.py --json
```

### Using the Unified Validator

```bash
# Run performance checks only
python scripts/validate/validate_all.py --performance

# Run all validators
python scripts/validate/validate_all.py
```

## Severity Levels

| Severity | Description | Exit Code | Action Required |
|----------|-------------|-----------|-----------------|
| **Error** | Critical performance issue | 1 | Must fix |
| **Warning** | Performance concern | 0 | Should fix |
| **Info** | Performance suggestion | 0 | Consider implementing |

---

## Database Performance Rules (PERF-001 to PERF-015)

### PERF-001: N+1 Query Detection
**Severity:** Warning

Accessing relationships in loops causes N+1 queries. For each item in a list, a separate query is executed.

```python
# Bad - N+1 queries
orders = db.query(Order).all()
for order in orders:
    customer_name = order.customer.name  # N+1 query!

# Good - Eager loading
orders = db.query(Order).options(
    joinedload(Order.customer)
).all()
for order in orders:
    customer_name = order.customer.name  # Already loaded
```

### PERF-002: Eager Loading for Known Relationships
**Severity:** Info

When you always need related data, use eager loading to reduce database round trips.

### PERF-003: Query Result Limiting
**Severity:** Warning

All list queries should have pagination or limits. Unbounded queries can cause memory issues and slow responses.

```python
# Bad
all_products = db.query(Product).all()

# Good
products = db.query(Product).limit(100).all()
# Or with pagination
products = db.query(Product).offset(skip).limit(limit).all()
```

### PERF-004: Index Usage for Filtered Columns
**Severity:** Info

Columns frequently used in WHERE clauses should have indexes:
- Foreign keys (store_id, customer_id)
- Status fields
- Date fields used for filtering
- Boolean flags used for filtering

### PERF-005: Select Only Needed Columns
**Severity:** Info

For large tables, select only the columns you need. Use `.with_entities()` or `load_only()` to reduce data transfer.

```python
# Good - Only load id and name columns
products = db.query(Product).options(
    load_only(Product.id, Product.name)
).all()
```

### PERF-006: Bulk Operations for Multiple Records
**Severity:** Warning

Use bulk operations instead of individual operations in loops.

```python
# Bad
for item in items:
    product = Product(**item)
    db.add(product)

# Good
products = [Product(**item) for item in items]
db.add_all(products)
```

### PERF-007: Connection Pool Configuration
**Severity:** Info

Configure database connection pool for optimal performance:
- `pool_size`: Number of persistent connections
- `max_overflow`: Additional connections allowed
- `pool_pre_ping`: Check connection health
- `pool_recycle`: Recycle connections periodically

### PERF-008: Use EXISTS for Existence Checks
**Severity:** Info

Use EXISTS or `.first() is not None` instead of `count() > 0`. EXISTS stops at first match, count() scans all matches.

```python
# Bad
if db.query(Order).filter_by(customer_id=id).count() > 0:
    ...

# Good
exists_query = db.query(exists().where(Order.customer_id == id))
if db.scalar(exists_query):
    ...
```

### PERF-009: Batch Updates Instead of Loops
**Severity:** Warning

Use `.update()` with filters instead of updating in a loop. One UPDATE statement is faster than N individual updates.

```python
# Bad
for product in products:
    product.is_active = False
    db.add(product)

# Good
db.query(Product).filter(
    Product.id.in_(product_ids)
).update({"is_active": False}, synchronize_session=False)
```

### PERF-010: Avoid SELECT * Patterns
**Severity:** Info

When you only need specific columns, don't load entire rows. This reduces memory usage and network transfer.

### PERF-011: Use Appropriate Join Strategies
**Severity:** Info

Choose the right join strategy:
- `joinedload`: Few related items, always needed
- `selectinload`: Many related items, always needed
- `subqueryload`: Complex queries, many related items
- `lazyload`: Rarely accessed relationships

### PERF-012: Transaction Scope Optimization
**Severity:** Warning

Keep transactions short and focused:
- Don't hold transactions during I/O
- Commit after bulk operations
- Use read-only transactions when possible

### PERF-013: Query Result Caching
**Severity:** Info

Consider caching for frequently accessed, rarely changed data like configuration tables and reference data.

### PERF-014: Composite Indexes for Multi-Column Filters
**Severity:** Info

Queries filtering on multiple columns benefit from composite indexes. Order columns by selectivity (most selective first).

### PERF-015: Avoid Correlated Subqueries
**Severity:** Info

Correlated subqueries execute once per row. Use JOINs or window functions instead when possible.

---

## Caching Performance Rules (PERF-016 to PERF-025)

### PERF-016: Cache Expensive Computations
**Severity:** Info

Computationally expensive operations should be cached: complex aggregations, external API results, template rendering, data transformations.

### PERF-017: Cache Key Includes Tenant Context
**Severity:** Warning

Multi-tenant cache keys must include store_id. Otherwise, cached data may leak between tenants.

```python
# Bad - Cache key missing tenant context
@cache.memoize()
def get_products():
    return db.query(Product).all()

# Good - Cache key includes store_id
@cache.memoize()
def get_products(store_id: int):
    return db.query(Product).filter_by(store_id=store_id).all()
```

### PERF-018: Cache TTL Configuration
**Severity:** Info

Cache entries should have appropriate TTL:
- Short TTL (1-5 min): Frequently changing data
- Medium TTL (5-60 min): Semi-static data
- Long TTL (1+ hour): Reference data

### PERF-019: Cache Invalidation Strategy
**Severity:** Warning

Define cache invalidation strategy: time-based (TTL), event-based (on data change), or manual (admin action).

### PERF-020: Response Caching Headers
**Severity:** Info

API responses can use HTTP caching headers: Cache-Control for browser/CDN caching, ETag for conditional requests.

### PERF-021 to PERF-025: Additional Caching Rules
**Severity:** Info

Query result caching, session-level caching, distributed cache for scalability, cache warming strategy, and cache hit rate monitoring.

---

## API Performance Rules (PERF-026 to PERF-035)

### PERF-026: Pagination Required for List Endpoints
**Severity:** Error

All list endpoints must support pagination. Unbounded lists cause memory exhaustion and slow response times.

```python
# Bad
@router.get("/products")
def list_products(db: Session):
    return db.query(Product).all()

# Good
@router.get("/products")
def list_products(
    skip: int = 0,
    limit: int = Query(default=20, le=100),
    db: Session = Depends(get_db)
):
    return db.query(Product).offset(skip).limit(limit).all()
```

### PERF-027: Reasonable Default Page Sizes
**Severity:** Warning

Default page sizes should be reasonable:
- Default: 20-50 items
- Maximum: 100-200 items

```python
# Bad
limit: int = Query(default=500, le=10000)

# Good
limit: int = Query(default=20, ge=1, le=100)
```

### PERF-028: Response Compression
**Severity:** Info

Enable response compression for large responses with GZip or Brotli.

### PERF-029: Efficient Serialization
**Severity:** Info

Use Pydantic's `response_model` for efficient serialization. Avoid manual dict conversion.

### PERF-030: Avoid Redundant Queries in Response
**Severity:** Warning

Don't trigger lazy-loaded relationships during serialization. Use eager loading or carefully control serialization.

### PERF-031: Streaming for Large Responses
**Severity:** Info

Use streaming responses for large data: file downloads, large exports (CSV, JSON), real-time data feeds.

### PERF-032 to PERF-035: Additional API Rules
**Severity:** Info/Warning

Conditional requests support, field selection support, avoid deep nesting in responses, and endpoint response time monitoring.

---

## Async & Concurrency Rules (PERF-036 to PERF-045)

### PERF-036: Async for I/O Operations
**Severity:** Info

Use async for I/O-bound operations: database queries, external API calls, file operations.

### PERF-037: Parallel Independent Operations
**Severity:** Warning

Multiple independent async operations should run in parallel. Use `asyncio.gather()` instead of sequential awaits.

```python
# Bad - Sequential awaits
user = await get_user(user_id)
orders = await get_orders(user_id)
preferences = await get_preferences(user_id)

# Good - Parallel execution
user, orders, preferences = await asyncio.gather(
    get_user(user_id),
    get_orders(user_id),
    get_preferences(user_id)
)
```

### PERF-038: Background Tasks for Slow Operations
**Severity:** Warning

Operations taking > 500ms should run in background: email sending, report generation, external API syncs, file processing.

### PERF-039: Connection Pooling for HTTP Clients
**Severity:** Warning

HTTP clients should reuse connections. Create client once, not per request.

```python
# Bad
def fetch_data(url):
    response = requests.get(url)  # New connection each time

# Good
async with httpx.AsyncClient() as client:
    response = await client.get(url)
```

### PERF-040: Timeout Configuration
**Severity:** Error

All external calls must have timeouts. Without timeouts, requests can hang indefinitely.

```python
# Bad
response = requests.get(url)

# Good
response = requests.get(url, timeout=30)
```

### PERF-041 to PERF-045: Additional Async Rules
**Severity:** Info

Connection pool limits, retry with backoff, circuit breaker pattern, task queues for heavy processing, and worker pool sizing.

---

## Memory Management Rules (PERF-046 to PERF-055)

### PERF-046: Generators for Large Datasets
**Severity:** Warning

Use generators/iterators for processing large datasets. Avoids loading everything into memory at once.

```python
# Bad - Loads all into memory
products = db.query(Product).all()
for product in products:
    process(product)

# Good - Stream processing
for product in db.query(Product).yield_per(100):
    process(product)
```

### PERF-047: Stream Large File Uploads
**Severity:** Warning

Large files should be streamed to disk, not held in memory.

```python
# Bad - Entire file in memory
content = await file.read()
with open(path, 'wb') as f:
    f.write(content)

# Good - Streaming
with open(path, 'wb') as f:
    while chunk := await file.read(8192):
        f.write(chunk)
```

### PERF-048: Chunked Processing for Imports
**Severity:** Warning

Bulk imports should process in chunks: read in batches, commit in batches, report progress periodically.

### PERF-049: Context Managers for Resources
**Severity:** Error

Use context managers for file operations. Ensures resources are properly released.

```python
# Bad
f = open('file.txt')
content = f.read()
f.close()  # May not run if exception

# Good
with open('file.txt') as f:
    content = f.read()
```

### PERF-050 to PERF-055: Additional Memory Rules
**Severity:** Info

Limit in-memory collections, string concatenation efficiency, efficient data structures, object pooling, weak references for caches, and slots for frequently instantiated classes.

---

## Frontend Performance Rules (PERF-056 to PERF-070)

### PERF-056: Debounce Search Inputs
**Severity:** Warning

Search inputs should debounce API calls. Recommended: 300-500ms delay.

```javascript
// Bad
<input @input="searchProducts($event.target.value)">

// Good
<input @input="debouncedSearch($event.target.value)">
// With: debouncedSearch = debounce(searchProducts, 300)
```

### PERF-057: Lazy Load Off-Screen Content
**Severity:** Info

Defer loading of off-screen content: modals, inactive tabs, below-the-fold content, images.

### PERF-058: Image Optimization
**Severity:** Warning

Images should be optimized:
- Use appropriate formats (WebP, AVIF)
- Serve responsive sizes
- Lazy load off-screen images
- Use CDN for static assets

```html
<img src="image.webp" loading="lazy" alt="Product">
```

### PERF-059: Minimize Alpine.js Watchers
**Severity:** Info

Excessive `$watch` calls impact performance. Use computed properties or event handlers instead.

### PERF-060: Virtual Scrolling for Long Lists
**Severity:** Info

Lists with 100+ items should use virtual scrolling. Only render visible items in the viewport.

### PERF-061: Minimize Bundle Size
**Severity:** Info

Reduce JavaScript bundle size: import only needed modules, use tree-shaking, split code by route.

### PERF-062: Reasonable Polling Intervals
**Severity:** Warning

Polling should be >= 10 seconds for non-critical updates.

```javascript
// Bad
setInterval(fetchUpdates, 1000);  // Every second

// Good
setInterval(fetchUpdates, 30000);  // Every 30 seconds
```

### PERF-063 to PERF-070: Additional Frontend Rules
**Severity:** Info/Warning

CSS containment, avoid layout thrashing, CSS animations over JavaScript, preload critical resources, defer non-critical JavaScript, minimize DOM nodes, efficient event handlers, and cache DOM queries.

---

## Configuration

All rules are defined in `.performance-rules/` directory:

```
.performance-rules/
├── _main.yaml     # Main configuration
├── database.yaml  # PERF-001 to PERF-015
├── caching.yaml   # PERF-016 to PERF-025
├── api.yaml       # PERF-026 to PERF-035
├── async.yaml     # PERF-036 to PERF-045
├── memory.yaml    # PERF-046 to PERF-055
└── frontend.yaml  # PERF-056 to PERF-070
```

## Suppressing Rules

Use noqa comments to suppress specific rules:

```python
# noqa: PERF-003 - This is intentionally unbounded for admin export
products = db.query(Product).all()
```

## Related Documentation

- [Architecture Rules](architecture-rules.md)
- [Security Rules](security-rules.md)
- [Code Quality Guide](code-quality.md)
- [Contributing Guide](contributing.md)

---

## Summary Statistics

| Category | Rules | Errors | Warnings | Info |
|----------|-------|--------|----------|------|
| Database | 15 | 0 | 5 | 10 |
| Caching | 10 | 0 | 2 | 8 |
| API | 10 | 1 | 3 | 6 |
| Async & Concurrency | 10 | 1 | 4 | 5 |
| Memory Management | 10 | 1 | 4 | 5 |
| Frontend | 15 | 0 | 4 | 11 |
| **Total** | **70** | **3** | **22** | **45** |

---

**Last Updated:** 2025-12-21
**Version:** 1.0