orion/docs/development/performance-rules.md

Performance Rules Reference

This document provides a comprehensive reference for all performance rules enforced by the scripts/validate/validate_performance.py validator.

Overview

The performance validator identifies potential performance issues and enforces best practices for efficient code across the codebase. Rules are organized by category and severity level.

Version: 1.0 Total Rules: 70 Configuration Directory: .performance-rules/

Running the Validator

Using Python Directly

# Check all files
python scripts/validate/validate_performance.py

# Verbose output
python scripts/validate/validate_performance.py -v

# Errors only
python scripts/validate/validate_performance.py --errors-only

# JSON output (for CI/CD)
python scripts/validate/validate_performance.py --json

Using the Unified Validator

# Run performance checks only
python scripts/validate/validate_all.py --performance

# Run all validators
python scripts/validate/validate_all.py

Severity Levels

Severity	Description	Exit Code	Action Required
Error	Critical performance issue	1	Must fix
Warning	Performance concern	0	Should fix
Info	Performance suggestion	0	Consider implementing

---

Database Performance Rules (PERF-001 to PERF-015)

PERF-001: N+1 Query Detection

Severity: Warning

Accessing relationships in loops causes N+1 queries. For each item in a list, a separate query is executed.

# Bad - N+1 queries
orders = db.query(Order).all()
for order in orders:
    customer_name = order.customer.name  # N+1 query!

# Good - Eager loading
orders = db.query(Order).options(
    joinedload(Order.customer)
).all()
for order in orders:
    customer_name = order.customer.name  # Already loaded

PERF-002: Eager Loading for Known Relationships

Severity: Info

When you always need related data, use eager loading to reduce database round trips.

PERF-003: Query Result Limiting

Severity: Warning

All list queries should have pagination or limits. Unbounded queries can cause memory issues and slow responses.

# Bad
all_products = db.query(Product).all()

# Good
products = db.query(Product).limit(100).all()
# Or with pagination
products = db.query(Product).offset(skip).limit(limit).all()

PERF-004: Index Usage for Filtered Columns

Severity: Info

Columns frequently used in WHERE clauses should have indexes:

• Foreign keys (store_id, customer_id)
• Status fields
• Date fields used for filtering
• Boolean flags used for filtering

PERF-005: Select Only Needed Columns

Severity: Info

For large tables, select only the columns you need. Use .with_entities() or load_only() to reduce data transfer.

# Good - Only load id and name columns
products = db.query(Product).options(
    load_only(Product.id, Product.name)
).all()

PERF-006: Bulk Operations for Multiple Records

Severity: Warning

Use bulk operations instead of individual operations in loops.

# Bad
for item in items:
    product = Product(**item)
    db.add(product)

# Good
products = [Product(**item) for item in items]
db.add_all(products)

PERF-007: Connection Pool Configuration

Severity: Info

Configure database connection pool for optimal performance:

• pool_size: Number of persistent connections
• max_overflow: Additional connections allowed
• pool_pre_ping: Check connection health
• pool_recycle: Recycle connections periodically

PERF-008: Use EXISTS for Existence Checks

Severity: Info

Use EXISTS or .first() is not None instead of count() > 0. EXISTS stops at first match, count() scans all matches.

# Bad
if db.query(Order).filter_by(customer_id=id).count() > 0:
    ...

# Good
exists_query = db.query(exists().where(Order.customer_id == id))
if db.scalar(exists_query):
    ...

PERF-009: Batch Updates Instead of Loops

Severity: Warning

Use .update() with filters instead of updating in a loop. One UPDATE statement is faster than N individual updates.

# Bad
for product in products:
    product.is_active = False
    db.add(product)

# Good
db.query(Product).filter(
    Product.id.in_(product_ids)
).update({"is_active": False}, synchronize_session=False)

PERF-010: Avoid SELECT * Patterns

Severity: Info

When you only need specific columns, don't load entire rows. This reduces memory usage and network transfer.

PERF-011: Use Appropriate Join Strategies

Severity: Info

Choose the right join strategy:

• joinedload: Few related items, always needed
• selectinload: Many related items, always needed
• subqueryload: Complex queries, many related items
• lazyload: Rarely accessed relationships

PERF-012: Transaction Scope Optimization

Severity: Warning

Keep transactions short and focused:

• Don't hold transactions during I/O
• Commit after bulk operations
• Use read-only transactions when possible

PERF-013: Query Result Caching

Severity: Info

Consider caching for frequently accessed, rarely changed data like configuration tables and reference data.

PERF-014: Composite Indexes for Multi-Column Filters

Severity: Info

Queries filtering on multiple columns benefit from composite indexes. Order columns by selectivity (most selective first).

PERF-015: Avoid Correlated Subqueries

Severity: Info

Correlated subqueries execute once per row. Use JOINs or window functions instead when possible.

---

Caching Performance Rules (PERF-016 to PERF-025)

PERF-016: Cache Expensive Computations

Severity: Info

Computationally expensive operations should be cached: complex aggregations, external API results, template rendering, data transformations.

PERF-017: Cache Key Includes Tenant Context

Severity: Warning

Multi-tenant cache keys must include store_id. Otherwise, cached data may leak between tenants.

# Bad - Cache key missing tenant context
@cache.memoize()
def get_products():
    return db.query(Product).all()

# Good - Cache key includes store_id
@cache.memoize()
def get_products(store_id: int):
    return db.query(Product).filter_by(store_id=store_id).all()

PERF-018: Cache TTL Configuration

Severity: Info

Cache entries should have appropriate TTL:

• Short TTL (1-5 min): Frequently changing data
• Medium TTL (5-60 min): Semi-static data
• Long TTL (1+ hour): Reference data

PERF-019: Cache Invalidation Strategy

Severity: Warning

Define cache invalidation strategy: time-based (TTL), event-based (on data change), or manual (admin action).

PERF-020: Response Caching Headers

Severity: Info

API responses can use HTTP caching headers: Cache-Control for browser/CDN caching, ETag for conditional requests.

PERF-021 to PERF-025: Additional Caching Rules

Severity: Info

Query result caching, session-level caching, distributed cache for scalability, cache warming strategy, and cache hit rate monitoring.

---

API Performance Rules (PERF-026 to PERF-035)

PERF-026: Pagination Required for List Endpoints

Severity: Error

All list endpoints must support pagination. Unbounded lists cause memory exhaustion and slow response times.

# Bad
@router.get("/products")
def list_products(db: Session):
    return db.query(Product).all()

# Good
@router.get("/products")
def list_products(
    skip: int = 0,
    limit: int = Query(default=20, le=100),
    db: Session = Depends(get_db)
):
    return db.query(Product).offset(skip).limit(limit).all()

PERF-027: Reasonable Default Page Sizes

Severity: Warning

Default page sizes should be reasonable:

• Default: 20-50 items
• Maximum: 100-200 items

# Bad
limit: int = Query(default=500, le=10000)

# Good
limit: int = Query(default=20, ge=1, le=100)

PERF-028: Response Compression

Severity: Info

Enable response compression for large responses with GZip or Brotli.

PERF-029: Efficient Serialization

Severity: Info

Use Pydantic's response_model for efficient serialization. Avoid manual dict conversion.

PERF-030: Avoid Redundant Queries in Response

Severity: Warning

Don't trigger lazy-loaded relationships during serialization. Use eager loading or carefully control serialization.

PERF-031: Streaming for Large Responses

Severity: Info

Use streaming responses for large data: file downloads, large exports (CSV, JSON), real-time data feeds.

PERF-032 to PERF-035: Additional API Rules

Severity: Info/Warning

Conditional requests support, field selection support, avoid deep nesting in responses, and endpoint response time monitoring.

---

Async & Concurrency Rules (PERF-036 to PERF-045)

PERF-036: Async for I/O Operations

Severity: Info

Use async for I/O-bound operations: database queries, external API calls, file operations.

PERF-037: Parallel Independent Operations

Severity: Warning

Multiple independent async operations should run in parallel. Use asyncio.gather() instead of sequential awaits.

# Bad - Sequential awaits
user = await get_user(user_id)
orders = await get_orders(user_id)
preferences = await get_preferences(user_id)

# Good - Parallel execution
user, orders, preferences = await asyncio.gather(
    get_user(user_id),
    get_orders(user_id),
    get_preferences(user_id)
)

PERF-038: Background Tasks for Slow Operations

Severity: Warning

Operations taking > 500ms should run in background: email sending, report generation, external API syncs, file processing.

PERF-039: Connection Pooling for HTTP Clients

Severity: Warning

HTTP clients should reuse connections. Create client once, not per request.

# Bad
def fetch_data(url):
    response = requests.get(url)  # New connection each time

# Good
async with httpx.AsyncClient() as client:
    response = await client.get(url)

PERF-040: Timeout Configuration

Severity: Error

All external calls must have timeouts. Without timeouts, requests can hang indefinitely.

# Bad
response = requests.get(url)

# Good
response = requests.get(url, timeout=30)

PERF-041 to PERF-045: Additional Async Rules

Severity: Info

Connection pool limits, retry with backoff, circuit breaker pattern, task queues for heavy processing, and worker pool sizing.

---

Memory Management Rules (PERF-046 to PERF-055)

PERF-046: Generators for Large Datasets

Severity: Warning

Use generators/iterators for processing large datasets. Avoids loading everything into memory at once.

# Bad - Loads all into memory
products = db.query(Product).all()
for product in products:
    process(product)

# Good - Stream processing
for product in db.query(Product).yield_per(100):
    process(product)

PERF-047: Stream Large File Uploads

Severity: Warning

Large files should be streamed to disk, not held in memory.

# Bad - Entire file in memory
content = await file.read()
with open(path, 'wb') as f:
    f.write(content)

# Good - Streaming
with open(path, 'wb') as f:
    while chunk := await file.read(8192):
        f.write(chunk)

PERF-048: Chunked Processing for Imports

Severity: Warning

Bulk imports should process in chunks: read in batches, commit in batches, report progress periodically.

PERF-049: Context Managers for Resources

Severity: Error

Use context managers for file operations. Ensures resources are properly released.

# Bad
f = open('file.txt')
content = f.read()
f.close()  # May not run if exception

# Good
with open('file.txt') as f:
    content = f.read()

PERF-050 to PERF-055: Additional Memory Rules

Severity: Info

Limit in-memory collections, string concatenation efficiency, efficient data structures, object pooling, weak references for caches, and slots for frequently instantiated classes.

---

Frontend Performance Rules (PERF-056 to PERF-070)

PERF-056: Debounce Search Inputs

Severity: Warning

Search inputs should debounce API calls. Recommended: 300-500ms delay.

// Bad
<input @input="searchProducts($event.target.value)">

// Good
<input @input="debouncedSearch($event.target.value)">
// With: debouncedSearch = debounce(searchProducts, 300)

PERF-057: Lazy Load Off-Screen Content

Severity: Info

Defer loading of off-screen content: modals, inactive tabs, below-the-fold content, images.

PERF-058: Image Optimization

Severity: Warning

Images should be optimized:

• Use appropriate formats (WebP, AVIF)
• Serve responsive sizes
• Lazy load off-screen images
• Use CDN for static assets

<img src="image.webp" loading="lazy" alt="Product">

PERF-059: Minimize Alpine.js Watchers

Severity: Info

Excessive $watch calls impact performance. Use computed properties or event handlers instead.

PERF-060: Virtual Scrolling for Long Lists

Severity: Info

Lists with 100+ items should use virtual scrolling. Only render visible items in the viewport.

PERF-061: Minimize Bundle Size

Severity: Info

Reduce JavaScript bundle size: import only needed modules, use tree-shaking, split code by route.

PERF-062: Reasonable Polling Intervals

Severity: Warning

Polling should be >= 10 seconds for non-critical updates.

// Bad
setInterval(fetchUpdates, 1000);  // Every second

// Good
setInterval(fetchUpdates, 30000);  // Every 30 seconds

PERF-063 to PERF-070: Additional Frontend Rules

Severity: Info/Warning

CSS containment, avoid layout thrashing, CSS animations over JavaScript, preload critical resources, defer non-critical JavaScript, minimize DOM nodes, efficient event handlers, and cache DOM queries.

---

Configuration

All rules are defined in .performance-rules/ directory:

.performance-rules/
├── _main.yaml     # Main configuration
├── database.yaml  # PERF-001 to PERF-015
├── caching.yaml   # PERF-016 to PERF-025
├── api.yaml       # PERF-026 to PERF-035
├── async.yaml     # PERF-036 to PERF-045
├── memory.yaml    # PERF-046 to PERF-055
└── frontend.yaml  # PERF-056 to PERF-070

Suppressing Rules

Use noqa comments to suppress specific rules:

# noqa: PERF-003 - This is intentionally unbounded for admin export
products = db.query(Product).all()

Related Documentation

• Architecture Rules
• Security Rules
• Code Quality Guide
• Contributing Guide

---

Summary Statistics

Category	Rules	Errors	Warnings	Info
Database	15	0	5	10
Caching	10	0	2	8
API	10	1	3	6
Async & Concurrency	10	1	4	5
Memory Management	10	1	4	5
Frontend	15	0	4	11
Total	70	3	22	45

---

Last Updated: 2025-12-21 Version: 1.0