Filters
-
+
-
+
No violations found
+
+
+
+
+
+
+
+
@@ -64,7 +80,7 @@
@@ -84,17 +100,29 @@
{% call table_wrapper() %}
- {{ table_header(['Rule', 'Severity', 'File', 'Line', 'Message', 'Status', 'Actions']) }}
+ {{ table_header(['Validator', 'Rule', 'Severity', 'File', 'Line', 'Message', 'Status', 'Actions']) }}
+
+
+
+
@@ -109,7 +137,8 @@
diff --git a/docs/development/performance-rules.md b/docs/development/performance-rules.md
new file mode 100644
index 00000000..f24ddad8
--- /dev/null
+++ b/docs/development/performance-rules.md
@@ -0,0 +1,570 @@
+# Performance Rules Reference
+
+This document provides a comprehensive reference for all performance rules enforced by the `scripts/validate_performance.py` validator.
+
+## Overview
+
+The performance validator identifies potential performance issues and enforces best practices for efficient code across the codebase. Rules are organized by category and severity level.
+
+**Version:** 1.0
+**Total Rules:** 70
+**Configuration Directory:** `.performance-rules/`
+
+## Running the Validator
+
+### Using Python Directly
+
+```bash
+# Check all files
+python scripts/validate_performance.py
+
+# Verbose output
+python scripts/validate_performance.py -v
+
+# Errors only
+python scripts/validate_performance.py --errors-only
+
+# JSON output (for CI/CD)
+python scripts/validate_performance.py --json
+```
+
+### Using the Unified Validator
+
+```bash
+# Run performance checks only
+python scripts/validate_all.py --performance
+
+# Run all validators
+python scripts/validate_all.py
+```
+
+## Severity Levels
+
+| Severity | Description | Exit Code | Action Required |
+|----------|-------------|-----------|-----------------|
+| **Error** | Critical performance issue | 1 | Must fix |
+| **Warning** | Performance concern | 0 | Should fix |
+| **Info** | Performance suggestion | 0 | Consider implementing |
+
+---
+
+## Database Performance Rules (PERF-001 to PERF-015)
+
+### PERF-001: N+1 Query Detection
+**Severity:** Warning
+
+Accessing relationships in loops causes N+1 queries. For each item in a list, a separate query is executed.
+
+```python
+# Bad - N+1 queries
+orders = db.query(Order).all()
+for order in orders:
+ customer_name = order.customer.name # N+1 query!
+
+# Good - Eager loading
+orders = db.query(Order).options(
+ joinedload(Order.customer)
+).all()
+for order in orders:
+ customer_name = order.customer.name # Already loaded
+```
+
+### PERF-002: Eager Loading for Known Relationships
+**Severity:** Info
+
+When you always need related data, use eager loading to reduce database round trips.
+
+### PERF-003: Query Result Limiting
+**Severity:** Warning
+
+All list queries should have pagination or limits. Unbounded queries can cause memory issues and slow responses.
+
+```python
+# Bad
+all_products = db.query(Product).all()
+
+# Good
+products = db.query(Product).limit(100).all()
+# Or with pagination
+products = db.query(Product).offset(skip).limit(limit).all()
+```
+
+### PERF-004: Index Usage for Filtered Columns
+**Severity:** Info
+
+Columns frequently used in WHERE clauses should have indexes:
+- Foreign keys (vendor_id, customer_id)
+- Status fields
+- Date fields used for filtering
+- Boolean flags used for filtering
+
+### PERF-005: Select Only Needed Columns
+**Severity:** Info
+
+For large tables, select only the columns you need. Use `.with_entities()` or `load_only()` to reduce data transfer.
+
+```python
+# Good - Only load id and name columns
+products = db.query(Product).options(
+ load_only(Product.id, Product.name)
+).all()
+```
+
+### PERF-006: Bulk Operations for Multiple Records
+**Severity:** Warning
+
+Use bulk operations instead of individual operations in loops.
+
+```python
+# Bad
+for item in items:
+ product = Product(**item)
+ db.add(product)
+
+# Good
+products = [Product(**item) for item in items]
+db.add_all(products)
+```
+
+### PERF-007: Connection Pool Configuration
+**Severity:** Info
+
+Configure database connection pool for optimal performance:
+- `pool_size`: Number of persistent connections
+- `max_overflow`: Additional connections allowed
+- `pool_pre_ping`: Check connection health
+- `pool_recycle`: Recycle connections periodically
+
+### PERF-008: Use EXISTS for Existence Checks
+**Severity:** Info
+
+Use EXISTS or `.first() is not None` instead of `count() > 0`. EXISTS stops at first match, count() scans all matches.
+
+```python
+# Bad
+if db.query(Order).filter_by(customer_id=id).count() > 0:
+ ...
+
+# Good
+exists_query = db.query(exists().where(Order.customer_id == id))
+if db.scalar(exists_query):
+ ...
+```
+
+### PERF-009: Batch Updates Instead of Loops
+**Severity:** Warning
+
+Use `.update()` with filters instead of updating in a loop. One UPDATE statement is faster than N individual updates.
+
+```python
+# Bad
+for product in products:
+ product.is_active = False
+ db.add(product)
+
+# Good
+db.query(Product).filter(
+ Product.id.in_(product_ids)
+).update({"is_active": False}, synchronize_session=False)
+```
+
+### PERF-010: Avoid SELECT * Patterns
+**Severity:** Info
+
+When you only need specific columns, don't load entire rows. This reduces memory usage and network transfer.
+
+### PERF-011: Use Appropriate Join Strategies
+**Severity:** Info
+
+Choose the right join strategy:
+- `joinedload`: Few related items, always needed
+- `selectinload`: Many related items, always needed
+- `subqueryload`: Complex queries, many related items
+- `lazyload`: Rarely accessed relationships
+
+### PERF-012: Transaction Scope Optimization
+**Severity:** Warning
+
+Keep transactions short and focused:
+- Don't hold transactions during I/O
+- Commit after bulk operations
+- Use read-only transactions when possible
+
+### PERF-013: Query Result Caching
+**Severity:** Info
+
+Consider caching for frequently accessed, rarely changed data like configuration tables and reference data.
+
+### PERF-014: Composite Indexes for Multi-Column Filters
+**Severity:** Info
+
+Queries filtering on multiple columns benefit from composite indexes. Order columns by selectivity (most selective first).
+
+### PERF-015: Avoid Correlated Subqueries
+**Severity:** Info
+
+Correlated subqueries execute once per row. Use JOINs or window functions instead when possible.
+
+---
+
+## Caching Performance Rules (PERF-016 to PERF-025)
+
+### PERF-016: Cache Expensive Computations
+**Severity:** Info
+
+Computationally expensive operations should be cached: complex aggregations, external API results, template rendering, data transformations.
+
+### PERF-017: Cache Key Includes Tenant Context
+**Severity:** Warning
+
+Multi-tenant cache keys must include vendor_id. Otherwise, cached data may leak between tenants.
+
+```python
+# Bad - Cache key missing tenant context
+@cache.memoize()
+def get_products():
+ return db.query(Product).all()
+
+# Good - Cache key includes vendor_id
+@cache.memoize()
+def get_products(vendor_id: int):
+ return db.query(Product).filter_by(vendor_id=vendor_id).all()
+```
+
+### PERF-018: Cache TTL Configuration
+**Severity:** Info
+
+Cache entries should have appropriate TTL:
+- Short TTL (1-5 min): Frequently changing data
+- Medium TTL (5-60 min): Semi-static data
+- Long TTL (1+ hour): Reference data
+
+### PERF-019: Cache Invalidation Strategy
+**Severity:** Warning
+
+Define cache invalidation strategy: time-based (TTL), event-based (on data change), or manual (admin action).
+
+### PERF-020: Response Caching Headers
+**Severity:** Info
+
+API responses can use HTTP caching headers: Cache-Control for browser/CDN caching, ETag for conditional requests.
+
+### PERF-021 to PERF-025: Additional Caching Rules
+**Severity:** Info
+
+Query result caching, session-level caching, distributed cache for scalability, cache warming strategy, and cache hit rate monitoring.
+
+---
+
+## API Performance Rules (PERF-026 to PERF-035)
+
+### PERF-026: Pagination Required for List Endpoints
+**Severity:** Error
+
+All list endpoints must support pagination. Unbounded lists cause memory exhaustion and slow response times.
+
+```python
+# Bad
+@router.get("/products")
+def list_products(db: Session):
+ return db.query(Product).all()
+
+# Good
+@router.get("/products")
+def list_products(
+ skip: int = 0,
+ limit: int = Query(default=20, le=100),
+ db: Session = Depends(get_db)
+):
+ return db.query(Product).offset(skip).limit(limit).all()
+```
+
+### PERF-027: Reasonable Default Page Sizes
+**Severity:** Warning
+
+Default page sizes should be reasonable:
+- Default: 20-50 items
+- Maximum: 100-200 items
+
+```python
+# Bad
+limit: int = Query(default=500, le=10000)
+
+# Good
+limit: int = Query(default=20, ge=1, le=100)
+```
+
+### PERF-028: Response Compression
+**Severity:** Info
+
+Enable response compression for large responses with GZip or Brotli.
+
+### PERF-029: Efficient Serialization
+**Severity:** Info
+
+Use Pydantic's `response_model` for efficient serialization. Avoid manual dict conversion.
+
+### PERF-030: Avoid Redundant Queries in Response
+**Severity:** Warning
+
+Don't trigger lazy-loaded relationships during serialization. Use eager loading or carefully control serialization.
+
+### PERF-031: Streaming for Large Responses
+**Severity:** Info
+
+Use streaming responses for large data: file downloads, large exports (CSV, JSON), real-time data feeds.
+
+### PERF-032 to PERF-035: Additional API Rules
+**Severity:** Info/Warning
+
+Conditional requests support, field selection support, avoid deep nesting in responses, and endpoint response time monitoring.
+
+---
+
+## Async & Concurrency Rules (PERF-036 to PERF-045)
+
+### PERF-036: Async for I/O Operations
+**Severity:** Info
+
+Use async for I/O-bound operations: database queries, external API calls, file operations.
+
+### PERF-037: Parallel Independent Operations
+**Severity:** Warning
+
+Multiple independent async operations should run in parallel. Use `asyncio.gather()` instead of sequential awaits.
+
+```python
+# Bad - Sequential awaits
+user = await get_user(user_id)
+orders = await get_orders(user_id)
+preferences = await get_preferences(user_id)
+
+# Good - Parallel execution
+user, orders, preferences = await asyncio.gather(
+ get_user(user_id),
+ get_orders(user_id),
+ get_preferences(user_id)
+)
+```
+
+### PERF-038: Background Tasks for Slow Operations
+**Severity:** Warning
+
+Operations taking > 500ms should run in background: email sending, report generation, external API syncs, file processing.
+
+### PERF-039: Connection Pooling for HTTP Clients
+**Severity:** Warning
+
+HTTP clients should reuse connections. Create client once, not per request.
+
+```python
+# Bad
+def fetch_data(url):
+ response = requests.get(url) # New connection each time
+
+# Good
+async with httpx.AsyncClient() as client:
+ response = await client.get(url)
+```
+
+### PERF-040: Timeout Configuration
+**Severity:** Error
+
+All external calls must have timeouts. Without timeouts, requests can hang indefinitely.
+
+```python
+# Bad
+response = requests.get(url)
+
+# Good
+response = requests.get(url, timeout=30)
+```
+
+### PERF-041 to PERF-045: Additional Async Rules
+**Severity:** Info
+
+Connection pool limits, retry with backoff, circuit breaker pattern, task queues for heavy processing, and worker pool sizing.
+
+---
+
+## Memory Management Rules (PERF-046 to PERF-055)
+
+### PERF-046: Generators for Large Datasets
+**Severity:** Warning
+
+Use generators/iterators for processing large datasets. Avoids loading everything into memory at once.
+
+```python
+# Bad - Loads all into memory
+products = db.query(Product).all()
+for product in products:
+ process(product)
+
+# Good - Stream processing
+for product in db.query(Product).yield_per(100):
+ process(product)
+```
+
+### PERF-047: Stream Large File Uploads
+**Severity:** Warning
+
+Large files should be streamed to disk, not held in memory.
+
+```python
+# Bad - Entire file in memory
+content = await file.read()
+with open(path, 'wb') as f:
+ f.write(content)
+
+# Good - Streaming
+with open(path, 'wb') as f:
+ while chunk := await file.read(8192):
+ f.write(chunk)
+```
+
+### PERF-048: Chunked Processing for Imports
+**Severity:** Warning
+
+Bulk imports should process in chunks: read in batches, commit in batches, report progress periodically.
+
+### PERF-049: Context Managers for Resources
+**Severity:** Error
+
+Use context managers for file operations. Ensures resources are properly released.
+
+```python
+# Bad
+f = open('file.txt')
+content = f.read()
+f.close() # May not run if exception
+
+# Good
+with open('file.txt') as f:
+ content = f.read()
+```
+
+### PERF-050 to PERF-055: Additional Memory Rules
+**Severity:** Info
+
+Limit in-memory collections, string concatenation efficiency, efficient data structures, object pooling, weak references for caches, and slots for frequently instantiated classes.
+
+---
+
+## Frontend Performance Rules (PERF-056 to PERF-070)
+
+### PERF-056: Debounce Search Inputs
+**Severity:** Warning
+
+Search inputs should debounce API calls. Recommended: 300-500ms delay.
+
+```javascript
+// Bad
+
+
+// Good
+
+// With: debouncedSearch = debounce(searchProducts, 300)
+```
+
+### PERF-057: Lazy Load Off-Screen Content
+**Severity:** Info
+
+Defer loading of off-screen content: modals, inactive tabs, below-the-fold content, images.
+
+### PERF-058: Image Optimization
+**Severity:** Warning
+
+Images should be optimized:
+- Use appropriate formats (WebP, AVIF)
+- Serve responsive sizes
+- Lazy load off-screen images
+- Use CDN for static assets
+
+```html
+
+```
+
+### PERF-059: Minimize Alpine.js Watchers
+**Severity:** Info
+
+Excessive `$watch` calls impact performance. Use computed properties or event handlers instead.
+
+### PERF-060: Virtual Scrolling for Long Lists
+**Severity:** Info
+
+Lists with 100+ items should use virtual scrolling. Only render visible items in the viewport.
+
+### PERF-061: Minimize Bundle Size
+**Severity:** Info
+
+Reduce JavaScript bundle size: import only needed modules, use tree-shaking, split code by route.
+
+### PERF-062: Reasonable Polling Intervals
+**Severity:** Warning
+
+Polling should be >= 10 seconds for non-critical updates.
+
+```javascript
+// Bad
+setInterval(fetchUpdates, 1000); // Every second
+
+// Good
+setInterval(fetchUpdates, 30000); // Every 30 seconds
+```
+
+### PERF-063 to PERF-070: Additional Frontend Rules
+**Severity:** Info/Warning
+
+CSS containment, avoid layout thrashing, CSS animations over JavaScript, preload critical resources, defer non-critical JavaScript, minimize DOM nodes, efficient event handlers, and cache DOM queries.
+
+---
+
+## Configuration
+
+All rules are defined in `.performance-rules/` directory:
+
+```
+.performance-rules/
+├── _main.yaml # Main configuration
+├── database.yaml # PERF-001 to PERF-015
+├── caching.yaml # PERF-016 to PERF-025
+├── api.yaml # PERF-026 to PERF-035
+├── async.yaml # PERF-036 to PERF-045
+├── memory.yaml # PERF-046 to PERF-055
+└── frontend.yaml # PERF-056 to PERF-070
+```
+
+## Suppressing Rules
+
+Use noqa comments to suppress specific rules:
+
+```python
+# noqa: PERF-003 - This is intentionally unbounded for admin export
+products = db.query(Product).all()
+```
+
+## Related Documentation
+
+- [Architecture Rules](architecture-rules.md)
+- [Security Rules](security-rules.md)
+- [Code Quality Guide](code-quality.md)
+- [Contributing Guide](contributing.md)
+
+---
+
+## Summary Statistics
+
+| Category | Rules | Errors | Warnings | Info |
+|----------|-------|--------|----------|------|
+| Database | 15 | 0 | 5 | 10 |
+| Caching | 10 | 0 | 2 | 8 |
+| API | 10 | 1 | 3 | 6 |
+| Async & Concurrency | 10 | 1 | 4 | 5 |
+| Memory Management | 10 | 1 | 4 | 5 |
+| Frontend | 15 | 0 | 4 | 11 |
+| **Total** | **70** | **3** | **22** | **45** |
+
+---
+
+**Last Updated:** 2025-12-21
+**Version:** 1.0
diff --git a/docs/development/security-rules.md b/docs/development/security-rules.md
new file mode 100644
index 00000000..4b7ec185
--- /dev/null
+++ b/docs/development/security-rules.md
@@ -0,0 +1,560 @@
+# Security Rules Reference
+
+This document provides a comprehensive reference for all security rules enforced by the `scripts/validate_security.py` validator.
+
+## Overview
+
+The security validator identifies potential security vulnerabilities and enforces security best practices across the codebase. Rules are organized by category and severity level.
+
+**Version:** 1.0
+**Total Rules:** 60
+**Configuration Directory:** `.security-rules/`
+
+## Running the Validator
+
+### Using Python Directly
+
+```bash
+# Check all files
+python scripts/validate_security.py
+
+# Verbose output
+python scripts/validate_security.py -v
+
+# Errors only
+python scripts/validate_security.py --errors-only
+
+# JSON output (for CI/CD)
+python scripts/validate_security.py --json
+```
+
+### Using the Unified Validator
+
+```bash
+# Run security checks only
+python scripts/validate_all.py --security
+
+# Run all validators
+python scripts/validate_all.py
+```
+
+## Severity Levels
+
+| Severity | Description | Exit Code | Action Required |
+|----------|-------------|-----------|-----------------|
+| **Error** | Critical security vulnerability | 1 | Must fix immediately |
+| **Warning** | Security concern | 0 | Should fix |
+| **Info** | Security suggestion | 0 | Consider implementing |
+
+---
+
+## Authentication Rules (SEC-001 to SEC-010)
+
+### SEC-001: No Hardcoded Credentials
+**Severity:** Error
+
+Credentials must never be hardcoded in source code. Use environment variables or secret management.
+
+```python
+# Bad
+api_key = "sk-1234567890abcdef"
+password = "admin123"
+
+# Good
+api_key = os.environ.get("API_KEY")
+password = settings.admin_password
+```
+
+### SEC-002: JWT Expiry Enforcement
+**Severity:** Error
+
+All JWT tokens must have expiration claims. Access tokens should expire in 15-60 minutes.
+
+### SEC-003: Password Hashing Required
+**Severity:** Error
+
+Passwords must be hashed using bcrypt, argon2, or scrypt. Never store or compare passwords in plain text.
+
+```python
+# Bad
+if user.password == password:
+ ...
+
+# Good
+if bcrypt.checkpw(password.encode(), user.hashed_password):
+ ...
+```
+
+### SEC-004: Session Regeneration After Auth
+**Severity:** Warning
+
+Session IDs should be regenerated after authentication to prevent session fixation attacks.
+
+### SEC-005: Brute Force Protection
+**Severity:** Warning
+
+Login endpoints should implement rate limiting or account lockout after failed attempts.
+
+### SEC-006: Secure Password Reset
+**Severity:** Warning
+
+Password reset tokens must be cryptographically random, expire within 1 hour, and be single-use.
+
+### SEC-007: Authentication on Sensitive Endpoints
+**Severity:** Error
+
+All endpoints except public ones must require authentication.
+
+### SEC-008: Token in Authorization Header
+**Severity:** Warning
+
+JWT tokens should be sent in Authorization header, not in URL parameters.
+
+### SEC-009: Logout Invalidates Tokens
+**Severity:** Warning
+
+Logout should invalidate or blacklist tokens.
+
+### SEC-010: Multi-Factor Authentication Support
+**Severity:** Info
+
+Consider implementing MFA for sensitive operations.
+
+---
+
+## Injection Prevention Rules (SEC-011 to SEC-020)
+
+### SEC-011: No Raw SQL Queries
+**Severity:** Error
+
+Use SQLAlchemy ORM or parameterized queries only. Never concatenate user input into SQL strings.
+
+```python
+# Bad
+db.execute(f"SELECT * FROM users WHERE name = '{name}'")
+
+# Good
+db.query(User).filter(User.name == name)
+```
+
+### SEC-012: No Shell Command Injection
+**Severity:** Error
+
+Never use `shell=True` with subprocess. Use subprocess with list arguments.
+
+```python
+# Bad
+subprocess.run(f"convert {filename}", shell=True)
+os.system(f"rm {filename}")
+
+# Good
+subprocess.run(["convert", filename])
+```
+
+### SEC-013: No Code Execution
+**Severity:** Error
+
+Never use `eval()` or `exec()` with user input.
+
+### SEC-014: Path Traversal Prevention
+**Severity:** Error
+
+Validate file paths to prevent directory traversal. Use `secure_filename()` for uploads.
+
+```python
+# Bad
+path = f"/uploads/{filename}"
+
+# Good
+from werkzeug.utils import secure_filename
+path = f"/uploads/{secure_filename(filename)}"
+```
+
+### SEC-015: XSS Prevention in Templates
+**Severity:** Error
+
+Use safe output methods in templates. Prefer `x-text` over `x-html` in Alpine.js.
+
+```html
+
+
+
+
+
+```
+
+### SEC-016: LDAP Injection Prevention
+**Severity:** Error
+
+Escape special characters in LDAP queries.
+
+### SEC-017: XML External Entity Prevention
+**Severity:** Error
+
+Disable external entities when parsing XML. Use `defusedxml`.
+
+```python
+# Bad
+import xml.etree.ElementTree as ET
+tree = ET.parse(user_file)
+
+# Good
+import defusedxml.ElementTree as ET
+tree = ET.parse(user_file)
+```
+
+### SEC-018: Template Injection Prevention
+**Severity:** Error
+
+Never render user input as template code.
+
+### SEC-019: SSRF Prevention
+**Severity:** Warning
+
+Validate URLs before making external requests. Whitelist allowed domains.
+
+### SEC-020: Deserialization Safety
+**Severity:** Error
+
+Never deserialize untrusted data with pickle. Use `yaml.safe_load()` instead of `yaml.load()`.
+
+```python
+# Bad
+data = pickle.loads(user_data)
+config = yaml.load(user_config)
+
+# Good
+config = yaml.safe_load(user_config)
+data = json.loads(user_data)
+```
+
+---
+
+## Data Protection Rules (SEC-021 to SEC-030)
+
+### SEC-021: PII Logging Prevention
+**Severity:** Error
+
+Never log passwords, tokens, credit cards, or sensitive PII.
+
+```python
+# Bad
+logger.info(f"User login: {username}, password: {password}")
+
+# Good
+logger.info(f"User login: {username}")
+```
+
+### SEC-022: Sensitive Data in URLs
+**Severity:** Error
+
+Sensitive data should not appear in URL query parameters. Use POST body or headers instead.
+
+### SEC-023: Mass Assignment Prevention
+**Severity:** Warning
+
+Use explicit field assignment, not `**kwargs` from user input.
+
+```python
+# Bad
+user = User(**request.json)
+
+# Good
+user = User(
+ name=request.json.get("name"),
+ email=request.json.get("email")
+)
+```
+
+### SEC-024: Error Message Information Leakage
+**Severity:** Error
+
+Error messages should not reveal internal details. No stack traces to users.
+
+### SEC-025: Secure Cookie Settings
+**Severity:** Error
+
+Cookies must have Secure, HttpOnly, SameSite attributes.
+
+### SEC-026: Encryption for Sensitive Data at Rest
+**Severity:** Info
+
+Consider encrypting sensitive data stored in the database.
+
+### SEC-027: Data Retention Limits
+**Severity:** Info
+
+Implement data retention policies.
+
+### SEC-028: Response Data Filtering
+**Severity:** Warning
+
+API responses should not include sensitive internal fields. Use Pydantic response models.
+
+### SEC-029: File Upload Validation
+**Severity:** Error
+
+Validate uploaded files by extension AND content type. Limit file size.
+
+### SEC-030: Backup Encryption
+**Severity:** Info
+
+Database backups should be encrypted.
+
+---
+
+## API Security Rules (SEC-031 to SEC-040)
+
+### SEC-031: CORS Origin Validation
+**Severity:** Error
+
+CORS must not allow all origins in production. Specify allowed origins explicitly.
+
+```python
+# Bad
+allow_origins=["*"]
+
+# Good
+allow_origins=["https://example.com", "https://api.example.com"]
+```
+
+### SEC-032: Rate Limiting on Sensitive Endpoints
+**Severity:** Warning
+
+Auth, password reset, and payment endpoints need rate limiting.
+
+### SEC-033: Security Headers
+**Severity:** Warning
+
+Configure security headers like X-Content-Type-Options, X-Frame-Options, Content-Security-Policy.
+
+### SEC-034: HTTPS Enforcement
+**Severity:** Error
+
+External URLs must use HTTPS. HTTP is only acceptable for localhost.
+
+### SEC-035: Request Size Limits
+**Severity:** Warning
+
+Limit request body size to prevent DoS attacks.
+
+### SEC-036: Input Validation with Pydantic
+**Severity:** Warning
+
+All API inputs should be validated using Pydantic models.
+
+### SEC-037: API Versioning
+**Severity:** Info
+
+APIs should be versioned for security update isolation.
+
+### SEC-038: Method Restrictions
+**Severity:** Warning
+
+Endpoints should only allow necessary HTTP methods.
+
+### SEC-039: Authentication Bypass Prevention
+**Severity:** Error
+
+Ensure authentication cannot be bypassed.
+
+### SEC-040: Timeout Configuration
+**Severity:** Warning
+
+All external calls must have timeouts configured.
+
+---
+
+## Cryptography Rules (SEC-041 to SEC-050)
+
+### SEC-041: Strong Hashing Algorithms
+**Severity:** Error
+
+Use bcrypt, argon2, scrypt for passwords. Use SHA-256 or stronger for general hashing. Never use MD5 or SHA1.
+
+```python
+# Bad
+import hashlib
+password_hash = hashlib.md5(password.encode()).hexdigest()
+
+# Good
+import bcrypt
+password_hash = bcrypt.hashpw(password.encode(), bcrypt.gensalt())
+```
+
+### SEC-042: Secure Random Generation
+**Severity:** Error
+
+Use the `secrets` module for security-sensitive randomness. Never use `random` module for tokens or keys.
+
+```python
+# Bad
+import random
+token = ''.join(random.choices(string.ascii_letters, k=32))
+
+# Good
+import secrets
+token = secrets.token_urlsafe(32)
+```
+
+### SEC-043: No Hardcoded Encryption Keys
+**Severity:** Error
+
+Encryption keys must come from environment variables or secret management services.
+
+### SEC-044: Strong Encryption Algorithms
+**Severity:** Error
+
+Use AES-256 or ChaCha20. Never use DES, 3DES, or RC4.
+
+### SEC-045: Proper IV/Nonce Usage
+**Severity:** Error
+
+Encryption IVs and nonces must be randomly generated and unique per encryption.
+
+### SEC-046: TLS Version Requirements
+**Severity:** Warning
+
+Enforce TLS 1.2 or higher. Disable SSLv2, SSLv3, TLS 1.0, TLS 1.1.
+
+### SEC-047: Certificate Verification
+**Severity:** Error
+
+Always verify SSL certificates. Never disable verification in production.
+
+```python
+# Bad
+requests.get(url, verify=False)
+
+# Good
+requests.get(url, verify=True)
+```
+
+### SEC-048: Key Derivation for Passwords
+**Severity:** Warning
+
+When deriving encryption keys from passwords, use PBKDF2 with 100K+ iterations, Argon2, or scrypt.
+
+### SEC-049: Secure Key Storage
+**Severity:** Info
+
+Encryption keys should be stored in environment variables, secret management, or HSMs.
+
+### SEC-050: Key Rotation Support
+**Severity:** Info
+
+Implement key rotation with multiple key versions.
+
+---
+
+## Audit & Logging Rules (SEC-051 to SEC-060)
+
+### SEC-051: Authentication Event Logging
+**Severity:** Warning
+
+Log authentication events including successful logins, failed attempts, logouts, and password changes.
+
+### SEC-052: Admin Action Audit Trail
+**Severity:** Warning
+
+All admin operations should be logged with admin user ID, action performed, target resource, timestamp, and IP address.
+
+### SEC-053: Data Modification Logging
+**Severity:** Info
+
+Log create/update/delete on sensitive data like user accounts, roles, financial transactions, and configuration changes.
+
+### SEC-054: Security Event Logging
+**Severity:** Warning
+
+Log security-relevant events like authorization failures, input validation failures, rate limit triggers, and suspicious activity.
+
+### SEC-055: Log Injection Prevention
+**Severity:** Warning
+
+Sanitize user input before logging. Newlines and control characters can corrupt logs.
+
+```python
+# Bad
+logger.info(f"User search: {request.query}")
+
+# Good
+logger.info(f"User search: {request.query!r}") # repr escapes
+```
+
+### SEC-056: Centralized Logging
+**Severity:** Info
+
+Use centralized logging for correlation across services, tamper-evident storage, and alerting.
+
+### SEC-057: Log Level Appropriateness
+**Severity:** Info
+
+Use appropriate log levels: ERROR for security failures, WARNING for suspicious activity, INFO for successful events.
+
+### SEC-058: Structured Logging Format
+**Severity:** Info
+
+Use structured logging (JSON) for easy parsing and searchability.
+
+### SEC-059: Audit Log Integrity
+**Severity:** Info
+
+Protect audit logs from tampering with append-only storage and cryptographic chaining.
+
+### SEC-060: Privacy-Aware Logging
+**Severity:** Warning
+
+Comply with data protection regulations. No PII in logs without consent.
+
+---
+
+## Configuration
+
+All rules are defined in `.security-rules/` directory:
+
+```
+.security-rules/
+├── _main.yaml # Main configuration
+├── authentication.yaml # SEC-001 to SEC-010
+├── injection.yaml # SEC-011 to SEC-020
+├── data_protection.yaml # SEC-021 to SEC-030
+├── api_security.yaml # SEC-031 to SEC-040
+├── cryptography.yaml # SEC-041 to SEC-050
+└── audit.yaml # SEC-051 to SEC-060
+```
+
+## Suppressing Rules
+
+Use noqa comments to suppress specific rules:
+
+```python
+# noqa: SEC-001 - This is a test file with intentional test credentials
+test_password = "test123"
+```
+
+## Related Documentation
+
+- [Architecture Rules](architecture-rules.md)
+- [Performance Rules](performance-rules.md)
+- [Code Quality Guide](code-quality.md)
+- [Contributing Guide](contributing.md)
+
+---
+
+## Summary Statistics
+
+| Category | Rules | Errors | Warnings | Info |
+|----------|-------|--------|----------|------|
+| Authentication | 10 | 4 | 5 | 1 |
+| Injection Prevention | 10 | 9 | 1 | 0 |
+| Data Protection | 10 | 4 | 3 | 3 |
+| API Security | 10 | 4 | 5 | 1 |
+| Cryptography | 10 | 6 | 2 | 2 |
+| Audit & Logging | 10 | 0 | 5 | 5 |
+| **Total** | **60** | **27** | **21** | **12** |
+
+---
+
+**Last Updated:** 2025-12-21
+**Version:** 1.0
diff --git a/mkdocs.yml b/mkdocs.yml
index 29202247..d62646ab 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -40,6 +40,7 @@ nav:
- Authentication & RBAC: architecture/auth-rbac.md
- Frontend Structure: architecture/frontend-structure.md
- Models Structure: architecture/models-structure.md
+ - Background Tasks: architecture/background-tasks.md
- API Consolidation:
- Proposal: architecture/api-consolidation-proposal.md
- Migration Status: architecture/api-migration-status.md
@@ -117,6 +118,8 @@ nav:
- Contributing Guide: development/contributing.md
- Code Quality: development/code-quality.md
- Architecture Rules: development/architecture-rules.md
+ - Security Rules: development/security-rules.md
+ - Performance Rules: development/performance-rules.md
- Code Quality Dashboard: development/code-quality-dashboard-implementation.md
- Icons Guide: development/icons-guide.md
- Naming Conventions: development/naming-conventions.md
diff --git a/models/schema/stats.py b/models/schema/stats.py
index 182f2dbb..8f7e6e5a 100644
--- a/models/schema/stats.py
+++ b/models/schema/stats.py
@@ -243,15 +243,29 @@ class VendorAnalyticsResponse(BaseModel):
# ============================================================================
+class ValidatorStats(BaseModel):
+ """Statistics for a single validator type."""
+
+ total_violations: int = 0
+ errors: int = 0
+ warnings: int = 0
+ last_scan: str | None = None
+
+
class CodeQualityDashboardStatsResponse(BaseModel):
"""Code quality dashboard statistics response schema.
Used by: GET /api/v1/admin/code-quality/stats
+
+ Supports multiple validator types: architecture, security, performance.
+ When validator_type is specified, returns stats for that type only.
+ When not specified, returns combined stats with per-validator breakdown.
"""
total_violations: int
errors: int
warnings: int
+ info: int = 0
open: int
assigned: int
resolved: int
@@ -263,6 +277,11 @@ class CodeQualityDashboardStatsResponse(BaseModel):
by_module: dict[str, Any] = Field(default_factory=dict)
top_files: list[dict[str, Any]] = Field(default_factory=list)
last_scan: str | None = None
+ validator_type: str | None = None # Set when filtering by type
+ by_validator: dict[str, ValidatorStats] = Field(
+ default_factory=dict,
+ description="Per-validator breakdown (architecture, security, performance)",
+ )
# ============================================================================
diff --git a/scripts/base_validator.py b/scripts/base_validator.py
new file mode 100755
index 00000000..ce2d4224
--- /dev/null
+++ b/scripts/base_validator.py
@@ -0,0 +1,465 @@
+#!/usr/bin/env python3
+"""
+Base Validator
+==============
+Shared base class for all validation scripts (architecture, security, performance).
+
+Provides common functionality for:
+- Loading YAML configuration
+- File pattern matching
+- Violation tracking
+- Output formatting (human-readable and JSON)
+"""
+
+import json
+import re
+import sys
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+from enum import Enum
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+
+class Severity(Enum):
+ """Validation severity levels"""
+
+ ERROR = "error"
+ WARNING = "warning"
+ INFO = "info"
+
+
+@dataclass
+class Violation:
+ """Represents a rule violation"""
+
+ rule_id: str
+ rule_name: str
+ severity: Severity
+ file_path: Path
+ line_number: int
+ message: str
+ context: str = ""
+ suggestion: str = ""
+
+
+@dataclass
+class FileResult:
+ """Results for a single file validation"""
+
+ file_path: Path
+ errors: int = 0
+ warnings: int = 0
+ info: int = 0
+
+ @property
+ def passed(self) -> bool:
+ return self.errors == 0
+
+ @property
+ def status(self) -> str:
+ if self.errors > 0:
+ return "FAILED"
+ if self.warnings > 0:
+ return "PASSED*"
+ return "PASSED"
+
+ @property
+ def status_icon(self) -> str:
+ if self.errors > 0:
+ return "❌"
+ if self.warnings > 0:
+ return "⚠️"
+ return "✅"
+
+
+@dataclass
+class ValidationResult:
+ """Results of validation"""
+
+ violations: list[Violation] = field(default_factory=list)
+ files_checked: int = 0
+ rules_applied: int = 0
+ file_results: list[FileResult] = field(default_factory=list)
+
+ def has_errors(self) -> bool:
+ """Check if there are any error-level violations"""
+ return any(v.severity == Severity.ERROR for v in self.violations)
+
+ def has_warnings(self) -> bool:
+ """Check if there are any warning-level violations"""
+ return any(v.severity == Severity.WARNING for v in self.violations)
+
+ def error_count(self) -> int:
+ return sum(1 for v in self.violations if v.severity == Severity.ERROR)
+
+ def warning_count(self) -> int:
+ return sum(1 for v in self.violations if v.severity == Severity.WARNING)
+
+ def info_count(self) -> int:
+ return sum(1 for v in self.violations if v.severity == Severity.INFO)
+
+
+class BaseValidator(ABC):
+ """Abstract base validator class"""
+
+ # Subclasses should override these
+ VALIDATOR_NAME = "Base Validator"
+ VALIDATOR_EMOJI = "🔍"
+ RULES_DIR_NAME = ".rules"
+ CONFIG_FILE_NAME = ".rules.yaml"
+
+ def __init__(self, config_path: Path = None, verbose: bool = False):
+ """Initialize validator with configuration"""
+ self.project_root = Path.cwd()
+ self.config_path = config_path or self.project_root / self.CONFIG_FILE_NAME
+ self.verbose = verbose
+ self.config = self._load_config()
+ self.result = ValidationResult()
+
+ def _load_config(self) -> dict[str, Any]:
+ """
+ Load validation rules from YAML config.
+
+ Supports two modes:
+ 1. Split directory mode: rules directory with multiple YAML files
+ 2. Single file mode: single YAML file (legacy)
+
+ The split directory mode takes precedence if it exists.
+ """
+ # Check for split directory mode first
+ rules_dir = self.project_root / self.RULES_DIR_NAME
+ if rules_dir.is_dir():
+ return self._load_config_from_directory(rules_dir)
+
+ # Fall back to single file mode
+ if not self.config_path.exists():
+ print(f"❌ Configuration file not found: {self.config_path}")
+ print(f" (Also checked for directory: {rules_dir})")
+ sys.exit(1)
+
+ with open(self.config_path) as f:
+ config = yaml.safe_load(f)
+
+ print(f"📋 Loaded {self.VALIDATOR_NAME}: {config.get('project', 'unknown')}")
+ return config
+
+ def _load_config_from_directory(self, rules_dir: Path) -> dict[str, Any]:
+ """
+ Load and merge configuration from split YAML files in a directory.
+
+ Reads _main.yaml first for base config, then merges all other YAML files.
+ """
+ config: dict[str, Any] = {}
+
+ # Load _main.yaml first (contains project info, principles, ignore patterns)
+ main_file = rules_dir / "_main.yaml"
+ if main_file.exists():
+ with open(main_file) as f:
+ config = yaml.safe_load(f) or {}
+
+ # Load all other YAML files and merge their contents
+ yaml_files = sorted(rules_dir.glob("*.yaml"))
+ for yaml_file in yaml_files:
+ if yaml_file.name == "_main.yaml":
+ continue # Already loaded
+
+ with open(yaml_file) as f:
+ file_config = yaml.safe_load(f) or {}
+
+ # Merge rule sections from this file into main config
+ for key, value in file_config.items():
+ if key.endswith("_rules") and isinstance(value, list):
+ # Merge rule lists
+ if key not in config:
+ config[key] = []
+ config[key].extend(value)
+ elif key not in config:
+ # Add new top-level keys
+ config[key] = value
+
+ print(f"📋 Loaded {self.VALIDATOR_NAME}: {config.get('project', 'unknown')}")
+ print(f" (from {len(yaml_files)} files in {rules_dir.name}/)")
+ return config
+
+ def _should_ignore_file(self, file_path: Path) -> bool:
+ """Check if a file should be ignored based on config patterns"""
+ import fnmatch
+
+ ignore_config = self.config.get("ignore", {})
+ ignore_files = ignore_config.get("files", [])
+
+ # Get relative path for matching
+ try:
+ rel_path = file_path.relative_to(self.project_root)
+ except ValueError:
+ rel_path = file_path
+
+ rel_path_str = str(rel_path)
+
+ for pattern in ignore_files:
+ # Handle glob patterns using fnmatch
+ if "*" in pattern:
+ # fnmatch handles *, **, and ? patterns correctly
+ if fnmatch.fnmatch(rel_path_str, pattern):
+ return True
+ # Also check each path component for patterns like **/.venv/**
+ # This handles cases where the pattern expects any prefix
+ if pattern.startswith("**/"):
+ # Try matching without the **/ prefix (e.g., .venv/** matches .venv/foo)
+ suffix_pattern = pattern[3:] # Remove "**/""
+ if fnmatch.fnmatch(rel_path_str, suffix_pattern):
+ return True
+ elif pattern in rel_path_str:
+ return True
+
+ return False
+
+ def _add_violation(
+ self,
+ rule_id: str,
+ rule_name: str,
+ severity: Severity,
+ file_path: Path,
+ line_number: int,
+ message: str,
+ context: str = "",
+ suggestion: str = "",
+ ):
+ """Add a violation to the results"""
+ # Check for inline noqa comment
+ if f"noqa: {rule_id.lower()}" in context.lower():
+ return
+
+ self.result.violations.append(
+ Violation(
+ rule_id=rule_id,
+ rule_name=rule_name,
+ severity=severity,
+ file_path=file_path,
+ line_number=line_number,
+ message=message,
+ context=context,
+ suggestion=suggestion,
+ )
+ )
+
+ def _get_rule(self, rule_id: str) -> dict | None:
+ """Look up a rule by ID across all rule categories"""
+ for key, value in self.config.items():
+ if key.endswith("_rules") and isinstance(value, list):
+ for rule in value:
+ if rule.get("id") == rule_id:
+ return rule
+ return None
+
+ def _check_pattern_in_file(
+ self,
+ file_path: Path,
+ content: str,
+ lines: list[str],
+ pattern: str,
+ rule_id: str,
+ rule_name: str,
+ severity: Severity,
+ message: str,
+ suggestion: str = "",
+ exclude_patterns: list[str] = None,
+ ):
+ """Check for a regex pattern in a file and report violations"""
+ exclude_patterns = exclude_patterns or []
+
+ for i, line in enumerate(lines, 1):
+ if re.search(pattern, line, re.IGNORECASE):
+ # Check exclusions
+ should_exclude = False
+ for exclude in exclude_patterns:
+ if exclude in line:
+ should_exclude = True
+ break
+
+ if not should_exclude:
+ self._add_violation(
+ rule_id=rule_id,
+ rule_name=rule_name,
+ severity=severity,
+ file_path=file_path,
+ line_number=i,
+ message=message,
+ context=line.strip()[:100],
+ suggestion=suggestion,
+ )
+
+ @abstractmethod
+ def validate_all(self, target_path: Path = None) -> ValidationResult:
+ """Validate all files in a directory - must be implemented by subclasses"""
+ pass
+
+ def validate_file(self, file_path: Path, quiet: bool = False) -> ValidationResult:
+ """Validate a single file"""
+ if not file_path.exists():
+ if not quiet:
+ print(f"❌ File not found: {file_path}")
+ return self.result
+
+ if not file_path.is_file():
+ if not quiet:
+ print(f"❌ Not a file: {file_path}")
+ return self.result
+
+ if not quiet:
+ print(f"\n{self.VALIDATOR_EMOJI} Validating single file: {file_path}\n")
+
+ # Resolve file path to absolute
+ file_path = file_path.resolve()
+
+ if self._should_ignore_file(file_path):
+ if not quiet:
+ print("⏭️ File is in ignore list, skipping")
+ return self.result
+
+ self.result.files_checked += 1
+
+ # Track violations before this file
+ violations_before = len(self.result.violations)
+
+ content = file_path.read_text()
+ lines = content.split("\n")
+
+ # Call subclass-specific validation
+ self._validate_file_content(file_path, content, lines)
+
+ # Calculate violations for this file
+ file_violations = self.result.violations[violations_before:]
+ errors = sum(1 for v in file_violations if v.severity == Severity.ERROR)
+ warnings = sum(1 for v in file_violations if v.severity == Severity.WARNING)
+ info = sum(1 for v in file_violations if v.severity == Severity.INFO)
+
+ # Track file result
+ self.result.file_results.append(
+ FileResult(file_path=file_path, errors=errors, warnings=warnings, info=info)
+ )
+
+ return self.result
+
+ @abstractmethod
+ def _validate_file_content(self, file_path: Path, content: str, lines: list[str]):
+ """Validate file content - must be implemented by subclasses"""
+ pass
+
+ def output_results(self, json_output: bool = False, errors_only: bool = False):
+ """Output validation results"""
+ if json_output:
+ self._output_json()
+ else:
+ self._output_human(errors_only)
+
+ def _output_json(self):
+ """Output results as JSON
+
+ Format matches code quality service expectations:
+ - file_path (not file)
+ - line_number (not line)
+ - total_violations count
+ """
+ try:
+ rel_base = self.project_root
+ except Exception:
+ rel_base = Path.cwd()
+
+ def get_relative_path(file_path: Path) -> str:
+ """Get relative path from project root"""
+ try:
+ return str(file_path.relative_to(rel_base))
+ except ValueError:
+ return str(file_path)
+
+ output = {
+ "validator": self.VALIDATOR_NAME,
+ "files_checked": self.result.files_checked,
+ "total_violations": len(self.result.violations),
+ "errors": self.result.error_count(),
+ "warnings": self.result.warning_count(),
+ "info": self.result.info_count(),
+ "violations": [
+ {
+ "rule_id": v.rule_id,
+ "rule_name": v.rule_name,
+ "severity": v.severity.value,
+ "file_path": get_relative_path(v.file_path),
+ "line_number": v.line_number,
+ "message": v.message,
+ "context": v.context,
+ "suggestion": v.suggestion,
+ }
+ for v in self.result.violations
+ ],
+ }
+ print(json.dumps(output, indent=2))
+
+ def _output_human(self, errors_only: bool = False):
+ """Output results in human-readable format"""
+ print("\n" + "=" * 80)
+ print(f"📊 {self.VALIDATOR_NAME.upper()} REPORT")
+ print("=" * 80)
+
+ errors = [v for v in self.result.violations if v.severity == Severity.ERROR]
+ warnings = [v for v in self.result.violations if v.severity == Severity.WARNING]
+ info = [v for v in self.result.violations if v.severity == Severity.INFO]
+
+ print(
+ f"\nFiles checked: {self.result.files_checked}"
+ )
+ print(
+ f"Findings: {len(errors)} errors, {len(warnings)} warnings, {len(info)} info"
+ )
+
+ if errors:
+ print(f"\n\n❌ ERRORS ({len(errors)}):")
+ print("-" * 80)
+ for v in errors:
+ self._print_violation(v)
+
+ if warnings and not errors_only:
+ print(f"\n\n⚠️ WARNINGS ({len(warnings)}):")
+ print("-" * 80)
+ for v in warnings:
+ self._print_violation(v)
+
+ if info and not errors_only:
+ print(f"\nℹ️ INFO ({len(info)}):")
+ print("-" * 80)
+ for v in info:
+ self._print_violation(v)
+
+ print("\n" + "=" * 80)
+ if errors:
+ print("❌ VALIDATION FAILED")
+ elif warnings:
+ print(f"⚠️ VALIDATION PASSED WITH {len(warnings)} WARNING(S)")
+ else:
+ print("✅ VALIDATION PASSED")
+ print("=" * 80)
+
+ def _print_violation(self, v: Violation):
+ """Print a single violation"""
+ try:
+ rel_path = v.file_path.relative_to(self.project_root)
+ except ValueError:
+ rel_path = v.file_path
+
+ print(f"\n [{v.rule_id}] {v.rule_name}")
+ print(f" File: {rel_path}:{v.line_number}")
+ print(f" Issue: {v.message}")
+ if v.context and self.verbose:
+ print(f" Context: {v.context}")
+ if v.suggestion:
+ print(f" 💡 Suggestion: {v.suggestion}")
+
+ def get_exit_code(self) -> int:
+ """Get appropriate exit code based on results"""
+ if self.result.has_errors():
+ return 1
+ return 0
diff --git a/scripts/validate_all.py b/scripts/validate_all.py
new file mode 100755
index 00000000..d864731a
--- /dev/null
+++ b/scripts/validate_all.py
@@ -0,0 +1,218 @@
+#!/usr/bin/env python3
+"""
+Unified Code Validator
+======================
+Runs all validation scripts (architecture, security, performance) in sequence.
+
+This provides a single entry point for comprehensive code validation,
+useful for CI/CD pipelines and pre-commit hooks.
+
+Usage:
+ python scripts/validate_all.py # Run all validators
+ python scripts/validate_all.py --security # Run only security validator
+ python scripts/validate_all.py --performance # Run only performance validator
+ python scripts/validate_all.py --architecture # Run only architecture validator
+ python scripts/validate_all.py -v # Verbose output
+ python scripts/validate_all.py --fail-fast # Stop on first failure
+ python scripts/validate_all.py --json # JSON output
+
+Options:
+ --architecture Run architecture validator
+ --security Run security validator
+ --performance Run performance validator
+ --fail-fast Stop on first validator failure
+ -v, --verbose Show detailed output
+ --errors-only Only show errors
+ --json Output results as JSON
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent))
+
+from base_validator import Severity
+
+
+def run_architecture_validator(verbose: bool = False) -> tuple[int, dict]:
+ """Run the architecture validator"""
+ try:
+ # Import dynamically to avoid circular imports
+ sys.path.insert(0, str(Path(__file__).parent.parent))
+ from scripts.validate_architecture import ArchitectureValidator
+
+ config_path = Path.cwd() / ".architecture-rules.yaml"
+ validator = ArchitectureValidator(config_path=config_path, verbose=verbose)
+ result = validator.validate_all()
+
+ return (
+ 1 if result.has_errors() else 0,
+ {
+ "name": "Architecture",
+ "files_checked": result.files_checked,
+ "errors": sum(1 for v in result.violations if v.severity.value == "error"),
+ "warnings": sum(1 for v in result.violations if v.severity.value == "warning"),
+ "info": sum(1 for v in result.violations if v.severity.value == "info"),
+ }
+ )
+ except ImportError as e:
+ print(f"⚠️ Architecture validator not available: {e}")
+ return 0, {"name": "Architecture", "skipped": True}
+ except Exception as e:
+ print(f"❌ Architecture validator failed: {e}")
+ return 1, {"name": "Architecture", "error": str(e)}
+
+
+def run_security_validator(verbose: bool = False) -> tuple[int, dict]:
+ """Run the security validator"""
+ try:
+ from validate_security import SecurityValidator
+
+ validator = SecurityValidator(verbose=verbose)
+ result = validator.validate_all()
+
+ return (
+ 1 if result.has_errors() else 0,
+ {
+ "name": "Security",
+ "files_checked": result.files_checked,
+ "errors": result.error_count(),
+ "warnings": result.warning_count(),
+ "info": result.info_count(),
+ }
+ )
+ except ImportError as e:
+ print(f"⚠️ Security validator not available: {e}")
+ return 0, {"name": "Security", "skipped": True}
+ except Exception as e:
+ print(f"❌ Security validator failed: {e}")
+ return 1, {"name": "Security", "error": str(e)}
+
+
+def run_performance_validator(verbose: bool = False) -> tuple[int, dict]:
+ """Run the performance validator"""
+ try:
+ from validate_performance import PerformanceValidator
+
+ validator = PerformanceValidator(verbose=verbose)
+ result = validator.validate_all()
+
+ return (
+ 1 if result.has_errors() else 0,
+ {
+ "name": "Performance",
+ "files_checked": result.files_checked,
+ "errors": result.error_count(),
+ "warnings": result.warning_count(),
+ "info": result.info_count(),
+ }
+ )
+ except ImportError as e:
+ print(f"⚠️ Performance validator not available: {e}")
+ return 0, {"name": "Performance", "skipped": True}
+ except Exception as e:
+ print(f"❌ Performance validator failed: {e}")
+ return 1, {"name": "Performance", "error": str(e)}
+
+
+def print_summary(results: list[dict], json_output: bool = False):
+ """Print validation summary"""
+ if json_output:
+ print(json.dumps({"validators": results}, indent=2))
+ return
+
+ print("\n" + "=" * 80)
+ print("📊 UNIFIED VALIDATION SUMMARY")
+ print("=" * 80)
+
+ total_errors = 0
+ total_warnings = 0
+ total_info = 0
+
+ for result in results:
+ if result.get("skipped"):
+ print(f"\n⏭️ {result['name']}: Skipped")
+ elif result.get("error"):
+ print(f"\n❌ {result['name']}: Error - {result['error']}")
+ else:
+ errors = result.get("errors", 0)
+ warnings = result.get("warnings", 0)
+ info = result.get("info", 0)
+ total_errors += errors
+ total_warnings += warnings
+ total_info += info
+
+ status = "✅" if errors == 0 else "❌"
+ print(f"\n{status} {result['name']}:")
+ print(f" Files: {result.get('files_checked', 0)}")
+ print(f" Errors: {errors}, Warnings: {warnings}, Info: {info}")
+
+ print("\n" + "-" * 80)
+ print(f"TOTAL: {total_errors} errors, {total_warnings} warnings, {total_info} info")
+ print("=" * 80)
+
+ if total_errors > 0:
+ print("❌ VALIDATION FAILED")
+ elif total_warnings > 0:
+ print(f"⚠️ VALIDATION PASSED WITH {total_warnings} WARNING(S)")
+ else:
+ print("✅ VALIDATION PASSED")
+ print("=" * 80)
+
+
+def main():
+ parser = argparse.ArgumentParser(
+ description="Unified code validator - runs architecture, security, and performance checks",
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ )
+ parser.add_argument("--architecture", action="store_true", help="Run architecture validator")
+ parser.add_argument("--security", action="store_true", help="Run security validator")
+ parser.add_argument("--performance", action="store_true", help="Run performance validator")
+ parser.add_argument("--fail-fast", action="store_true", help="Stop on first failure")
+ parser.add_argument("-v", "--verbose", action="store_true", help="Verbose output")
+ parser.add_argument("--errors-only", action="store_true", help="Only show errors")
+ parser.add_argument("--json", action="store_true", help="JSON output")
+
+ args = parser.parse_args()
+
+ # If no specific validators selected, run all
+ run_all = not (args.architecture or args.security or args.performance)
+
+ print("\n🔍 UNIFIED CODE VALIDATION")
+ print("=" * 80)
+
+ validators = []
+ if run_all or args.architecture:
+ validators.append(("Architecture", run_architecture_validator))
+ if run_all or args.security:
+ validators.append(("Security", run_security_validator))
+ if run_all or args.performance:
+ validators.append(("Performance", run_performance_validator))
+
+ results = []
+ exit_code = 0
+
+ for name, validator_func in validators:
+ print(f"\n{'=' * 40}")
+ print(f"🔍 Running {name} Validator...")
+ print("=" * 40)
+
+ code, result = validator_func(verbose=args.verbose)
+
+ results.append(result)
+
+ if code != 0:
+ exit_code = 1
+ if args.fail_fast:
+ print(f"\n❌ {name} validator failed. Stopping (--fail-fast)")
+ break
+
+ print_summary(results, json_output=args.json)
+ sys.exit(exit_code)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/validate_performance.py b/scripts/validate_performance.py
new file mode 100755
index 00000000..d7da7603
--- /dev/null
+++ b/scripts/validate_performance.py
@@ -0,0 +1,648 @@
+#!/usr/bin/env python3
+"""
+Performance Validator
+=====================
+Validates code against performance rules defined in .performance-rules/
+
+This script checks for common performance issues:
+- N+1 query patterns
+- Missing pagination
+- Inefficient database operations
+- Memory management issues
+- Frontend performance anti-patterns
+- Missing timeouts and connection pooling
+
+Usage:
+ python scripts/validate_performance.py # Check all files
+ python scripts/validate_performance.py -d app/services/ # Check specific directory
+ python scripts/validate_performance.py -f app/api/v1/products.py # Check single file
+ python scripts/validate_performance.py -v # Verbose output
+ python scripts/validate_performance.py --json # JSON output
+ python scripts/validate_performance.py --errors-only # Only show errors
+
+Options:
+ -f, --file PATH Validate a single file
+ -d, --folder PATH Validate all files in a directory (recursive)
+ -v, --verbose Show detailed output including context
+ --errors-only Only show errors, suppress warnings and info
+ --json Output results as JSON
+"""
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent))
+
+from base_validator import BaseValidator, Severity, ValidationResult
+
+
+class PerformanceValidator(BaseValidator):
+ """Performance-focused code validator"""
+
+ VALIDATOR_NAME = "Performance Validator"
+ VALIDATOR_EMOJI = "⚡"
+ RULES_DIR_NAME = ".performance-rules"
+ CONFIG_FILE_NAME = ".performance-rules.yaml"
+
+ def validate_all(self, target_path: Path = None) -> ValidationResult:
+ """Validate all files for performance issues"""
+ print(f"\n{self.VALIDATOR_EMOJI} Starting performance validation...\n")
+
+ target = target_path or self.project_root
+
+ # Validate Python files
+ self._validate_python_files(target)
+
+ # Validate JavaScript files
+ self._validate_javascript_files(target)
+
+ # Validate HTML templates
+ self._validate_template_files(target)
+
+ return self.result
+
+ def _validate_python_files(self, target: Path):
+ """Validate all Python files for performance issues"""
+ print("🐍 Validating Python files...")
+
+ for py_file in target.rglob("*.py"):
+ if self._should_ignore_file(py_file):
+ continue
+
+ self.result.files_checked += 1
+ content = py_file.read_text()
+ lines = content.split("\n")
+ self._validate_python_performance(py_file, content, lines)
+
+ def _validate_javascript_files(self, target: Path):
+ """Validate all JavaScript files for performance issues"""
+ print("🟨 Validating JavaScript files...")
+
+ for js_file in target.rglob("*.js"):
+ if self._should_ignore_file(js_file):
+ continue
+
+ self.result.files_checked += 1
+ content = js_file.read_text()
+ lines = content.split("\n")
+ self._validate_javascript_performance(js_file, content, lines)
+
+ def _validate_template_files(self, target: Path):
+ """Validate all HTML template files for performance issues"""
+ print("📄 Validating template files...")
+
+ for html_file in target.rglob("*.html"):
+ if self._should_ignore_file(html_file):
+ continue
+
+ self.result.files_checked += 1
+ content = html_file.read_text()
+ lines = content.split("\n")
+ self._validate_template_performance(html_file, content, lines)
+
+ def _validate_file_content(self, file_path: Path, content: str, lines: list[str]):
+ """Validate file content based on file type"""
+ if file_path.suffix == ".py":
+ self._validate_python_performance(file_path, content, lines)
+ elif file_path.suffix == ".js":
+ self._validate_javascript_performance(file_path, content, lines)
+ elif file_path.suffix == ".html":
+ self._validate_template_performance(file_path, content, lines)
+
+ def _validate_python_performance(self, file_path: Path, content: str, lines: list[str]):
+ """Validate Python file for performance issues"""
+ file_path_str = str(file_path)
+
+ # PERF-001: N+1 query detection
+ self._check_n_plus_1_queries(file_path, content, lines)
+
+ # PERF-003: Query result limiting
+ self._check_query_limiting(file_path, content, lines)
+
+ # PERF-006: Bulk operations
+ self._check_bulk_operations(file_path, content, lines)
+
+ # PERF-008: Use EXISTS for existence checks
+ self._check_existence_checks(file_path, content, lines)
+
+ # PERF-009: Batch updates
+ self._check_batch_updates(file_path, content, lines)
+
+ # PERF-026: Pagination for API endpoints
+ if "/api/" in file_path_str:
+ self._check_api_pagination(file_path, content, lines)
+
+ # PERF-037: Parallel async operations
+ self._check_parallel_async(file_path, content, lines)
+
+ # PERF-040: Timeout configuration
+ self._check_timeout_config(file_path, content, lines)
+
+ # PERF-046: Generators for large datasets
+ self._check_generators(file_path, content, lines)
+
+ # PERF-047: Stream file uploads
+ if "upload" in file_path_str.lower() or "file" in file_path_str.lower():
+ self._check_file_streaming(file_path, content, lines)
+
+ # PERF-048: Chunked processing
+ if "import" in file_path_str.lower() or "csv" in file_path_str.lower():
+ self._check_chunked_processing(file_path, content, lines)
+
+ # PERF-049: Context managers for files
+ self._check_context_managers(file_path, content, lines)
+
+ # PERF-051: String concatenation
+ self._check_string_concatenation(file_path, content, lines)
+
+ def _validate_javascript_performance(self, file_path: Path, content: str, lines: list[str]):
+ """Validate JavaScript file for performance issues"""
+ # PERF-056: Debounce search inputs
+ self._check_debounce(file_path, content, lines)
+
+ # PERF-062: Polling intervals
+ self._check_polling_intervals(file_path, content, lines)
+
+ # PERF-064: Layout thrashing
+ self._check_layout_thrashing(file_path, content, lines)
+
+ def _validate_template_performance(self, file_path: Path, content: str, lines: list[str]):
+ """Validate HTML template file for performance issues"""
+ # PERF-058: Image lazy loading
+ self._check_image_lazy_loading(file_path, content, lines)
+
+ # PERF-067: Script defer/async
+ self._check_script_loading(file_path, content, lines)
+
+ # =========================================================================
+ # Database Performance Checks
+ # =========================================================================
+
+ def _check_n_plus_1_queries(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-001: Check for N+1 query patterns"""
+ # Look for patterns like: for item in items: item.relationship.attribute
+ in_for_loop = False
+ for_line_num = 0
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ # Track for loops over query results
+ if re.search(r'for\s+\w+\s+in\s+.*\.(all|query)', line):
+ in_for_loop = True
+ for_line_num = i
+ elif in_for_loop and stripped and not stripped.startswith("#"):
+ # Check for relationship access in loop
+ if re.search(r'\.\w+\.\w+', line) and "(" not in line:
+ # Could be accessing a relationship
+ if any(rel in line for rel in [".customer.", ".vendor.", ".order.", ".product.", ".user."]):
+ self._add_violation(
+ rule_id="PERF-001",
+ rule_name="N+1 query detection",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Possible N+1 query - relationship accessed in loop",
+ context=line.strip()[:80],
+ suggestion="Use joinedload() or selectinload() for eager loading",
+ )
+ in_for_loop = False
+
+ # Reset on dedent
+ if in_for_loop and line and not line.startswith(" " * 4) and i > for_line_num + 1:
+ in_for_loop = False
+
+ def _check_query_limiting(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-003: Check for unbounded query results"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'\.all\(\)', line):
+ # Check if there's a limit or filter before
+ context_start = max(0, i - 5)
+ context_lines = lines[context_start:i]
+ context_text = "\n".join(context_lines)
+
+ if "limit" not in context_text.lower() and "filter" not in context_text.lower():
+ if "# noqa" in line or "# bounded" in line:
+ continue
+ self._add_violation(
+ rule_id="PERF-003",
+ rule_name="Query result limiting",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Query may return unbounded results",
+ context=line.strip()[:80],
+ suggestion="Add .limit() or pagination for large tables",
+ )
+
+ def _check_bulk_operations(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-006: Check for individual operations in loops"""
+ in_for_loop = False
+ for_indent = 0
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ # Track for loops
+ if re.search(r'for\s+\w+\s+in\s+', line):
+ in_for_loop = True
+ for_indent = len(line) - len(line.lstrip())
+ elif in_for_loop:
+ current_indent = len(line) - len(line.lstrip()) if line.strip() else for_indent + 4
+
+ if current_indent <= for_indent and stripped:
+ in_for_loop = False
+ elif "db.add(" in line or ".save(" in line:
+ self._add_violation(
+ rule_id="PERF-006",
+ rule_name="Bulk operations for multiple records",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Individual db.add() in loop - consider bulk operations",
+ context=line.strip()[:80],
+ suggestion="Use db.add_all() or bulk_insert_mappings()",
+ )
+
+ def _check_existence_checks(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-008: Check for inefficient existence checks"""
+ patterns = [
+ (r'\.count\(\)\s*>\s*0', "count() > 0"),
+ (r'\.count\(\)\s*>=\s*1', "count() >= 1"),
+ (r'\.count\(\)\s*!=\s*0', "count() != 0"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line):
+ self._add_violation(
+ rule_id="PERF-008",
+ rule_name="Use EXISTS for existence checks",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message=f"{issue} scans all rows - use EXISTS instead",
+ context=line.strip()[:80],
+ suggestion="Use db.scalar(exists().where(...)) or .first() is not None",
+ )
+
+ def _check_batch_updates(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-009: Check for updates in loops"""
+ in_for_loop = False
+ for_indent = 0
+ loop_var = ""
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ # Track for loops
+ match = re.search(r'for\s+(\w+)\s+in\s+', line)
+ if match:
+ in_for_loop = True
+ for_indent = len(line) - len(line.lstrip())
+ loop_var = match.group(1)
+ elif in_for_loop:
+ current_indent = len(line) - len(line.lstrip()) if line.strip() else for_indent + 4
+
+ if current_indent <= for_indent and stripped:
+ in_for_loop = False
+ elif loop_var and f"{loop_var}." in line and "=" in line and "==" not in line:
+ # Attribute assignment in loop
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-009",
+ rule_name="Batch updates instead of loops",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Individual updates in loop - consider batch update",
+ context=line.strip()[:80],
+ suggestion="Use .update({...}) with filters for batch updates",
+ )
+
+ # =========================================================================
+ # API Performance Checks
+ # =========================================================================
+
+ def _check_api_pagination(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-026: Check for missing pagination in list endpoints"""
+ # Look for GET endpoints that return lists
+ in_endpoint = False
+ endpoint_line = 0
+ has_pagination = False
+
+ for i, line in enumerate(lines, 1):
+ # Track router decorators
+ if re.search(r'@router\.(get|post)', line):
+ in_endpoint = True
+ endpoint_line = i
+ has_pagination = False
+ elif in_endpoint:
+ # Check for pagination parameters
+ if re.search(r'(skip|offset|page|limit)', line):
+ has_pagination = True
+ # Check for function end
+ if re.search(r'^def\s+\w+', line.lstrip()) and i > endpoint_line + 1:
+ in_endpoint = False
+ # Check for .all() without pagination
+ if ".all()" in line and not has_pagination:
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-026",
+ rule_name="Pagination required for list endpoints",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="List endpoint may lack pagination",
+ context=line.strip()[:80],
+ suggestion="Add skip/limit parameters for pagination",
+ )
+
+ # =========================================================================
+ # Async Performance Checks
+ # =========================================================================
+
+ def _check_parallel_async(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-037: Check for sequential awaits that could be parallel"""
+ await_count = 0
+ await_lines = []
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ if stripped.startswith("await "):
+ await_count += 1
+ await_lines.append(i)
+
+ # Check for 3+ sequential awaits
+ if await_count >= 3:
+ # Verify they're sequential (within 5 lines of each other)
+ if all(await_lines[j+1] - await_lines[j] <= 2 for j in range(len(await_lines)-1)):
+ self._add_violation(
+ rule_id="PERF-037",
+ rule_name="Parallel independent operations",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=await_lines[0],
+ message=f"{await_count} sequential awaits - consider asyncio.gather()",
+ context="Multiple await statements",
+ suggestion="Use asyncio.gather() for independent async operations",
+ )
+ await_count = 0
+ await_lines = []
+ elif stripped and not stripped.startswith("#"):
+ # Reset on non-await, non-empty line
+ if await_count > 0:
+ await_count = 0
+ await_lines = []
+
+ def _check_timeout_config(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-040: Check for missing timeouts on HTTP clients"""
+ if "requests" not in content and "httpx" not in content and "aiohttp" not in content:
+ return
+
+ patterns = [
+ r'requests\.(get|post|put|delete|patch)\s*\([^)]+\)',
+ r'httpx\.(get|post|put|delete|patch)\s*\([^)]+\)',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line) and "timeout" not in line:
+ self._add_violation(
+ rule_id="PERF-040",
+ rule_name="Timeout configuration",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="HTTP request without timeout",
+ context=line.strip()[:80],
+ suggestion="Add timeout parameter to prevent hanging requests",
+ )
+
+ # =========================================================================
+ # Memory Performance Checks
+ # =========================================================================
+
+ def _check_generators(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-046: Check for loading large datasets into memory"""
+ for i, line in enumerate(lines, 1):
+ # Check for .all() followed by iteration
+ if ".all()" in line:
+ # Look ahead for iteration
+ if i < len(lines):
+ next_lines = "\n".join(lines[i:min(i+3, len(lines))])
+ if "for " in next_lines and "in" in next_lines:
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-046",
+ rule_name="Generators for large datasets",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message=".all() loads everything into memory before iteration",
+ context=line.strip()[:80],
+ suggestion="Use .yield_per(100) for large result sets",
+ )
+
+ def _check_file_streaming(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-047: Check for loading entire files into memory"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'await\s+\w+\.read\(\)', line) and "chunk" not in line:
+ self._add_violation(
+ rule_id="PERF-047",
+ rule_name="Stream large file uploads",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Full file read into memory",
+ context=line.strip()[:80],
+ suggestion="Stream large files: while chunk := await file.read(8192)",
+ )
+
+ def _check_chunked_processing(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-048: Check for chunked processing in imports"""
+ if "chunk" not in content.lower() and "batch" not in content.lower():
+ # Check if file processes multiple records
+ if "for " in content and ("csv" in content.lower() or "import" in content.lower()):
+ self._add_violation(
+ rule_id="PERF-048",
+ rule_name="Chunked processing for imports",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=1,
+ message="Import processing may benefit from chunking",
+ context="File processes multiple records",
+ suggestion="Process in chunks with periodic commits",
+ )
+
+ def _check_context_managers(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-049: Check for file handles without context managers"""
+ for i, line in enumerate(lines, 1):
+ # Check for file open without 'with'
+ if re.search(r'^\s*\w+\s*=\s*open\s*\(', line):
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-049",
+ rule_name="Context managers for resources",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="File opened without context manager",
+ context=line.strip()[:80],
+ suggestion="Use 'with open(...) as f:' to ensure cleanup",
+ )
+
+ def _check_string_concatenation(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-051: Check for inefficient string concatenation in loops"""
+ in_for_loop = False
+ for_indent = 0
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ if re.search(r'for\s+\w+\s+in\s+', line):
+ in_for_loop = True
+ for_indent = len(line) - len(line.lstrip())
+ elif in_for_loop:
+ current_indent = len(line) - len(line.lstrip()) if line.strip() else for_indent + 4
+
+ if current_indent <= for_indent and stripped:
+ in_for_loop = False
+ elif re.search(r'\w+\s*\+=\s*["\']|str\s*\(', line):
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-051",
+ rule_name="String concatenation efficiency",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="String concatenation in loop",
+ context=line.strip()[:80],
+ suggestion="Use ''.join() or StringIO for many concatenations",
+ )
+
+ # =========================================================================
+ # Frontend Performance Checks
+ # =========================================================================
+
+ def _check_debounce(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-056: Check for search inputs without debounce"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'@(input|keyup)=".*search.*fetch', line, re.IGNORECASE):
+ if "debounce" not in content.lower():
+ self._add_violation(
+ rule_id="PERF-056",
+ rule_name="Debounce search inputs",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Search input triggers API call without debounce",
+ context=line.strip()[:80],
+ suggestion="Add 300-500ms debounce to prevent excessive API calls",
+ )
+
+ def _check_polling_intervals(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-062: Check for too-frequent polling"""
+ for i, line in enumerate(lines, 1):
+ match = re.search(r'setInterval\s*\([^,]+,\s*(\d+)\s*\)', line)
+ if match:
+ interval = int(match.group(1))
+ if interval < 10000: # Less than 10 seconds
+ if "# real-time" not in line and "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-062",
+ rule_name="Reasonable polling intervals",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message=f"Polling interval {interval}ms is very frequent",
+ context=line.strip()[:80],
+ suggestion="Use >= 10 second intervals for non-critical updates",
+ )
+
+ def _check_layout_thrashing(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-064: Check for layout thrashing patterns"""
+ for i, line in enumerate(lines, 1):
+ # Check for read then write patterns
+ if re.search(r'(offsetHeight|offsetWidth|clientHeight|clientWidth)', line):
+ if i < len(lines):
+ next_line = lines[i] if i < len(lines) else ""
+ if "style" in next_line:
+ self._add_violation(
+ rule_id="PERF-064",
+ rule_name="Avoid layout thrashing",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="DOM read followed by write can cause layout thrashing",
+ context=line.strip()[:80],
+ suggestion="Batch DOM reads, then batch DOM writes",
+ )
+
+ def _check_image_lazy_loading(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-058: Check for images without lazy loading"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r']*src=', line):
+ if 'loading="lazy"' not in line and "x-intersect" not in line:
+ if "logo" not in line.lower() and "icon" not in line.lower():
+ self._add_violation(
+ rule_id="PERF-058",
+ rule_name="Image optimization",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Image without lazy loading",
+ context=line.strip()[:80],
+ suggestion='Add loading="lazy" for off-screen images',
+ )
+
+ def _check_script_loading(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-067: Check for script tags without defer/async"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r']*src=', line):
+ if "defer" not in line and "async" not in line:
+ if "alpine" not in line.lower() and "htmx" not in line.lower():
+ self._add_violation(
+ rule_id="PERF-067",
+ rule_name="Defer non-critical JavaScript",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Script tag without defer/async",
+ context=line.strip()[:80],
+ suggestion="Add defer for non-critical scripts",
+ )
+
+
+def main():
+ parser = argparse.ArgumentParser(
+ description="Performance code validator",
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ )
+ parser.add_argument("-f", "--file", type=Path, help="Validate a single file")
+ parser.add_argument("-d", "--folder", type=Path, help="Validate a directory")
+ parser.add_argument("-v", "--verbose", action="store_true", help="Verbose output")
+ parser.add_argument("--errors-only", action="store_true", help="Only show errors")
+ parser.add_argument("--json", action="store_true", help="JSON output")
+
+ args = parser.parse_args()
+
+ validator = PerformanceValidator(verbose=args.verbose)
+
+ if args.file:
+ validator.validate_file(args.file)
+ elif args.folder:
+ validator.validate_all(args.folder)
+ else:
+ validator.validate_all()
+
+ validator.output_results(json_output=args.json, errors_only=args.errors_only)
+ sys.exit(validator.get_exit_code())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/validate_security.py b/scripts/validate_security.py
new file mode 100755
index 00000000..753b8c90
--- /dev/null
+++ b/scripts/validate_security.py
@@ -0,0 +1,718 @@
+#!/usr/bin/env python3
+"""
+Security Validator
+==================
+Validates code against security rules defined in .security-rules/
+
+This script checks for common security vulnerabilities:
+- Hardcoded credentials and secrets
+- SQL injection patterns
+- Command injection risks
+- XSS vulnerabilities
+- Insecure cryptography
+- Authentication weaknesses
+- Data exposure risks
+
+Usage:
+ python scripts/validate_security.py # Check all files
+ python scripts/validate_security.py -d app/api/ # Check specific directory
+ python scripts/validate_security.py -f app/api/v1/auth.py # Check single file
+ python scripts/validate_security.py -v # Verbose output
+ python scripts/validate_security.py --json # JSON output
+ python scripts/validate_security.py --errors-only # Only show errors
+
+Options:
+ -f, --file PATH Validate a single file
+ -d, --folder PATH Validate all files in a directory (recursive)
+ -v, --verbose Show detailed output including context
+ --errors-only Only show errors, suppress warnings and info
+ --json Output results as JSON
+"""
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent))
+
+from base_validator import BaseValidator, Severity, ValidationResult
+
+
+class SecurityValidator(BaseValidator):
+ """Security-focused code validator"""
+
+ VALIDATOR_NAME = "Security Validator"
+ VALIDATOR_EMOJI = "🔒"
+ RULES_DIR_NAME = ".security-rules"
+ CONFIG_FILE_NAME = ".security-rules.yaml"
+
+ def validate_all(self, target_path: Path = None) -> ValidationResult:
+ """Validate all files for security issues"""
+ print(f"\n{self.VALIDATOR_EMOJI} Starting security validation...\n")
+
+ target = target_path or self.project_root
+
+ # Validate Python files
+ self._validate_python_files(target)
+
+ # Validate JavaScript files
+ self._validate_javascript_files(target)
+
+ # Validate HTML templates
+ self._validate_template_files(target)
+
+ # Validate configuration files
+ self._validate_config_files(target)
+
+ return self.result
+
+ def _validate_python_files(self, target: Path):
+ """Validate all Python files for security issues"""
+ print("🐍 Validating Python files...")
+
+ for py_file in target.rglob("*.py"):
+ if self._should_ignore_file(py_file):
+ continue
+
+ self.result.files_checked += 1
+ content = py_file.read_text()
+ lines = content.split("\n")
+ self._validate_python_security(py_file, content, lines)
+
+ def _validate_javascript_files(self, target: Path):
+ """Validate all JavaScript files for security issues"""
+ print("🟨 Validating JavaScript files...")
+
+ for js_file in target.rglob("*.js"):
+ if self._should_ignore_file(js_file):
+ continue
+
+ self.result.files_checked += 1
+ content = js_file.read_text()
+ lines = content.split("\n")
+ self._validate_javascript_security(js_file, content, lines)
+
+ def _validate_template_files(self, target: Path):
+ """Validate all HTML template files for security issues"""
+ print("📄 Validating template files...")
+
+ for html_file in target.rglob("*.html"):
+ if self._should_ignore_file(html_file):
+ continue
+
+ self.result.files_checked += 1
+ content = html_file.read_text()
+ lines = content.split("\n")
+ self._validate_template_security(html_file, content, lines)
+
+ def _validate_config_files(self, target: Path):
+ """Validate configuration files for security issues"""
+ print("⚙️ Validating configuration files...")
+
+ config_patterns = ["*.yaml", "*.yml", "*.json", "*.toml", "*.ini", "*.env*"]
+ for pattern in config_patterns:
+ for config_file in target.rglob(pattern):
+ if self._should_ignore_file(config_file):
+ continue
+ if config_file.suffix in [".yaml", ".yml", ".json"]:
+ self.result.files_checked += 1
+ content = config_file.read_text()
+ lines = content.split("\n")
+ self._validate_config_security(config_file, content, lines)
+
+ def _validate_file_content(self, file_path: Path, content: str, lines: list[str]):
+ """Validate file content based on file type"""
+ if file_path.suffix == ".py":
+ self._validate_python_security(file_path, content, lines)
+ elif file_path.suffix == ".js":
+ self._validate_javascript_security(file_path, content, lines)
+ elif file_path.suffix == ".html":
+ self._validate_template_security(file_path, content, lines)
+ elif file_path.suffix in [".yaml", ".yml", ".json"]:
+ self._validate_config_security(file_path, content, lines)
+
+ def _validate_python_security(self, file_path: Path, content: str, lines: list[str]):
+ """Validate Python file for security issues"""
+ file_path_str = str(file_path)
+
+ # SEC-001: Hardcoded credentials
+ self._check_hardcoded_credentials(file_path, content, lines)
+
+ # SEC-011: SQL injection
+ self._check_sql_injection(file_path, content, lines)
+
+ # SEC-012: Command injection
+ self._check_command_injection(file_path, content, lines)
+
+ # SEC-013: Code execution
+ self._check_code_execution(file_path, content, lines)
+
+ # SEC-014: Path traversal
+ if "upload" in file_path_str.lower() or "file" in file_path_str.lower():
+ self._check_path_traversal(file_path, content, lines)
+
+ # SEC-020: Unsafe deserialization
+ self._check_unsafe_deserialization(file_path, content, lines)
+
+ # SEC-021: PII logging
+ self._check_pii_logging(file_path, content, lines)
+
+ # SEC-024: Error information leakage
+ self._check_error_leakage(file_path, content, lines)
+
+ # SEC-034: HTTPS enforcement
+ self._check_https_enforcement(file_path, content, lines)
+
+ # SEC-040: Timeout configuration
+ self._check_timeout_configuration(file_path, content, lines)
+
+ # SEC-041: Weak hashing
+ self._check_weak_hashing(file_path, content, lines)
+
+ # SEC-042: Insecure random
+ self._check_insecure_random(file_path, content, lines)
+
+ # SEC-043: Hardcoded encryption keys
+ self._check_hardcoded_keys(file_path, content, lines)
+
+ # SEC-047: Certificate verification
+ self._check_certificate_verification(file_path, content, lines)
+
+ # Auth file specific checks
+ if "auth" in file_path_str.lower():
+ self._check_jwt_expiry(file_path, content, lines)
+
+ def _validate_javascript_security(self, file_path: Path, content: str, lines: list[str]):
+ """Validate JavaScript file for security issues"""
+ # SEC-022: Sensitive data in URLs
+ self._check_sensitive_url_params_js(file_path, content, lines)
+
+ # Check for eval usage
+ for i, line in enumerate(lines, 1):
+ if re.search(r'\beval\s*\(', line) and "//" not in line.split("eval")[0]:
+ self._add_violation(
+ rule_id="SEC-013",
+ rule_name="No code execution",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message="eval() allows arbitrary code execution",
+ context=line.strip()[:80],
+ suggestion="Use JSON.parse() for JSON or other safe alternatives",
+ )
+
+ # Check for innerHTML with user input
+ for i, line in enumerate(lines, 1):
+ if re.search(r'\.innerHTML\s*=', line) and "//" not in line.split("innerHTML")[0]:
+ self._add_violation(
+ rule_id="SEC-015",
+ rule_name="XSS prevention",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="innerHTML can lead to XSS if used with untrusted input",
+ context=line.strip()[:80],
+ suggestion="Use textContent for text or sanitize HTML input",
+ )
+
+ def _validate_template_security(self, file_path: Path, content: str, lines: list[str]):
+ """Validate HTML template file for security issues"""
+ # SEC-015: XSS via |safe filter
+ for i, line in enumerate(lines, 1):
+ if re.search(r'\|\s*safe(?!\s*[{#].*sanitized)', line):
+ self._add_violation(
+ rule_id="SEC-015",
+ rule_name="XSS prevention in templates",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="|safe filter disables auto-escaping - ensure content is sanitized",
+ context=line.strip()[:80],
+ suggestion="Mark with {# sanitized #} comment if content is sanitized",
+ )
+
+ # Check for x-html with dynamic content
+ for i, line in enumerate(lines, 1):
+ if re.search(r'x-html="[^"]*\w', line) and "sanitized" not in line.lower():
+ self._add_violation(
+ rule_id="SEC-015",
+ rule_name="XSS prevention in templates",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="x-html renders raw HTML - ensure content is safe",
+ context=line.strip()[:80],
+ suggestion="Use x-text for text content or sanitize HTML",
+ )
+
+ def _validate_config_security(self, file_path: Path, content: str, lines: list[str]):
+ """Validate configuration file for security issues"""
+ # Check for hardcoded secrets in config
+ secret_patterns = [
+ (r'password\s*[=:]\s*["\'][^"\']{4,}["\']', "password"),
+ (r'secret\s*[=:]\s*["\'][^"\']{8,}["\']', "secret"),
+ (r'api_key\s*[=:]\s*["\'][A-Za-z0-9_-]{16,}["\']', "API key"),
+ (r'token\s*[=:]\s*["\'][A-Za-z0-9._-]{20,}["\']', "token"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ # Skip comments
+ stripped = line.strip()
+ if stripped.startswith("#") or stripped.startswith("//"):
+ continue
+
+ for pattern, secret_type in secret_patterns:
+ if re.search(pattern, line, re.IGNORECASE):
+ # Check for environment variable references
+ if "${" in line or "os.getenv" in line or "environ" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-001",
+ rule_name="No hardcoded credentials",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"Possible hardcoded {secret_type} in configuration",
+ context=line.strip()[:60] + "...",
+ suggestion="Use environment variables for secrets",
+ )
+
+ # =========================================================================
+ # Specific Security Checks
+ # =========================================================================
+
+ def _check_hardcoded_credentials(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-001: Check for hardcoded credentials"""
+ patterns = [
+ (r'password\s*=\s*["\'][^"\']{4,}["\']', "password"),
+ (r'api_key\s*=\s*["\'][A-Za-z0-9_-]{16,}["\']', "API key"),
+ (r'secret_key\s*=\s*["\'][^"\']{8,}["\']', "secret key"),
+ (r'auth_token\s*=\s*["\'][A-Za-z0-9._-]{20,}["\']', "auth token"),
+ (r'AWS_SECRET.*=\s*["\'][^"\']+["\']', "AWS secret"),
+ (r'STRIPE_.*KEY.*=\s*["\'][^"\']+["\']', "Stripe key"),
+ ]
+
+ exclude_patterns = [
+ "os.getenv", "os.environ", "settings.", '""', "''",
+ "# noqa", "# test", "password_hash", "example"
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, secret_type in patterns:
+ if re.search(pattern, line, re.IGNORECASE):
+ # Check exclusions
+ if any(exc in line for exc in exclude_patterns):
+ continue
+ self._add_violation(
+ rule_id="SEC-001",
+ rule_name="No hardcoded credentials",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"Possible hardcoded {secret_type}",
+ context=line.strip()[:60] + "...",
+ suggestion="Use environment variables or secret management",
+ )
+
+ def _check_sql_injection(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-011: Check for SQL injection vulnerabilities"""
+ patterns = [
+ r'execute\s*\(\s*f["\']',
+ r'execute\s*\([^)]*\s*\+\s*',
+ r'execute\s*\([^)]*%[^)]*%',
+ r'text\s*\(\s*f["\']',
+ r'\.raw\s*\(\s*f["\']',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line):
+ if "# noqa" in line or "# safe" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-011",
+ rule_name="No raw SQL queries",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message="Possible SQL injection - use parameterized queries",
+ context=line.strip()[:80],
+ suggestion="Use SQLAlchemy ORM or parameterized queries with :param syntax",
+ )
+
+ def _check_command_injection(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-012: Check for command injection vulnerabilities"""
+ patterns = [
+ (r'subprocess.*shell\s*=\s*True', "shell=True in subprocess"),
+ (r'os\.system\s*\(', "os.system()"),
+ (r'os\.popen\s*\(', "os.popen()"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line):
+ if "# noqa" in line or "# safe" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-012",
+ rule_name="No shell command injection",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"{issue} allows command injection",
+ context=line.strip()[:80],
+ suggestion="Use subprocess with list arguments, shell=False",
+ )
+
+ def _check_code_execution(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-013: Check for code execution vulnerabilities"""
+ patterns = [
+ (r'eval\s*\([^)]*request', "eval with request data"),
+ (r'eval\s*\([^)]*input', "eval with user input"),
+ (r'exec\s*\([^)]*request', "exec with request data"),
+ (r'__import__\s*\([^)]*request', "__import__ with request data"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line, re.IGNORECASE):
+ self._add_violation(
+ rule_id="SEC-013",
+ rule_name="No code execution",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"{issue} allows arbitrary code execution",
+ context=line.strip()[:80],
+ suggestion="Never use eval/exec with user input",
+ )
+
+ def _check_path_traversal(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-014: Check for path traversal vulnerabilities"""
+ # Check if file has path operations with user input
+ has_secure_filename = "secure_filename" in content or "basename" in content
+
+ patterns = [
+ r'open\s*\([^)]*request',
+ r'open\s*\([^)]*\+',
+ r'Path\s*\([^)]*request',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line, re.IGNORECASE):
+ if has_secure_filename:
+ continue
+ self._add_violation(
+ rule_id="SEC-014",
+ rule_name="Path traversal prevention",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Possible path traversal - validate file paths",
+ context=line.strip()[:80],
+ suggestion="Use secure_filename() and validate paths against allowed directories",
+ )
+
+ def _check_unsafe_deserialization(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-020: Check for unsafe deserialization"""
+ patterns = [
+ (r'pickle\.loads?\s*\(', "pickle deserialization"),
+ (r'yaml\.load\s*\([^,)]+\)(?!.*SafeLoader)', "yaml.load without SafeLoader"),
+ (r'marshal\.loads?\s*\(', "marshal deserialization"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line):
+ if "# noqa" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-020",
+ rule_name="Deserialization safety",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"Unsafe {issue} can lead to code execution",
+ context=line.strip()[:80],
+ suggestion="Use json.loads() or yaml.safe_load() instead",
+ )
+
+ def _check_pii_logging(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-021: Check for PII in logs"""
+ patterns = [
+ (r'log\w*\.[a-z]+\([^)]*password', "password in log"),
+ (r'log\w*\.[a-z]+\([^)]*credit_card', "credit card in log"),
+ (r'log\w*\.[a-z]+\([^)]*ssn', "SSN in log"),
+ (r'print\s*\([^)]*password', "password in print"),
+ ]
+
+ exclude = ["password_hash", "password_reset", "password_changed", "# noqa"]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line, re.IGNORECASE):
+ if any(exc in line for exc in exclude):
+ continue
+ self._add_violation(
+ rule_id="SEC-021",
+ rule_name="PII logging prevention",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"Possible {issue}",
+ context=line.strip()[:60] + "...",
+ suggestion="Never log sensitive data - redact or omit",
+ )
+
+ def _check_error_leakage(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-024: Check for error information leakage"""
+ patterns = [
+ r'traceback\.format_exc\(\).*detail',
+ r'traceback\.format_exc\(\).*response',
+ r'str\(e\).*HTTPException',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line):
+ if "logger" in line or "# noqa" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-024",
+ rule_name="Error message information leakage",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Internal error details may be exposed to users",
+ context=line.strip()[:80],
+ suggestion="Log errors internally, return generic message to users",
+ )
+
+ def _check_https_enforcement(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-034: Check for HTTP instead of HTTPS"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'http://(?!localhost|127\.0\.0\.1|0\.0\.0\.0|\$)', line):
+ if "# noqa" in line or "example.com" in line or "schemas" in line:
+ continue
+ if "http://www.w3.org" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-034",
+ rule_name="HTTPS enforcement",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="HTTP URL found - use HTTPS for security",
+ context=line.strip()[:80],
+ suggestion="Replace http:// with https://",
+ )
+
+ def _check_timeout_configuration(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-040: Check for missing timeouts on external calls"""
+ # Check for requests/httpx calls without timeout
+ if "requests" in content or "httpx" in content or "aiohttp" in content:
+ has_timeout_import = "timeout" in content.lower()
+
+ patterns = [
+ r'requests\.(get|post|put|delete|patch)\s*\([^)]+\)(?!.*timeout)',
+ r'httpx\.(get|post|put|delete|patch)\s*\([^)]+\)(?!.*timeout)',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line) and "timeout" not in line:
+ self._add_violation(
+ rule_id="SEC-040",
+ rule_name="Timeout configuration",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="HTTP request without timeout - can hang indefinitely",
+ context=line.strip()[:80],
+ suggestion="Add timeout parameter: requests.get(url, timeout=30)",
+ )
+
+ def _check_weak_hashing(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-041: Check for weak hashing algorithms"""
+ patterns = [
+ (r'hashlib\.md5\s*\(', "MD5"),
+ (r'hashlib\.sha1\s*\(', "SHA1"),
+ (r'MD5\.new\s*\(', "MD5"),
+ (r'SHA\.new\s*\(', "SHA1"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, algo in patterns:
+ if re.search(pattern, line):
+ if "# noqa" in line or "# checksum" in line or "# file hash" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-041",
+ rule_name="Strong hashing algorithms",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message=f"{algo} is cryptographically weak",
+ context=line.strip()[:80],
+ suggestion="Use SHA-256 or stronger for security purposes",
+ )
+
+ def _check_insecure_random(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-042: Check for insecure random number generation"""
+ # Only check if file appears to deal with security
+ security_context = any(
+ word in content.lower()
+ for word in ["token", "secret", "key", "session", "csrf", "nonce", "salt"]
+ )
+
+ if not security_context:
+ return
+
+ patterns = [
+ r'random\.random\s*\(',
+ r'random\.randint\s*\(',
+ r'random\.choice\s*\(',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line):
+ if "# noqa" in line or "# not security" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-042",
+ rule_name="Secure random generation",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="random module is not cryptographically secure",
+ context=line.strip()[:80],
+ suggestion="Use secrets module for security-sensitive randomness",
+ )
+
+ def _check_hardcoded_keys(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-043: Check for hardcoded encryption keys"""
+ patterns = [
+ r'ENCRYPTION_KEY\s*=\s*["\'][^"\']+["\']',
+ r'SECRET_KEY\s*=\s*["\'][A-Za-z0-9+/=]{16,}["\']',
+ r'AES_KEY\s*=\s*["\']',
+ r'PRIVATE_KEY\s*=\s*["\']-----BEGIN',
+ ]
+
+ exclude = ["os.getenv", "os.environ", "settings.", '""', "# test"]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line):
+ if any(exc in line for exc in exclude):
+ continue
+ self._add_violation(
+ rule_id="SEC-043",
+ rule_name="No hardcoded encryption keys",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message="Hardcoded encryption key found",
+ context=line.strip()[:50] + "...",
+ suggestion="Use environment variables for encryption keys",
+ )
+
+ def _check_certificate_verification(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-047: Check for disabled certificate verification"""
+ patterns = [
+ (r'verify\s*=\s*False', "SSL verification disabled"),
+ (r'CERT_NONE', "Certificate verification disabled"),
+ (r'check_hostname\s*=\s*False', "Hostname verification disabled"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line):
+ if "# noqa" in line or "# test" in line or "DEBUG" in line:
+ continue
+ self._add_violation(
+ rule_id="SEC-047",
+ rule_name="Certificate verification",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message=f"{issue} - vulnerable to MITM attacks",
+ context=line.strip()[:80],
+ suggestion="Always verify SSL certificates in production",
+ )
+
+ def _check_jwt_expiry(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-002: Check for JWT tokens without expiry"""
+ if "jwt.encode" in content and "exp" not in content:
+ # Find the jwt.encode line
+ for i, line in enumerate(lines, 1):
+ if "jwt.encode" in line:
+ self._add_violation(
+ rule_id="SEC-002",
+ rule_name="JWT expiry enforcement",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="JWT token may not have expiration claim",
+ context=line.strip()[:80],
+ suggestion="Include 'exp' claim with appropriate expiration",
+ )
+ break
+
+ def _check_sensitive_url_params_js(self, file_path: Path, content: str, lines: list[str]):
+ """SEC-022: Check for sensitive data in URLs (JavaScript)"""
+ patterns = [
+ r'\?password=',
+ r'&password=',
+ r'\?token=(?!type)',
+ r'&token=(?!type)',
+ r'\?api_key=',
+ r'&api_key=',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line):
+ self._add_violation(
+ rule_id="SEC-022",
+ rule_name="Sensitive data in URLs",
+ severity=Severity.ERROR,
+ file_path=file_path,
+ line_number=i,
+ message="Sensitive data in URL query parameters",
+ context=line.strip()[:80],
+ suggestion="Send sensitive data in request body or headers",
+ )
+
+
+def main():
+ parser = argparse.ArgumentParser(
+ description="Security code validator",
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ )
+ parser.add_argument("-f", "--file", type=Path, help="Validate a single file")
+ parser.add_argument("-d", "--folder", type=Path, help="Validate a directory")
+ parser.add_argument("-v", "--verbose", action="store_true", help="Verbose output")
+ parser.add_argument("--errors-only", action="store_true", help="Only show errors")
+ parser.add_argument("--json", action="store_true", help="JSON output")
+
+ args = parser.parse_args()
+
+ validator = SecurityValidator(verbose=args.verbose)
+
+ if args.file:
+ validator.validate_file(args.file)
+ elif args.folder:
+ validator.validate_all(args.folder)
+ else:
+ validator.validate_all()
+
+ validator.output_results(json_output=args.json, errors_only=args.errors_only)
+ sys.exit(validator.get_exit_code())
+
+
+if __name__ == "__main__":
+ main()
diff --git a/static/admin/js/code-quality-violations.js b/static/admin/js/code-quality-violations.js
index 438a41a8..f6385602 100644
--- a/static/admin/js/code-quality-violations.js
+++ b/static/admin/js/code-quality-violations.js
@@ -25,6 +25,7 @@ function codeQualityViolations() {
total_pages: 0
},
filters: {
+ validator_type: '',
severity: '',
status: '',
rule_id: '',
@@ -34,6 +35,7 @@ function codeQualityViolations() {
async init() {
// Load filters from URL params
const params = new URLSearchParams(window.location.search);
+ this.filters.validator_type = params.get('validator_type') || '';
this.filters.severity = params.get('severity') || '';
this.filters.status = params.get('status') || '';
this.filters.rule_id = params.get('rule_id') || '';
@@ -53,6 +55,7 @@ function codeQualityViolations() {
page_size: this.pagination.page_size.toString()
};
+ if (this.filters.validator_type) params.validator_type = this.filters.validator_type;
if (this.filters.severity) params.severity = this.filters.severity;
if (this.filters.status) params.status = this.filters.status;
if (this.filters.rule_id) params.rule_id = this.filters.rule_id;
@@ -168,6 +171,7 @@ function codeQualityViolations() {
updateURL() {
const params = new URLSearchParams();
+ if (this.filters.validator_type) params.set('validator_type', this.filters.validator_type);
if (this.filters.severity) params.set('severity', this.filters.severity);
if (this.filters.status) params.set('status', this.filters.status);
if (this.filters.rule_id) params.set('rule_id', this.filters.rule_id);
diff --git a/static/shared/js/icons.js b/static/shared/js/icons.js
index 616ad1ba..bdf94d30 100644
--- a/static/shared/js/icons.js
+++ b/static/shared/js/icons.js
@@ -167,7 +167,11 @@ const Icons = {
// Trust & Payment Icons
'cash': ``,
'support': ``,
- 'emoji-happy': ``
+ 'emoji-happy': ``,
+
+ // Messaging
+ 'chat-bubble-left-right': ``,
+ 'chat-bubble-left': ``
};
/**
+```
+
+### PERF-059: Minimize Alpine.js Watchers
+**Severity:** Info
+
+Excessive `$watch` calls impact performance. Use computed properties or event handlers instead.
+
+### PERF-060: Virtual Scrolling for Long Lists
+**Severity:** Info
+
+Lists with 100+ items should use virtual scrolling. Only render visible items in the viewport.
+
+### PERF-061: Minimize Bundle Size
+**Severity:** Info
+
+Reduce JavaScript bundle size: import only needed modules, use tree-shaking, split code by route.
+
+### PERF-062: Reasonable Polling Intervals
+**Severity:** Warning
+
+Polling should be >= 10 seconds for non-critical updates.
+
+```javascript
+// Bad
+setInterval(fetchUpdates, 1000); // Every second
+
+// Good
+setInterval(fetchUpdates, 30000); // Every 30 seconds
+```
+
+### PERF-063 to PERF-070: Additional Frontend Rules
+**Severity:** Info/Warning
+
+CSS containment, avoid layout thrashing, CSS animations over JavaScript, preload critical resources, defer non-critical JavaScript, minimize DOM nodes, efficient event handlers, and cache DOM queries.
+
+---
+
+## Configuration
+
+All rules are defined in `.performance-rules/` directory:
+
+```
+.performance-rules/
+├── _main.yaml # Main configuration
+├── database.yaml # PERF-001 to PERF-015
+├── caching.yaml # PERF-016 to PERF-025
+├── api.yaml # PERF-026 to PERF-035
+├── async.yaml # PERF-036 to PERF-045
+├── memory.yaml # PERF-046 to PERF-055
+└── frontend.yaml # PERF-056 to PERF-070
+```
+
+## Suppressing Rules
+
+Use noqa comments to suppress specific rules:
+
+```python
+# noqa: PERF-003 - This is intentionally unbounded for admin export
+products = db.query(Product).all()
+```
+
+## Related Documentation
+
+- [Architecture Rules](architecture-rules.md)
+- [Security Rules](security-rules.md)
+- [Code Quality Guide](code-quality.md)
+- [Contributing Guide](contributing.md)
+
+---
+
+## Summary Statistics
+
+| Category | Rules | Errors | Warnings | Info |
+|----------|-------|--------|----------|------|
+| Database | 15 | 0 | 5 | 10 |
+| Caching | 10 | 0 | 2 | 8 |
+| API | 10 | 1 | 3 | 6 |
+| Async & Concurrency | 10 | 1 | 4 | 5 |
+| Memory Management | 10 | 1 | 4 | 5 |
+| Frontend | 15 | 0 | 4 | 11 |
+| **Total** | **70** | **3** | **22** | **45** |
+
+---
+
+**Last Updated:** 2025-12-21
+**Version:** 1.0
diff --git a/docs/development/security-rules.md b/docs/development/security-rules.md
new file mode 100644
index 00000000..4b7ec185
--- /dev/null
+++ b/docs/development/security-rules.md
@@ -0,0 +1,560 @@
+# Security Rules Reference
+
+This document provides a comprehensive reference for all security rules enforced by the `scripts/validate_security.py` validator.
+
+## Overview
+
+The security validator identifies potential security vulnerabilities and enforces security best practices across the codebase. Rules are organized by category and severity level.
+
+**Version:** 1.0
+**Total Rules:** 60
+**Configuration Directory:** `.security-rules/`
+
+## Running the Validator
+
+### Using Python Directly
+
+```bash
+# Check all files
+python scripts/validate_security.py
+
+# Verbose output
+python scripts/validate_security.py -v
+
+# Errors only
+python scripts/validate_security.py --errors-only
+
+# JSON output (for CI/CD)
+python scripts/validate_security.py --json
+```
+
+### Using the Unified Validator
+
+```bash
+# Run security checks only
+python scripts/validate_all.py --security
+
+# Run all validators
+python scripts/validate_all.py
+```
+
+## Severity Levels
+
+| Severity | Description | Exit Code | Action Required |
+|----------|-------------|-----------|-----------------|
+| **Error** | Critical security vulnerability | 1 | Must fix immediately |
+| **Warning** | Security concern | 0 | Should fix |
+| **Info** | Security suggestion | 0 | Consider implementing |
+
+---
+
+## Authentication Rules (SEC-001 to SEC-010)
+
+### SEC-001: No Hardcoded Credentials
+**Severity:** Error
+
+Credentials must never be hardcoded in source code. Use environment variables or secret management.
+
+```python
+# Bad
+api_key = "sk-1234567890abcdef"
+password = "admin123"
+
+# Good
+api_key = os.environ.get("API_KEY")
+password = settings.admin_password
+```
+
+### SEC-002: JWT Expiry Enforcement
+**Severity:** Error
+
+All JWT tokens must have expiration claims. Access tokens should expire in 15-60 minutes.
+
+### SEC-003: Password Hashing Required
+**Severity:** Error
+
+Passwords must be hashed using bcrypt, argon2, or scrypt. Never store or compare passwords in plain text.
+
+```python
+# Bad
+if user.password == password:
+ ...
+
+# Good
+if bcrypt.checkpw(password.encode(), user.hashed_password):
+ ...
+```
+
+### SEC-004: Session Regeneration After Auth
+**Severity:** Warning
+
+Session IDs should be regenerated after authentication to prevent session fixation attacks.
+
+### SEC-005: Brute Force Protection
+**Severity:** Warning
+
+Login endpoints should implement rate limiting or account lockout after failed attempts.
+
+### SEC-006: Secure Password Reset
+**Severity:** Warning
+
+Password reset tokens must be cryptographically random, expire within 1 hour, and be single-use.
+
+### SEC-007: Authentication on Sensitive Endpoints
+**Severity:** Error
+
+All endpoints except public ones must require authentication.
+
+### SEC-008: Token in Authorization Header
+**Severity:** Warning
+
+JWT tokens should be sent in Authorization header, not in URL parameters.
+
+### SEC-009: Logout Invalidates Tokens
+**Severity:** Warning
+
+Logout should invalidate or blacklist tokens.
+
+### SEC-010: Multi-Factor Authentication Support
+**Severity:** Info
+
+Consider implementing MFA for sensitive operations.
+
+---
+
+## Injection Prevention Rules (SEC-011 to SEC-020)
+
+### SEC-011: No Raw SQL Queries
+**Severity:** Error
+
+Use SQLAlchemy ORM or parameterized queries only. Never concatenate user input into SQL strings.
+
+```python
+# Bad
+db.execute(f"SELECT * FROM users WHERE name = '{name}'")
+
+# Good
+db.query(User).filter(User.name == name)
+```
+
+### SEC-012: No Shell Command Injection
+**Severity:** Error
+
+Never use `shell=True` with subprocess. Use subprocess with list arguments.
+
+```python
+# Bad
+subprocess.run(f"convert {filename}", shell=True)
+os.system(f"rm {filename}")
+
+# Good
+subprocess.run(["convert", filename])
+```
+
+### SEC-013: No Code Execution
+**Severity:** Error
+
+Never use `eval()` or `exec()` with user input.
+
+### SEC-014: Path Traversal Prevention
+**Severity:** Error
+
+Validate file paths to prevent directory traversal. Use `secure_filename()` for uploads.
+
+```python
+# Bad
+path = f"/uploads/{filename}"
+
+# Good
+from werkzeug.utils import secure_filename
+path = f"/uploads/{secure_filename(filename)}"
+```
+
+### SEC-015: XSS Prevention in Templates
+**Severity:** Error
+
+Use safe output methods in templates. Prefer `x-text` over `x-html` in Alpine.js.
+
+```html
+
+
+
+
+
+```
+
+### SEC-016: LDAP Injection Prevention
+**Severity:** Error
+
+Escape special characters in LDAP queries.
+
+### SEC-017: XML External Entity Prevention
+**Severity:** Error
+
+Disable external entities when parsing XML. Use `defusedxml`.
+
+```python
+# Bad
+import xml.etree.ElementTree as ET
+tree = ET.parse(user_file)
+
+# Good
+import defusedxml.ElementTree as ET
+tree = ET.parse(user_file)
+```
+
+### SEC-018: Template Injection Prevention
+**Severity:** Error
+
+Never render user input as template code.
+
+### SEC-019: SSRF Prevention
+**Severity:** Warning
+
+Validate URLs before making external requests. Whitelist allowed domains.
+
+### SEC-020: Deserialization Safety
+**Severity:** Error
+
+Never deserialize untrusted data with pickle. Use `yaml.safe_load()` instead of `yaml.load()`.
+
+```python
+# Bad
+data = pickle.loads(user_data)
+config = yaml.load(user_config)
+
+# Good
+config = yaml.safe_load(user_config)
+data = json.loads(user_data)
+```
+
+---
+
+## Data Protection Rules (SEC-021 to SEC-030)
+
+### SEC-021: PII Logging Prevention
+**Severity:** Error
+
+Never log passwords, tokens, credit cards, or sensitive PII.
+
+```python
+# Bad
+logger.info(f"User login: {username}, password: {password}")
+
+# Good
+logger.info(f"User login: {username}")
+```
+
+### SEC-022: Sensitive Data in URLs
+**Severity:** Error
+
+Sensitive data should not appear in URL query parameters. Use POST body or headers instead.
+
+### SEC-023: Mass Assignment Prevention
+**Severity:** Warning
+
+Use explicit field assignment, not `**kwargs` from user input.
+
+```python
+# Bad
+user = User(**request.json)
+
+# Good
+user = User(
+ name=request.json.get("name"),
+ email=request.json.get("email")
+)
+```
+
+### SEC-024: Error Message Information Leakage
+**Severity:** Error
+
+Error messages should not reveal internal details. No stack traces to users.
+
+### SEC-025: Secure Cookie Settings
+**Severity:** Error
+
+Cookies must have Secure, HttpOnly, SameSite attributes.
+
+### SEC-026: Encryption for Sensitive Data at Rest
+**Severity:** Info
+
+Consider encrypting sensitive data stored in the database.
+
+### SEC-027: Data Retention Limits
+**Severity:** Info
+
+Implement data retention policies.
+
+### SEC-028: Response Data Filtering
+**Severity:** Warning
+
+API responses should not include sensitive internal fields. Use Pydantic response models.
+
+### SEC-029: File Upload Validation
+**Severity:** Error
+
+Validate uploaded files by extension AND content type. Limit file size.
+
+### SEC-030: Backup Encryption
+**Severity:** Info
+
+Database backups should be encrypted.
+
+---
+
+## API Security Rules (SEC-031 to SEC-040)
+
+### SEC-031: CORS Origin Validation
+**Severity:** Error
+
+CORS must not allow all origins in production. Specify allowed origins explicitly.
+
+```python
+# Bad
+allow_origins=["*"]
+
+# Good
+allow_origins=["https://example.com", "https://api.example.com"]
+```
+
+### SEC-032: Rate Limiting on Sensitive Endpoints
+**Severity:** Warning
+
+Auth, password reset, and payment endpoints need rate limiting.
+
+### SEC-033: Security Headers
+**Severity:** Warning
+
+Configure security headers like X-Content-Type-Options, X-Frame-Options, Content-Security-Policy.
+
+### SEC-034: HTTPS Enforcement
+**Severity:** Error
+
+External URLs must use HTTPS. HTTP is only acceptable for localhost.
+
+### SEC-035: Request Size Limits
+**Severity:** Warning
+
+Limit request body size to prevent DoS attacks.
+
+### SEC-036: Input Validation with Pydantic
+**Severity:** Warning
+
+All API inputs should be validated using Pydantic models.
+
+### SEC-037: API Versioning
+**Severity:** Info
+
+APIs should be versioned for security update isolation.
+
+### SEC-038: Method Restrictions
+**Severity:** Warning
+
+Endpoints should only allow necessary HTTP methods.
+
+### SEC-039: Authentication Bypass Prevention
+**Severity:** Error
+
+Ensure authentication cannot be bypassed.
+
+### SEC-040: Timeout Configuration
+**Severity:** Warning
+
+All external calls must have timeouts configured.
+
+---
+
+## Cryptography Rules (SEC-041 to SEC-050)
+
+### SEC-041: Strong Hashing Algorithms
+**Severity:** Error
+
+Use bcrypt, argon2, scrypt for passwords. Use SHA-256 or stronger for general hashing. Never use MD5 or SHA1.
+
+```python
+# Bad
+import hashlib
+password_hash = hashlib.md5(password.encode()).hexdigest()
+
+# Good
+import bcrypt
+password_hash = bcrypt.hashpw(password.encode(), bcrypt.gensalt())
+```
+
+### SEC-042: Secure Random Generation
+**Severity:** Error
+
+Use the `secrets` module for security-sensitive randomness. Never use `random` module for tokens or keys.
+
+```python
+# Bad
+import random
+token = ''.join(random.choices(string.ascii_letters, k=32))
+
+# Good
+import secrets
+token = secrets.token_urlsafe(32)
+```
+
+### SEC-043: No Hardcoded Encryption Keys
+**Severity:** Error
+
+Encryption keys must come from environment variables or secret management services.
+
+### SEC-044: Strong Encryption Algorithms
+**Severity:** Error
+
+Use AES-256 or ChaCha20. Never use DES, 3DES, or RC4.
+
+### SEC-045: Proper IV/Nonce Usage
+**Severity:** Error
+
+Encryption IVs and nonces must be randomly generated and unique per encryption.
+
+### SEC-046: TLS Version Requirements
+**Severity:** Warning
+
+Enforce TLS 1.2 or higher. Disable SSLv2, SSLv3, TLS 1.0, TLS 1.1.
+
+### SEC-047: Certificate Verification
+**Severity:** Error
+
+Always verify SSL certificates. Never disable verification in production.
+
+```python
+# Bad
+requests.get(url, verify=False)
+
+# Good
+requests.get(url, verify=True)
+```
+
+### SEC-048: Key Derivation for Passwords
+**Severity:** Warning
+
+When deriving encryption keys from passwords, use PBKDF2 with 100K+ iterations, Argon2, or scrypt.
+
+### SEC-049: Secure Key Storage
+**Severity:** Info
+
+Encryption keys should be stored in environment variables, secret management, or HSMs.
+
+### SEC-050: Key Rotation Support
+**Severity:** Info
+
+Implement key rotation with multiple key versions.
+
+---
+
+## Audit & Logging Rules (SEC-051 to SEC-060)
+
+### SEC-051: Authentication Event Logging
+**Severity:** Warning
+
+Log authentication events including successful logins, failed attempts, logouts, and password changes.
+
+### SEC-052: Admin Action Audit Trail
+**Severity:** Warning
+
+All admin operations should be logged with admin user ID, action performed, target resource, timestamp, and IP address.
+
+### SEC-053: Data Modification Logging
+**Severity:** Info
+
+Log create/update/delete on sensitive data like user accounts, roles, financial transactions, and configuration changes.
+
+### SEC-054: Security Event Logging
+**Severity:** Warning
+
+Log security-relevant events like authorization failures, input validation failures, rate limit triggers, and suspicious activity.
+
+### SEC-055: Log Injection Prevention
+**Severity:** Warning
+
+Sanitize user input before logging. Newlines and control characters can corrupt logs.
+
+```python
+# Bad
+logger.info(f"User search: {request.query}")
+
+# Good
+logger.info(f"User search: {request.query!r}") # repr escapes
+```
+
+### SEC-056: Centralized Logging
+**Severity:** Info
+
+Use centralized logging for correlation across services, tamper-evident storage, and alerting.
+
+### SEC-057: Log Level Appropriateness
+**Severity:** Info
+
+Use appropriate log levels: ERROR for security failures, WARNING for suspicious activity, INFO for successful events.
+
+### SEC-058: Structured Logging Format
+**Severity:** Info
+
+Use structured logging (JSON) for easy parsing and searchability.
+
+### SEC-059: Audit Log Integrity
+**Severity:** Info
+
+Protect audit logs from tampering with append-only storage and cryptographic chaining.
+
+### SEC-060: Privacy-Aware Logging
+**Severity:** Warning
+
+Comply with data protection regulations. No PII in logs without consent.
+
+---
+
+## Configuration
+
+All rules are defined in `.security-rules/` directory:
+
+```
+.security-rules/
+├── _main.yaml # Main configuration
+├── authentication.yaml # SEC-001 to SEC-010
+├── injection.yaml # SEC-011 to SEC-020
+├── data_protection.yaml # SEC-021 to SEC-030
+├── api_security.yaml # SEC-031 to SEC-040
+├── cryptography.yaml # SEC-041 to SEC-050
+└── audit.yaml # SEC-051 to SEC-060
+```
+
+## Suppressing Rules
+
+Use noqa comments to suppress specific rules:
+
+```python
+# noqa: SEC-001 - This is a test file with intentional test credentials
+test_password = "test123"
+```
+
+## Related Documentation
+
+- [Architecture Rules](architecture-rules.md)
+- [Performance Rules](performance-rules.md)
+- [Code Quality Guide](code-quality.md)
+- [Contributing Guide](contributing.md)
+
+---
+
+## Summary Statistics
+
+| Category | Rules | Errors | Warnings | Info |
+|----------|-------|--------|----------|------|
+| Authentication | 10 | 4 | 5 | 1 |
+| Injection Prevention | 10 | 9 | 1 | 0 |
+| Data Protection | 10 | 4 | 3 | 3 |
+| API Security | 10 | 4 | 5 | 1 |
+| Cryptography | 10 | 6 | 2 | 2 |
+| Audit & Logging | 10 | 0 | 5 | 5 |
+| **Total** | **60** | **27** | **21** | **12** |
+
+---
+
+**Last Updated:** 2025-12-21
+**Version:** 1.0
diff --git a/mkdocs.yml b/mkdocs.yml
index 29202247..d62646ab 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -40,6 +40,7 @@ nav:
- Authentication & RBAC: architecture/auth-rbac.md
- Frontend Structure: architecture/frontend-structure.md
- Models Structure: architecture/models-structure.md
+ - Background Tasks: architecture/background-tasks.md
- API Consolidation:
- Proposal: architecture/api-consolidation-proposal.md
- Migration Status: architecture/api-migration-status.md
@@ -117,6 +118,8 @@ nav:
- Contributing Guide: development/contributing.md
- Code Quality: development/code-quality.md
- Architecture Rules: development/architecture-rules.md
+ - Security Rules: development/security-rules.md
+ - Performance Rules: development/performance-rules.md
- Code Quality Dashboard: development/code-quality-dashboard-implementation.md
- Icons Guide: development/icons-guide.md
- Naming Conventions: development/naming-conventions.md
diff --git a/models/schema/stats.py b/models/schema/stats.py
index 182f2dbb..8f7e6e5a 100644
--- a/models/schema/stats.py
+++ b/models/schema/stats.py
@@ -243,15 +243,29 @@ class VendorAnalyticsResponse(BaseModel):
# ============================================================================
+class ValidatorStats(BaseModel):
+ """Statistics for a single validator type."""
+
+ total_violations: int = 0
+ errors: int = 0
+ warnings: int = 0
+ last_scan: str | None = None
+
+
class CodeQualityDashboardStatsResponse(BaseModel):
"""Code quality dashboard statistics response schema.
Used by: GET /api/v1/admin/code-quality/stats
+
+ Supports multiple validator types: architecture, security, performance.
+ When validator_type is specified, returns stats for that type only.
+ When not specified, returns combined stats with per-validator breakdown.
"""
total_violations: int
errors: int
warnings: int
+ info: int = 0
open: int
assigned: int
resolved: int
@@ -263,6 +277,11 @@ class CodeQualityDashboardStatsResponse(BaseModel):
by_module: dict[str, Any] = Field(default_factory=dict)
top_files: list[dict[str, Any]] = Field(default_factory=list)
last_scan: str | None = None
+ validator_type: str | None = None # Set when filtering by type
+ by_validator: dict[str, ValidatorStats] = Field(
+ default_factory=dict,
+ description="Per-validator breakdown (architecture, security, performance)",
+ )
# ============================================================================
diff --git a/scripts/base_validator.py b/scripts/base_validator.py
new file mode 100755
index 00000000..ce2d4224
--- /dev/null
+++ b/scripts/base_validator.py
@@ -0,0 +1,465 @@
+#!/usr/bin/env python3
+"""
+Base Validator
+==============
+Shared base class for all validation scripts (architecture, security, performance).
+
+Provides common functionality for:
+- Loading YAML configuration
+- File pattern matching
+- Violation tracking
+- Output formatting (human-readable and JSON)
+"""
+
+import json
+import re
+import sys
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+from enum import Enum
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+
+class Severity(Enum):
+ """Validation severity levels"""
+
+ ERROR = "error"
+ WARNING = "warning"
+ INFO = "info"
+
+
+@dataclass
+class Violation:
+ """Represents a rule violation"""
+
+ rule_id: str
+ rule_name: str
+ severity: Severity
+ file_path: Path
+ line_number: int
+ message: str
+ context: str = ""
+ suggestion: str = ""
+
+
+@dataclass
+class FileResult:
+ """Results for a single file validation"""
+
+ file_path: Path
+ errors: int = 0
+ warnings: int = 0
+ info: int = 0
+
+ @property
+ def passed(self) -> bool:
+ return self.errors == 0
+
+ @property
+ def status(self) -> str:
+ if self.errors > 0:
+ return "FAILED"
+ if self.warnings > 0:
+ return "PASSED*"
+ return "PASSED"
+
+ @property
+ def status_icon(self) -> str:
+ if self.errors > 0:
+ return "❌"
+ if self.warnings > 0:
+ return "⚠️"
+ return "✅"
+
+
+@dataclass
+class ValidationResult:
+ """Results of validation"""
+
+ violations: list[Violation] = field(default_factory=list)
+ files_checked: int = 0
+ rules_applied: int = 0
+ file_results: list[FileResult] = field(default_factory=list)
+
+ def has_errors(self) -> bool:
+ """Check if there are any error-level violations"""
+ return any(v.severity == Severity.ERROR for v in self.violations)
+
+ def has_warnings(self) -> bool:
+ """Check if there are any warning-level violations"""
+ return any(v.severity == Severity.WARNING for v in self.violations)
+
+ def error_count(self) -> int:
+ return sum(1 for v in self.violations if v.severity == Severity.ERROR)
+
+ def warning_count(self) -> int:
+ return sum(1 for v in self.violations if v.severity == Severity.WARNING)
+
+ def info_count(self) -> int:
+ return sum(1 for v in self.violations if v.severity == Severity.INFO)
+
+
+class BaseValidator(ABC):
+ """Abstract base validator class"""
+
+ # Subclasses should override these
+ VALIDATOR_NAME = "Base Validator"
+ VALIDATOR_EMOJI = "🔍"
+ RULES_DIR_NAME = ".rules"
+ CONFIG_FILE_NAME = ".rules.yaml"
+
+ def __init__(self, config_path: Path = None, verbose: bool = False):
+ """Initialize validator with configuration"""
+ self.project_root = Path.cwd()
+ self.config_path = config_path or self.project_root / self.CONFIG_FILE_NAME
+ self.verbose = verbose
+ self.config = self._load_config()
+ self.result = ValidationResult()
+
+ def _load_config(self) -> dict[str, Any]:
+ """
+ Load validation rules from YAML config.
+
+ Supports two modes:
+ 1. Split directory mode: rules directory with multiple YAML files
+ 2. Single file mode: single YAML file (legacy)
+
+ The split directory mode takes precedence if it exists.
+ """
+ # Check for split directory mode first
+ rules_dir = self.project_root / self.RULES_DIR_NAME
+ if rules_dir.is_dir():
+ return self._load_config_from_directory(rules_dir)
+
+ # Fall back to single file mode
+ if not self.config_path.exists():
+ print(f"❌ Configuration file not found: {self.config_path}")
+ print(f" (Also checked for directory: {rules_dir})")
+ sys.exit(1)
+
+ with open(self.config_path) as f:
+ config = yaml.safe_load(f)
+
+ print(f"📋 Loaded {self.VALIDATOR_NAME}: {config.get('project', 'unknown')}")
+ return config
+
+ def _load_config_from_directory(self, rules_dir: Path) -> dict[str, Any]:
+ """
+ Load and merge configuration from split YAML files in a directory.
+
+ Reads _main.yaml first for base config, then merges all other YAML files.
+ """
+ config: dict[str, Any] = {}
+
+ # Load _main.yaml first (contains project info, principles, ignore patterns)
+ main_file = rules_dir / "_main.yaml"
+ if main_file.exists():
+ with open(main_file) as f:
+ config = yaml.safe_load(f) or {}
+
+ # Load all other YAML files and merge their contents
+ yaml_files = sorted(rules_dir.glob("*.yaml"))
+ for yaml_file in yaml_files:
+ if yaml_file.name == "_main.yaml":
+ continue # Already loaded
+
+ with open(yaml_file) as f:
+ file_config = yaml.safe_load(f) or {}
+
+ # Merge rule sections from this file into main config
+ for key, value in file_config.items():
+ if key.endswith("_rules") and isinstance(value, list):
+ # Merge rule lists
+ if key not in config:
+ config[key] = []
+ config[key].extend(value)
+ elif key not in config:
+ # Add new top-level keys
+ config[key] = value
+
+ print(f"📋 Loaded {self.VALIDATOR_NAME}: {config.get('project', 'unknown')}")
+ print(f" (from {len(yaml_files)} files in {rules_dir.name}/)")
+ return config
+
+ def _should_ignore_file(self, file_path: Path) -> bool:
+ """Check if a file should be ignored based on config patterns"""
+ import fnmatch
+
+ ignore_config = self.config.get("ignore", {})
+ ignore_files = ignore_config.get("files", [])
+
+ # Get relative path for matching
+ try:
+ rel_path = file_path.relative_to(self.project_root)
+ except ValueError:
+ rel_path = file_path
+
+ rel_path_str = str(rel_path)
+
+ for pattern in ignore_files:
+ # Handle glob patterns using fnmatch
+ if "*" in pattern:
+ # fnmatch handles *, **, and ? patterns correctly
+ if fnmatch.fnmatch(rel_path_str, pattern):
+ return True
+ # Also check each path component for patterns like **/.venv/**
+ # This handles cases where the pattern expects any prefix
+ if pattern.startswith("**/"):
+ # Try matching without the **/ prefix (e.g., .venv/** matches .venv/foo)
+ suffix_pattern = pattern[3:] # Remove "**/""
+ if fnmatch.fnmatch(rel_path_str, suffix_pattern):
+ return True
+ elif pattern in rel_path_str:
+ return True
+
+ return False
+
+ def _add_violation(
+ self,
+ rule_id: str,
+ rule_name: str,
+ severity: Severity,
+ file_path: Path,
+ line_number: int,
+ message: str,
+ context: str = "",
+ suggestion: str = "",
+ ):
+ """Add a violation to the results"""
+ # Check for inline noqa comment
+ if f"noqa: {rule_id.lower()}" in context.lower():
+ return
+
+ self.result.violations.append(
+ Violation(
+ rule_id=rule_id,
+ rule_name=rule_name,
+ severity=severity,
+ file_path=file_path,
+ line_number=line_number,
+ message=message,
+ context=context,
+ suggestion=suggestion,
+ )
+ )
+
+ def _get_rule(self, rule_id: str) -> dict | None:
+ """Look up a rule by ID across all rule categories"""
+ for key, value in self.config.items():
+ if key.endswith("_rules") and isinstance(value, list):
+ for rule in value:
+ if rule.get("id") == rule_id:
+ return rule
+ return None
+
+ def _check_pattern_in_file(
+ self,
+ file_path: Path,
+ content: str,
+ lines: list[str],
+ pattern: str,
+ rule_id: str,
+ rule_name: str,
+ severity: Severity,
+ message: str,
+ suggestion: str = "",
+ exclude_patterns: list[str] = None,
+ ):
+ """Check for a regex pattern in a file and report violations"""
+ exclude_patterns = exclude_patterns or []
+
+ for i, line in enumerate(lines, 1):
+ if re.search(pattern, line, re.IGNORECASE):
+ # Check exclusions
+ should_exclude = False
+ for exclude in exclude_patterns:
+ if exclude in line:
+ should_exclude = True
+ break
+
+ if not should_exclude:
+ self._add_violation(
+ rule_id=rule_id,
+ rule_name=rule_name,
+ severity=severity,
+ file_path=file_path,
+ line_number=i,
+ message=message,
+ context=line.strip()[:100],
+ suggestion=suggestion,
+ )
+
+ @abstractmethod
+ def validate_all(self, target_path: Path = None) -> ValidationResult:
+ """Validate all files in a directory - must be implemented by subclasses"""
+ pass
+
+ def validate_file(self, file_path: Path, quiet: bool = False) -> ValidationResult:
+ """Validate a single file"""
+ if not file_path.exists():
+ if not quiet:
+ print(f"❌ File not found: {file_path}")
+ return self.result
+
+ if not file_path.is_file():
+ if not quiet:
+ print(f"❌ Not a file: {file_path}")
+ return self.result
+
+ if not quiet:
+ print(f"\n{self.VALIDATOR_EMOJI} Validating single file: {file_path}\n")
+
+ # Resolve file path to absolute
+ file_path = file_path.resolve()
+
+ if self._should_ignore_file(file_path):
+ if not quiet:
+ print("⏭️ File is in ignore list, skipping")
+ return self.result
+
+ self.result.files_checked += 1
+
+ # Track violations before this file
+ violations_before = len(self.result.violations)
+
+ content = file_path.read_text()
+ lines = content.split("\n")
+
+ # Call subclass-specific validation
+ self._validate_file_content(file_path, content, lines)
+
+ # Calculate violations for this file
+ file_violations = self.result.violations[violations_before:]
+ errors = sum(1 for v in file_violations if v.severity == Severity.ERROR)
+ warnings = sum(1 for v in file_violations if v.severity == Severity.WARNING)
+ info = sum(1 for v in file_violations if v.severity == Severity.INFO)
+
+ # Track file result
+ self.result.file_results.append(
+ FileResult(file_path=file_path, errors=errors, warnings=warnings, info=info)
+ )
+
+ return self.result
+
+ @abstractmethod
+ def _validate_file_content(self, file_path: Path, content: str, lines: list[str]):
+ """Validate file content - must be implemented by subclasses"""
+ pass
+
+ def output_results(self, json_output: bool = False, errors_only: bool = False):
+ """Output validation results"""
+ if json_output:
+ self._output_json()
+ else:
+ self._output_human(errors_only)
+
+ def _output_json(self):
+ """Output results as JSON
+
+ Format matches code quality service expectations:
+ - file_path (not file)
+ - line_number (not line)
+ - total_violations count
+ """
+ try:
+ rel_base = self.project_root
+ except Exception:
+ rel_base = Path.cwd()
+
+ def get_relative_path(file_path: Path) -> str:
+ """Get relative path from project root"""
+ try:
+ return str(file_path.relative_to(rel_base))
+ except ValueError:
+ return str(file_path)
+
+ output = {
+ "validator": self.VALIDATOR_NAME,
+ "files_checked": self.result.files_checked,
+ "total_violations": len(self.result.violations),
+ "errors": self.result.error_count(),
+ "warnings": self.result.warning_count(),
+ "info": self.result.info_count(),
+ "violations": [
+ {
+ "rule_id": v.rule_id,
+ "rule_name": v.rule_name,
+ "severity": v.severity.value,
+ "file_path": get_relative_path(v.file_path),
+ "line_number": v.line_number,
+ "message": v.message,
+ "context": v.context,
+ "suggestion": v.suggestion,
+ }
+ for v in self.result.violations
+ ],
+ }
+ print(json.dumps(output, indent=2))
+
+ def _output_human(self, errors_only: bool = False):
+ """Output results in human-readable format"""
+ print("\n" + "=" * 80)
+ print(f"📊 {self.VALIDATOR_NAME.upper()} REPORT")
+ print("=" * 80)
+
+ errors = [v for v in self.result.violations if v.severity == Severity.ERROR]
+ warnings = [v for v in self.result.violations if v.severity == Severity.WARNING]
+ info = [v for v in self.result.violations if v.severity == Severity.INFO]
+
+ print(
+ f"\nFiles checked: {self.result.files_checked}"
+ )
+ print(
+ f"Findings: {len(errors)} errors, {len(warnings)} warnings, {len(info)} info"
+ )
+
+ if errors:
+ print(f"\n\n❌ ERRORS ({len(errors)}):")
+ print("-" * 80)
+ for v in errors:
+ self._print_violation(v)
+
+ if warnings and not errors_only:
+ print(f"\n\n⚠️ WARNINGS ({len(warnings)}):")
+ print("-" * 80)
+ for v in warnings:
+ self._print_violation(v)
+
+ if info and not errors_only:
+ print(f"\nℹ️ INFO ({len(info)}):")
+ print("-" * 80)
+ for v in info:
+ self._print_violation(v)
+
+ print("\n" + "=" * 80)
+ if errors:
+ print("❌ VALIDATION FAILED")
+ elif warnings:
+ print(f"⚠️ VALIDATION PASSED WITH {len(warnings)} WARNING(S)")
+ else:
+ print("✅ VALIDATION PASSED")
+ print("=" * 80)
+
+ def _print_violation(self, v: Violation):
+ """Print a single violation"""
+ try:
+ rel_path = v.file_path.relative_to(self.project_root)
+ except ValueError:
+ rel_path = v.file_path
+
+ print(f"\n [{v.rule_id}] {v.rule_name}")
+ print(f" File: {rel_path}:{v.line_number}")
+ print(f" Issue: {v.message}")
+ if v.context and self.verbose:
+ print(f" Context: {v.context}")
+ if v.suggestion:
+ print(f" 💡 Suggestion: {v.suggestion}")
+
+ def get_exit_code(self) -> int:
+ """Get appropriate exit code based on results"""
+ if self.result.has_errors():
+ return 1
+ return 0
diff --git a/scripts/validate_all.py b/scripts/validate_all.py
new file mode 100755
index 00000000..d864731a
--- /dev/null
+++ b/scripts/validate_all.py
@@ -0,0 +1,218 @@
+#!/usr/bin/env python3
+"""
+Unified Code Validator
+======================
+Runs all validation scripts (architecture, security, performance) in sequence.
+
+This provides a single entry point for comprehensive code validation,
+useful for CI/CD pipelines and pre-commit hooks.
+
+Usage:
+ python scripts/validate_all.py # Run all validators
+ python scripts/validate_all.py --security # Run only security validator
+ python scripts/validate_all.py --performance # Run only performance validator
+ python scripts/validate_all.py --architecture # Run only architecture validator
+ python scripts/validate_all.py -v # Verbose output
+ python scripts/validate_all.py --fail-fast # Stop on first failure
+ python scripts/validate_all.py --json # JSON output
+
+Options:
+ --architecture Run architecture validator
+ --security Run security validator
+ --performance Run performance validator
+ --fail-fast Stop on first validator failure
+ -v, --verbose Show detailed output
+ --errors-only Only show errors
+ --json Output results as JSON
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent))
+
+from base_validator import Severity
+
+
+def run_architecture_validator(verbose: bool = False) -> tuple[int, dict]:
+ """Run the architecture validator"""
+ try:
+ # Import dynamically to avoid circular imports
+ sys.path.insert(0, str(Path(__file__).parent.parent))
+ from scripts.validate_architecture import ArchitectureValidator
+
+ config_path = Path.cwd() / ".architecture-rules.yaml"
+ validator = ArchitectureValidator(config_path=config_path, verbose=verbose)
+ result = validator.validate_all()
+
+ return (
+ 1 if result.has_errors() else 0,
+ {
+ "name": "Architecture",
+ "files_checked": result.files_checked,
+ "errors": sum(1 for v in result.violations if v.severity.value == "error"),
+ "warnings": sum(1 for v in result.violations if v.severity.value == "warning"),
+ "info": sum(1 for v in result.violations if v.severity.value == "info"),
+ }
+ )
+ except ImportError as e:
+ print(f"⚠️ Architecture validator not available: {e}")
+ return 0, {"name": "Architecture", "skipped": True}
+ except Exception as e:
+ print(f"❌ Architecture validator failed: {e}")
+ return 1, {"name": "Architecture", "error": str(e)}
+
+
+def run_security_validator(verbose: bool = False) -> tuple[int, dict]:
+ """Run the security validator"""
+ try:
+ from validate_security import SecurityValidator
+
+ validator = SecurityValidator(verbose=verbose)
+ result = validator.validate_all()
+
+ return (
+ 1 if result.has_errors() else 0,
+ {
+ "name": "Security",
+ "files_checked": result.files_checked,
+ "errors": result.error_count(),
+ "warnings": result.warning_count(),
+ "info": result.info_count(),
+ }
+ )
+ except ImportError as e:
+ print(f"⚠️ Security validator not available: {e}")
+ return 0, {"name": "Security", "skipped": True}
+ except Exception as e:
+ print(f"❌ Security validator failed: {e}")
+ return 1, {"name": "Security", "error": str(e)}
+
+
+def run_performance_validator(verbose: bool = False) -> tuple[int, dict]:
+ """Run the performance validator"""
+ try:
+ from validate_performance import PerformanceValidator
+
+ validator = PerformanceValidator(verbose=verbose)
+ result = validator.validate_all()
+
+ return (
+ 1 if result.has_errors() else 0,
+ {
+ "name": "Performance",
+ "files_checked": result.files_checked,
+ "errors": result.error_count(),
+ "warnings": result.warning_count(),
+ "info": result.info_count(),
+ }
+ )
+ except ImportError as e:
+ print(f"⚠️ Performance validator not available: {e}")
+ return 0, {"name": "Performance", "skipped": True}
+ except Exception as e:
+ print(f"❌ Performance validator failed: {e}")
+ return 1, {"name": "Performance", "error": str(e)}
+
+
+def print_summary(results: list[dict], json_output: bool = False):
+ """Print validation summary"""
+ if json_output:
+ print(json.dumps({"validators": results}, indent=2))
+ return
+
+ print("\n" + "=" * 80)
+ print("📊 UNIFIED VALIDATION SUMMARY")
+ print("=" * 80)
+
+ total_errors = 0
+ total_warnings = 0
+ total_info = 0
+
+ for result in results:
+ if result.get("skipped"):
+ print(f"\n⏭️ {result['name']}: Skipped")
+ elif result.get("error"):
+ print(f"\n❌ {result['name']}: Error - {result['error']}")
+ else:
+ errors = result.get("errors", 0)
+ warnings = result.get("warnings", 0)
+ info = result.get("info", 0)
+ total_errors += errors
+ total_warnings += warnings
+ total_info += info
+
+ status = "✅" if errors == 0 else "❌"
+ print(f"\n{status} {result['name']}:")
+ print(f" Files: {result.get('files_checked', 0)}")
+ print(f" Errors: {errors}, Warnings: {warnings}, Info: {info}")
+
+ print("\n" + "-" * 80)
+ print(f"TOTAL: {total_errors} errors, {total_warnings} warnings, {total_info} info")
+ print("=" * 80)
+
+ if total_errors > 0:
+ print("❌ VALIDATION FAILED")
+ elif total_warnings > 0:
+ print(f"⚠️ VALIDATION PASSED WITH {total_warnings} WARNING(S)")
+ else:
+ print("✅ VALIDATION PASSED")
+ print("=" * 80)
+
+
+def main():
+ parser = argparse.ArgumentParser(
+ description="Unified code validator - runs architecture, security, and performance checks",
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ )
+ parser.add_argument("--architecture", action="store_true", help="Run architecture validator")
+ parser.add_argument("--security", action="store_true", help="Run security validator")
+ parser.add_argument("--performance", action="store_true", help="Run performance validator")
+ parser.add_argument("--fail-fast", action="store_true", help="Stop on first failure")
+ parser.add_argument("-v", "--verbose", action="store_true", help="Verbose output")
+ parser.add_argument("--errors-only", action="store_true", help="Only show errors")
+ parser.add_argument("--json", action="store_true", help="JSON output")
+
+ args = parser.parse_args()
+
+ # If no specific validators selected, run all
+ run_all = not (args.architecture or args.security or args.performance)
+
+ print("\n🔍 UNIFIED CODE VALIDATION")
+ print("=" * 80)
+
+ validators = []
+ if run_all or args.architecture:
+ validators.append(("Architecture", run_architecture_validator))
+ if run_all or args.security:
+ validators.append(("Security", run_security_validator))
+ if run_all or args.performance:
+ validators.append(("Performance", run_performance_validator))
+
+ results = []
+ exit_code = 0
+
+ for name, validator_func in validators:
+ print(f"\n{'=' * 40}")
+ print(f"🔍 Running {name} Validator...")
+ print("=" * 40)
+
+ code, result = validator_func(verbose=args.verbose)
+
+ results.append(result)
+
+ if code != 0:
+ exit_code = 1
+ if args.fail_fast:
+ print(f"\n❌ {name} validator failed. Stopping (--fail-fast)")
+ break
+
+ print_summary(results, json_output=args.json)
+ sys.exit(exit_code)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/validate_performance.py b/scripts/validate_performance.py
new file mode 100755
index 00000000..d7da7603
--- /dev/null
+++ b/scripts/validate_performance.py
@@ -0,0 +1,648 @@
+#!/usr/bin/env python3
+"""
+Performance Validator
+=====================
+Validates code against performance rules defined in .performance-rules/
+
+This script checks for common performance issues:
+- N+1 query patterns
+- Missing pagination
+- Inefficient database operations
+- Memory management issues
+- Frontend performance anti-patterns
+- Missing timeouts and connection pooling
+
+Usage:
+ python scripts/validate_performance.py # Check all files
+ python scripts/validate_performance.py -d app/services/ # Check specific directory
+ python scripts/validate_performance.py -f app/api/v1/products.py # Check single file
+ python scripts/validate_performance.py -v # Verbose output
+ python scripts/validate_performance.py --json # JSON output
+ python scripts/validate_performance.py --errors-only # Only show errors
+
+Options:
+ -f, --file PATH Validate a single file
+ -d, --folder PATH Validate all files in a directory (recursive)
+ -v, --verbose Show detailed output including context
+ --errors-only Only show errors, suppress warnings and info
+ --json Output results as JSON
+"""
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+# Add parent directory to path for imports
+sys.path.insert(0, str(Path(__file__).parent))
+
+from base_validator import BaseValidator, Severity, ValidationResult
+
+
+class PerformanceValidator(BaseValidator):
+ """Performance-focused code validator"""
+
+ VALIDATOR_NAME = "Performance Validator"
+ VALIDATOR_EMOJI = "⚡"
+ RULES_DIR_NAME = ".performance-rules"
+ CONFIG_FILE_NAME = ".performance-rules.yaml"
+
+ def validate_all(self, target_path: Path = None) -> ValidationResult:
+ """Validate all files for performance issues"""
+ print(f"\n{self.VALIDATOR_EMOJI} Starting performance validation...\n")
+
+ target = target_path or self.project_root
+
+ # Validate Python files
+ self._validate_python_files(target)
+
+ # Validate JavaScript files
+ self._validate_javascript_files(target)
+
+ # Validate HTML templates
+ self._validate_template_files(target)
+
+ return self.result
+
+ def _validate_python_files(self, target: Path):
+ """Validate all Python files for performance issues"""
+ print("🐍 Validating Python files...")
+
+ for py_file in target.rglob("*.py"):
+ if self._should_ignore_file(py_file):
+ continue
+
+ self.result.files_checked += 1
+ content = py_file.read_text()
+ lines = content.split("\n")
+ self._validate_python_performance(py_file, content, lines)
+
+ def _validate_javascript_files(self, target: Path):
+ """Validate all JavaScript files for performance issues"""
+ print("🟨 Validating JavaScript files...")
+
+ for js_file in target.rglob("*.js"):
+ if self._should_ignore_file(js_file):
+ continue
+
+ self.result.files_checked += 1
+ content = js_file.read_text()
+ lines = content.split("\n")
+ self._validate_javascript_performance(js_file, content, lines)
+
+ def _validate_template_files(self, target: Path):
+ """Validate all HTML template files for performance issues"""
+ print("📄 Validating template files...")
+
+ for html_file in target.rglob("*.html"):
+ if self._should_ignore_file(html_file):
+ continue
+
+ self.result.files_checked += 1
+ content = html_file.read_text()
+ lines = content.split("\n")
+ self._validate_template_performance(html_file, content, lines)
+
+ def _validate_file_content(self, file_path: Path, content: str, lines: list[str]):
+ """Validate file content based on file type"""
+ if file_path.suffix == ".py":
+ self._validate_python_performance(file_path, content, lines)
+ elif file_path.suffix == ".js":
+ self._validate_javascript_performance(file_path, content, lines)
+ elif file_path.suffix == ".html":
+ self._validate_template_performance(file_path, content, lines)
+
+ def _validate_python_performance(self, file_path: Path, content: str, lines: list[str]):
+ """Validate Python file for performance issues"""
+ file_path_str = str(file_path)
+
+ # PERF-001: N+1 query detection
+ self._check_n_plus_1_queries(file_path, content, lines)
+
+ # PERF-003: Query result limiting
+ self._check_query_limiting(file_path, content, lines)
+
+ # PERF-006: Bulk operations
+ self._check_bulk_operations(file_path, content, lines)
+
+ # PERF-008: Use EXISTS for existence checks
+ self._check_existence_checks(file_path, content, lines)
+
+ # PERF-009: Batch updates
+ self._check_batch_updates(file_path, content, lines)
+
+ # PERF-026: Pagination for API endpoints
+ if "/api/" in file_path_str:
+ self._check_api_pagination(file_path, content, lines)
+
+ # PERF-037: Parallel async operations
+ self._check_parallel_async(file_path, content, lines)
+
+ # PERF-040: Timeout configuration
+ self._check_timeout_config(file_path, content, lines)
+
+ # PERF-046: Generators for large datasets
+ self._check_generators(file_path, content, lines)
+
+ # PERF-047: Stream file uploads
+ if "upload" in file_path_str.lower() or "file" in file_path_str.lower():
+ self._check_file_streaming(file_path, content, lines)
+
+ # PERF-048: Chunked processing
+ if "import" in file_path_str.lower() or "csv" in file_path_str.lower():
+ self._check_chunked_processing(file_path, content, lines)
+
+ # PERF-049: Context managers for files
+ self._check_context_managers(file_path, content, lines)
+
+ # PERF-051: String concatenation
+ self._check_string_concatenation(file_path, content, lines)
+
+ def _validate_javascript_performance(self, file_path: Path, content: str, lines: list[str]):
+ """Validate JavaScript file for performance issues"""
+ # PERF-056: Debounce search inputs
+ self._check_debounce(file_path, content, lines)
+
+ # PERF-062: Polling intervals
+ self._check_polling_intervals(file_path, content, lines)
+
+ # PERF-064: Layout thrashing
+ self._check_layout_thrashing(file_path, content, lines)
+
+ def _validate_template_performance(self, file_path: Path, content: str, lines: list[str]):
+ """Validate HTML template file for performance issues"""
+ # PERF-058: Image lazy loading
+ self._check_image_lazy_loading(file_path, content, lines)
+
+ # PERF-067: Script defer/async
+ self._check_script_loading(file_path, content, lines)
+
+ # =========================================================================
+ # Database Performance Checks
+ # =========================================================================
+
+ def _check_n_plus_1_queries(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-001: Check for N+1 query patterns"""
+ # Look for patterns like: for item in items: item.relationship.attribute
+ in_for_loop = False
+ for_line_num = 0
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ # Track for loops over query results
+ if re.search(r'for\s+\w+\s+in\s+.*\.(all|query)', line):
+ in_for_loop = True
+ for_line_num = i
+ elif in_for_loop and stripped and not stripped.startswith("#"):
+ # Check for relationship access in loop
+ if re.search(r'\.\w+\.\w+', line) and "(" not in line:
+ # Could be accessing a relationship
+ if any(rel in line for rel in [".customer.", ".vendor.", ".order.", ".product.", ".user."]):
+ self._add_violation(
+ rule_id="PERF-001",
+ rule_name="N+1 query detection",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Possible N+1 query - relationship accessed in loop",
+ context=line.strip()[:80],
+ suggestion="Use joinedload() or selectinload() for eager loading",
+ )
+ in_for_loop = False
+
+ # Reset on dedent
+ if in_for_loop and line and not line.startswith(" " * 4) and i > for_line_num + 1:
+ in_for_loop = False
+
+ def _check_query_limiting(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-003: Check for unbounded query results"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'\.all\(\)', line):
+ # Check if there's a limit or filter before
+ context_start = max(0, i - 5)
+ context_lines = lines[context_start:i]
+ context_text = "\n".join(context_lines)
+
+ if "limit" not in context_text.lower() and "filter" not in context_text.lower():
+ if "# noqa" in line or "# bounded" in line:
+ continue
+ self._add_violation(
+ rule_id="PERF-003",
+ rule_name="Query result limiting",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Query may return unbounded results",
+ context=line.strip()[:80],
+ suggestion="Add .limit() or pagination for large tables",
+ )
+
+ def _check_bulk_operations(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-006: Check for individual operations in loops"""
+ in_for_loop = False
+ for_indent = 0
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ # Track for loops
+ if re.search(r'for\s+\w+\s+in\s+', line):
+ in_for_loop = True
+ for_indent = len(line) - len(line.lstrip())
+ elif in_for_loop:
+ current_indent = len(line) - len(line.lstrip()) if line.strip() else for_indent + 4
+
+ if current_indent <= for_indent and stripped:
+ in_for_loop = False
+ elif "db.add(" in line or ".save(" in line:
+ self._add_violation(
+ rule_id="PERF-006",
+ rule_name="Bulk operations for multiple records",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Individual db.add() in loop - consider bulk operations",
+ context=line.strip()[:80],
+ suggestion="Use db.add_all() or bulk_insert_mappings()",
+ )
+
+ def _check_existence_checks(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-008: Check for inefficient existence checks"""
+ patterns = [
+ (r'\.count\(\)\s*>\s*0', "count() > 0"),
+ (r'\.count\(\)\s*>=\s*1', "count() >= 1"),
+ (r'\.count\(\)\s*!=\s*0', "count() != 0"),
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern, issue in patterns:
+ if re.search(pattern, line):
+ self._add_violation(
+ rule_id="PERF-008",
+ rule_name="Use EXISTS for existence checks",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message=f"{issue} scans all rows - use EXISTS instead",
+ context=line.strip()[:80],
+ suggestion="Use db.scalar(exists().where(...)) or .first() is not None",
+ )
+
+ def _check_batch_updates(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-009: Check for updates in loops"""
+ in_for_loop = False
+ for_indent = 0
+ loop_var = ""
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ # Track for loops
+ match = re.search(r'for\s+(\w+)\s+in\s+', line)
+ if match:
+ in_for_loop = True
+ for_indent = len(line) - len(line.lstrip())
+ loop_var = match.group(1)
+ elif in_for_loop:
+ current_indent = len(line) - len(line.lstrip()) if line.strip() else for_indent + 4
+
+ if current_indent <= for_indent and stripped:
+ in_for_loop = False
+ elif loop_var and f"{loop_var}." in line and "=" in line and "==" not in line:
+ # Attribute assignment in loop
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-009",
+ rule_name="Batch updates instead of loops",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Individual updates in loop - consider batch update",
+ context=line.strip()[:80],
+ suggestion="Use .update({...}) with filters for batch updates",
+ )
+
+ # =========================================================================
+ # API Performance Checks
+ # =========================================================================
+
+ def _check_api_pagination(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-026: Check for missing pagination in list endpoints"""
+ # Look for GET endpoints that return lists
+ in_endpoint = False
+ endpoint_line = 0
+ has_pagination = False
+
+ for i, line in enumerate(lines, 1):
+ # Track router decorators
+ if re.search(r'@router\.(get|post)', line):
+ in_endpoint = True
+ endpoint_line = i
+ has_pagination = False
+ elif in_endpoint:
+ # Check for pagination parameters
+ if re.search(r'(skip|offset|page|limit)', line):
+ has_pagination = True
+ # Check for function end
+ if re.search(r'^def\s+\w+', line.lstrip()) and i > endpoint_line + 1:
+ in_endpoint = False
+ # Check for .all() without pagination
+ if ".all()" in line and not has_pagination:
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-026",
+ rule_name="Pagination required for list endpoints",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="List endpoint may lack pagination",
+ context=line.strip()[:80],
+ suggestion="Add skip/limit parameters for pagination",
+ )
+
+ # =========================================================================
+ # Async Performance Checks
+ # =========================================================================
+
+ def _check_parallel_async(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-037: Check for sequential awaits that could be parallel"""
+ await_count = 0
+ await_lines = []
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ if stripped.startswith("await "):
+ await_count += 1
+ await_lines.append(i)
+
+ # Check for 3+ sequential awaits
+ if await_count >= 3:
+ # Verify they're sequential (within 5 lines of each other)
+ if all(await_lines[j+1] - await_lines[j] <= 2 for j in range(len(await_lines)-1)):
+ self._add_violation(
+ rule_id="PERF-037",
+ rule_name="Parallel independent operations",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=await_lines[0],
+ message=f"{await_count} sequential awaits - consider asyncio.gather()",
+ context="Multiple await statements",
+ suggestion="Use asyncio.gather() for independent async operations",
+ )
+ await_count = 0
+ await_lines = []
+ elif stripped and not stripped.startswith("#"):
+ # Reset on non-await, non-empty line
+ if await_count > 0:
+ await_count = 0
+ await_lines = []
+
+ def _check_timeout_config(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-040: Check for missing timeouts on HTTP clients"""
+ if "requests" not in content and "httpx" not in content and "aiohttp" not in content:
+ return
+
+ patterns = [
+ r'requests\.(get|post|put|delete|patch)\s*\([^)]+\)',
+ r'httpx\.(get|post|put|delete|patch)\s*\([^)]+\)',
+ ]
+
+ for i, line in enumerate(lines, 1):
+ for pattern in patterns:
+ if re.search(pattern, line) and "timeout" not in line:
+ self._add_violation(
+ rule_id="PERF-040",
+ rule_name="Timeout configuration",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="HTTP request without timeout",
+ context=line.strip()[:80],
+ suggestion="Add timeout parameter to prevent hanging requests",
+ )
+
+ # =========================================================================
+ # Memory Performance Checks
+ # =========================================================================
+
+ def _check_generators(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-046: Check for loading large datasets into memory"""
+ for i, line in enumerate(lines, 1):
+ # Check for .all() followed by iteration
+ if ".all()" in line:
+ # Look ahead for iteration
+ if i < len(lines):
+ next_lines = "\n".join(lines[i:min(i+3, len(lines))])
+ if "for " in next_lines and "in" in next_lines:
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-046",
+ rule_name="Generators for large datasets",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message=".all() loads everything into memory before iteration",
+ context=line.strip()[:80],
+ suggestion="Use .yield_per(100) for large result sets",
+ )
+
+ def _check_file_streaming(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-047: Check for loading entire files into memory"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'await\s+\w+\.read\(\)', line) and "chunk" not in line:
+ self._add_violation(
+ rule_id="PERF-047",
+ rule_name="Stream large file uploads",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="Full file read into memory",
+ context=line.strip()[:80],
+ suggestion="Stream large files: while chunk := await file.read(8192)",
+ )
+
+ def _check_chunked_processing(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-048: Check for chunked processing in imports"""
+ if "chunk" not in content.lower() and "batch" not in content.lower():
+ # Check if file processes multiple records
+ if "for " in content and ("csv" in content.lower() or "import" in content.lower()):
+ self._add_violation(
+ rule_id="PERF-048",
+ rule_name="Chunked processing for imports",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=1,
+ message="Import processing may benefit from chunking",
+ context="File processes multiple records",
+ suggestion="Process in chunks with periodic commits",
+ )
+
+ def _check_context_managers(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-049: Check for file handles without context managers"""
+ for i, line in enumerate(lines, 1):
+ # Check for file open without 'with'
+ if re.search(r'^\s*\w+\s*=\s*open\s*\(', line):
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-049",
+ rule_name="Context managers for resources",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="File opened without context manager",
+ context=line.strip()[:80],
+ suggestion="Use 'with open(...) as f:' to ensure cleanup",
+ )
+
+ def _check_string_concatenation(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-051: Check for inefficient string concatenation in loops"""
+ in_for_loop = False
+ for_indent = 0
+
+ for i, line in enumerate(lines, 1):
+ stripped = line.strip()
+
+ if re.search(r'for\s+\w+\s+in\s+', line):
+ in_for_loop = True
+ for_indent = len(line) - len(line.lstrip())
+ elif in_for_loop:
+ current_indent = len(line) - len(line.lstrip()) if line.strip() else for_indent + 4
+
+ if current_indent <= for_indent and stripped:
+ in_for_loop = False
+ elif re.search(r'\w+\s*\+=\s*["\']|str\s*\(', line):
+ if "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-051",
+ rule_name="String concatenation efficiency",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="String concatenation in loop",
+ context=line.strip()[:80],
+ suggestion="Use ''.join() or StringIO for many concatenations",
+ )
+
+ # =========================================================================
+ # Frontend Performance Checks
+ # =========================================================================
+
+ def _check_debounce(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-056: Check for search inputs without debounce"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'@(input|keyup)=".*search.*fetch', line, re.IGNORECASE):
+ if "debounce" not in content.lower():
+ self._add_violation(
+ rule_id="PERF-056",
+ rule_name="Debounce search inputs",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message="Search input triggers API call without debounce",
+ context=line.strip()[:80],
+ suggestion="Add 300-500ms debounce to prevent excessive API calls",
+ )
+
+ def _check_polling_intervals(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-062: Check for too-frequent polling"""
+ for i, line in enumerate(lines, 1):
+ match = re.search(r'setInterval\s*\([^,]+,\s*(\d+)\s*\)', line)
+ if match:
+ interval = int(match.group(1))
+ if interval < 10000: # Less than 10 seconds
+ if "# real-time" not in line and "# noqa" not in line:
+ self._add_violation(
+ rule_id="PERF-062",
+ rule_name="Reasonable polling intervals",
+ severity=Severity.WARNING,
+ file_path=file_path,
+ line_number=i,
+ message=f"Polling interval {interval}ms is very frequent",
+ context=line.strip()[:80],
+ suggestion="Use >= 10 second intervals for non-critical updates",
+ )
+
+ def _check_layout_thrashing(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-064: Check for layout thrashing patterns"""
+ for i, line in enumerate(lines, 1):
+ # Check for read then write patterns
+ if re.search(r'(offsetHeight|offsetWidth|clientHeight|clientWidth)', line):
+ if i < len(lines):
+ next_line = lines[i] if i < len(lines) else ""
+ if "style" in next_line:
+ self._add_violation(
+ rule_id="PERF-064",
+ rule_name="Avoid layout thrashing",
+ severity=Severity.INFO,
+ file_path=file_path,
+ line_number=i,
+ message="DOM read followed by write can cause layout thrashing",
+ context=line.strip()[:80],
+ suggestion="Batch DOM reads, then batch DOM writes",
+ )
+
+ def _check_image_lazy_loading(self, file_path: Path, content: str, lines: list[str]):
+ """PERF-058: Check for images without lazy loading"""
+ for i, line in enumerate(lines, 1):
+ if re.search(r'