orion/docs/operations/image-storage.md

# Image Storage System

Documentation for the platform's image storage and management system.

## Overview

The Orion platform uses a self-hosted image storage system with:

- **Sharded directory structure** for filesystem performance
- **Automatic WebP conversion** for optimization
- **Multiple size variants** for different use cases
- **CDN-ready architecture** for scaling

## Storage Architecture

### Directory Structure

Images are stored in a sharded directory structure to prevent filesystem performance degradation:

```
/static/uploads/
  └── products/
      ├── 00/                    # First 2 chars of hash
      │   ├── 1a/               # Next 2 chars
      │   │   ├── 001a2b3c_original.webp
      │   │   ├── 001a2b3c_800.webp
      │   │   └── 001a2b3c_200.webp
      │   └── 2b/
      │       └── ...
      ├── 01/
      │   └── ...
      └── ff/
          └── ...
```

### Hash Generation

The file hash is generated from:
```python
hash = md5(f"{store_id}:{product_id}:{timestamp}:{original_filename}")[:8]
```

This ensures:
- Unique file paths
- Even distribution across directories
- Predictable file locations

## Image Variants

Each uploaded image generates multiple variants:

| Variant | Max Dimensions | Format | Use Case |
|---------|---------------|--------|----------|
| `original` | As uploaded (max 2000px) | WebP | Detail view, zoom |
| `800` | 800×800 | WebP | Product cards |
| `200` | 200×200 | WebP | Thumbnails, grids |

### Size Estimates

| Original Size | After Processing | Storage per Image |
|---------------|------------------|-------------------|
| 2MB JPEG | ~200KB (original) + 80KB (800) + 15KB (200) | ~295KB |
| 500KB JPEG | ~150KB (original) + 60KB (800) + 12KB (200) | ~222KB |
| 100KB JPEG | ~80KB (original) + 40KB (800) + 10KB (200) | ~130KB |

**Average: ~200KB per image (all variants)**

## Upload Process

### API Endpoint

```http
POST /api/v1/admin/images/upload
Content-Type: multipart/form-data

file: <binary>
store_id: 123
product_id: 456 (optional, for product images)
type: product|category|banner
```

### Response

```json
{
  "success": true,
  "image": {
    "id": "001a2b3c",
    "urls": {
      "original": "/uploads/products/00/1a/001a2b3c_original.webp",
      "medium": "/uploads/products/00/1a/001a2b3c_800.webp",
      "thumb": "/uploads/products/00/1a/001a2b3c_200.webp"
    },
    "size_bytes": 295000,
    "dimensions": {
      "width": 1200,
      "height": 1200
    }
  }
}
```

## Configuration

### Environment Variables

```bash
# Image storage configuration
IMAGE_UPLOAD_DIR=/var/www/uploads
IMAGE_MAX_SIZE_MB=10
IMAGE_ALLOWED_TYPES=jpg,jpeg,png,gif,webp
IMAGE_QUALITY=85
IMAGE_MAX_DIMENSION=2000
```

### Python Configuration

```python
# app/core/config.py
class ImageSettings:
    UPLOAD_DIR: str = "/static/uploads"
    MAX_SIZE_MB: int = 10
    ALLOWED_TYPES: list = ["jpg", "jpeg", "png", "gif", "webp"]
    QUALITY: int = 85
    MAX_DIMENSION: int = 2000

    # Generated sizes
    SIZES: dict = {
        "original": None,  # No resize, just optimize
        "medium": 800,
        "thumb": 200,
    }
```

## Performance Guidelines

### Filesystem Limits

| Files per Directory | Status | Action |
|---------------------|--------|--------|
| < 10,000 | OK | None needed |
| 10,000 - 50,000 | Monitor | Plan migration |
| 50,000 - 100,000 | Warning | Increase sharding depth |
| > 100,000 | Critical | Migrate to object storage |

### Capacity Planning

| Products | Images (5/product) | Total Files (3 sizes) | Storage |
|----------|--------------------|-----------------------|---------|
| 10,000 | 50,000 | 150,000 | 30 GB |
| 50,000 | 250,000 | 750,000 | 150 GB |
| 100,000 | 500,000 | 1,500,000 | 300 GB |
| 500,000 | 2,500,000 | 7,500,000 | 1.5 TB |

## CDN Integration

For production deployments, configure a CDN for image delivery:

### Cloudflare (Recommended)

1. Set up Cloudflare for your domain
2. Configure page rules for `/uploads/*`:
   - Cache Level: Cache Everything
   - Edge Cache TTL: 1 month
   - Browser Cache TTL: 1 week

### nginx Configuration

```nginx
location /uploads/ {
    alias /var/www/uploads/;
    expires 30d;
    add_header Cache-Control "public, immutable";
    add_header X-Content-Type-Options nosniff;

    # WebP fallback for older browsers
    location ~ \.(jpg|jpeg|png)$ {
        try_files $uri$webp_suffix $uri =404;
    }
}
```

## Maintenance

### Cleanup Orphaned Images

Remove images not referenced by any product:

```bash
# Run via admin CLI
python -m scripts.cleanup_orphaned_images --dry-run
python -m scripts.cleanup_orphaned_images --execute
```

### Regenerate Variants

If image quality settings change:

```bash
# Regenerate all variants for a store
python -m scripts.regenerate_images --store-id 123

# Regenerate all variants (use with caution)
python -m scripts.regenerate_images --all
```

## Monitoring

### Metrics to Track

- Total file count
- Storage used (GB)
- Files per directory (max)
- Upload success rate
- Average processing time

### Health Checks

The platform health page includes image storage metrics:

- Current file count
- Storage usage
- Directory distribution
- Processing queue status

## Troubleshooting

### Common Issues

**Upload fails with "File too large"**
- Check `IMAGE_MAX_SIZE_MB` setting
- Verify nginx `client_max_body_size`

**Images not displaying**
- Check file permissions (should be readable by web server)
- Verify URL paths match actual file locations

**Slow uploads**
- Check disk I/O performance
- Consider async processing queue

## Related Documentation

- [Capacity Planning](../architecture/capacity-planning.md)
- [Platform Health](platform-health.md)
- [Capacity Monitoring](capacity-monitoring.md)