orion/docs/operations/image-storage.md

Image Storage System

Documentation for the platform's image storage and management system.

Overview

The Orion platform uses a self-hosted image storage system with:

• Sharded directory structure for filesystem performance
• Automatic WebP conversion for optimization
• Multiple size variants for different use cases
• CDN-ready architecture for scaling

Storage Architecture

Directory Structure

Images are stored in a sharded directory structure to prevent filesystem performance degradation:

/static/uploads/
  └── products/
      ├── 00/                    # First 2 chars of hash
      │   ├── 1a/               # Next 2 chars
      │   │   ├── 001a2b3c_original.webp
      │   │   ├── 001a2b3c_800.webp
      │   │   └── 001a2b3c_200.webp
      │   └── 2b/
      │       └── ...
      ├── 01/
      │   └── ...
      └── ff/
          └── ...

Hash Generation

The file hash is generated from:

hash = md5(f"{store_id}:{product_id}:{timestamp}:{original_filename}")[:8]

This ensures:

• Unique file paths
• Even distribution across directories
• Predictable file locations

Image Variants

Each uploaded image generates multiple variants:

Variant	Max Dimensions	Format	Use Case
original	As uploaded (max 2000px)	WebP	Detail view, zoom
800	800×800	WebP	Product cards
200	200×200	WebP	Thumbnails, grids

Size Estimates

Original Size	After Processing	Storage per Image
2MB JPEG	~200KB (original) + 80KB (800) + 15KB (200)	~295KB
500KB JPEG	~150KB (original) + 60KB (800) + 12KB (200)	~222KB
100KB JPEG	~80KB (original) + 40KB (800) + 10KB (200)	~130KB

Average: ~200KB per image (all variants)

Upload Process

API Endpoint

POST /api/v1/admin/images/upload
Content-Type: multipart/form-data

file: <binary>
store_id: 123
product_id: 456 (optional, for product images)
type: product|category|banner

Response

{
  "success": true,
  "image": {
    "id": "001a2b3c",
    "urls": {
      "original": "/uploads/products/00/1a/001a2b3c_original.webp",
      "medium": "/uploads/products/00/1a/001a2b3c_800.webp",
      "thumb": "/uploads/products/00/1a/001a2b3c_200.webp"
    },
    "size_bytes": 295000,
    "dimensions": {
      "width": 1200,
      "height": 1200
    }
  }
}

Configuration

Environment Variables

# Image storage configuration
IMAGE_UPLOAD_DIR=/var/www/uploads
IMAGE_MAX_SIZE_MB=10
IMAGE_ALLOWED_TYPES=jpg,jpeg,png,gif,webp
IMAGE_QUALITY=85
IMAGE_MAX_DIMENSION=2000

Python Configuration

# app/core/config.py
class ImageSettings:
    UPLOAD_DIR: str = "/static/uploads"
    MAX_SIZE_MB: int = 10
    ALLOWED_TYPES: list = ["jpg", "jpeg", "png", "gif", "webp"]
    QUALITY: int = 85
    MAX_DIMENSION: int = 2000

    # Generated sizes
    SIZES: dict = {
        "original": None,  # No resize, just optimize
        "medium": 800,
        "thumb": 200,
    }

Performance Guidelines

Filesystem Limits

Files per Directory	Status	Action
< 10,000	OK	None needed
10,000 - 50,000	Monitor	Plan migration
50,000 - 100,000	Warning	Increase sharding depth
> 100,000	Critical	Migrate to object storage

Capacity Planning

Products	Images (5/product)	Total Files (3 sizes)	Storage
10,000	50,000	150,000	30 GB
50,000	250,000	750,000	150 GB
100,000	500,000	1,500,000	300 GB
500,000	2,500,000	7,500,000	1.5 TB

CDN Integration

For production deployments, configure a CDN for image delivery:

Cloudflare (Recommended)

1. Set up Cloudflare for your domain
2. Configure page rules for /uploads/*:
   • Cache Level: Cache Everything
   • Edge Cache TTL: 1 month
   • Browser Cache TTL: 1 week

nginx Configuration

location /uploads/ {
    alias /var/www/uploads/;
    expires 30d;
    add_header Cache-Control "public, immutable";
    add_header X-Content-Type-Options nosniff;

    # WebP fallback for older browsers
    location ~ \.(jpg|jpeg|png)$ {
        try_files $uri$webp_suffix $uri =404;
    }
}

Maintenance

Cleanup Orphaned Images

Remove images not referenced by any product:

# Run via admin CLI
python -m scripts.cleanup_orphaned_images --dry-run
python -m scripts.cleanup_orphaned_images --execute

Regenerate Variants

If image quality settings change:

# Regenerate all variants for a store
python -m scripts.regenerate_images --store-id 123

# Regenerate all variants (use with caution)
python -m scripts.regenerate_images --all

Monitoring

Metrics to Track

• Total file count
• Storage used (GB)
• Files per directory (max)
• Upload success rate
• Average processing time

Health Checks

The platform health page includes image storage metrics:

• Current file count
• Storage usage
• Directory distribution
• Processing queue status

Troubleshooting

Common Issues

Upload fails with "File too large"

• Check IMAGE_MAX_SIZE_MB setting
• Verify nginx client_max_body_size

Images not displaying

• Check file permissions (should be readable by web server)
• Verify URL paths match actual file locations

Slow uploads

• Check disk I/O performance
• Consider async processing queue

Related Documentation

• Capacity Planning
• Platform Health
• Capacity Monitoring