orion/docs/deployment/production.md

Traditional VPS Deployment

This guide covers deploying Orion to a traditional VPS (Ubuntu 22.04+) without containers.

Best for: Teams who want direct server access and familiar Linux administration.

---

Prerequisites

• Ubuntu 22.04 LTS or newer
• 4GB+ RAM recommended
• Root or sudo access
• Domain name with DNS configured

---

Quick Start

# 1. Install system packages
sudo apt update && sudo apt upgrade -y
sudo apt install -y nginx postgresql-15 redis-server python3.11 python3.11-venv git curl

# 2. Create application user
sudo useradd -m -s /bin/bash orion

# 3. Setup PostgreSQL
sudo -u postgres createuser orion_user
sudo -u postgres createdb orion_db -O orion_user
sudo -u postgres psql -c "ALTER USER orion_user WITH PASSWORD 'your-secure-password';"

# 4. Clone and setup application
sudo su - orion
git clone <repository-url> ~/app
cd ~/app
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# 5. Configure environment
cp .env.example .env
nano .env  # Edit with production values

# 6. Initialize database
alembic upgrade head
python scripts/seed/init_production.py

# 7. Exit orion user
exit

---

Systemd Services

Main Application

sudo nano /etc/systemd/system/orion.service

[Unit]
Description=Orion API Server
After=network.target postgresql.service redis.service

[Service]
User=orion
Group=orion
WorkingDirectory=/home/orion/app
Environment="PATH=/home/orion/app/.venv/bin"
EnvironmentFile=/home/orion/app/.env
ExecStart=/home/orion/app/.venv/bin/uvicorn main:app --host 127.0.0.1 --port 8000 --workers 4
Restart=always
RestartSec=3
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Celery Worker

sudo nano /etc/systemd/system/orion-celery.service

[Unit]
Description=Orion Celery Worker
After=network.target redis.service postgresql.service

[Service]
User=orion
Group=orion
WorkingDirectory=/home/orion/app
Environment="PATH=/home/orion/app/.venv/bin"
EnvironmentFile=/home/orion/app/.env
ExecStart=/home/orion/app/.venv/bin/celery -A app.core.celery_config worker --loglevel=info -Q default,long_running,scheduled --concurrency=4
Restart=always
RestartSec=3
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Celery Beat (Scheduler)

sudo nano /etc/systemd/system/orion-celery-beat.service

[Unit]
Description=Orion Celery Beat Scheduler
After=network.target redis.service

[Service]
User=orion
Group=orion
WorkingDirectory=/home/orion/app
Environment="PATH=/home/orion/app/.venv/bin"
EnvironmentFile=/home/orion/app/.env
ExecStart=/home/orion/app/.venv/bin/celery -A app.core.celery_config beat --loglevel=info
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Flower (Task Monitoring)

sudo nano /etc/systemd/system/orion-flower.service

[Unit]
Description=Orion Flower Task Monitor
After=network.target redis.service

[Service]
User=orion
Group=orion
WorkingDirectory=/home/orion/app
Environment="PATH=/home/orion/app/.venv/bin"
EnvironmentFile=/home/orion/app/.env
ExecStart=/home/orion/app/.venv/bin/celery -A app.core.celery_config flower --port=5555
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Enable Services

sudo systemctl daemon-reload
sudo systemctl enable orion orion-celery orion-celery-beat orion-flower
sudo systemctl start orion orion-celery orion-celery-beat orion-flower

---

Nginx Configuration

sudo nano /etc/nginx/sites-available/orion

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

# Main HTTPS server
server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    # SSL (managed by Certbot)
    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    # Security headers
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;

    # Logging
    access_log /var/log/nginx/orion.access.log;
    error_log /var/log/nginx/orion.error.log;

    # Static files (served directly)
    location /static {
        alias /home/orion/app/static;
        expires 30d;
        add_header Cache-Control "public, immutable";
        access_log off;
    }

    # Uploaded files
    location /uploads {
        alias /home/orion/app/uploads;
        expires 7d;
        add_header Cache-Control "public";
    }

    # Application
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }

    # Block sensitive files
    location ~ /\. {
        deny all;
    }
    location ~ \.env$ {
        deny all;
    }
}

Enable Site

sudo ln -s /etc/nginx/sites-available/orion /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

SSL with Certbot

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com

---

Firewall

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable

---

Daily Operations

View Logs

# Application logs
sudo journalctl -u orion -f

# Celery logs
sudo journalctl -u orion-celery -f

# Nginx logs
sudo tail -f /var/log/nginx/orion.access.log
sudo tail -f /var/log/nginx/orion.error.log

# PostgreSQL logs
sudo tail -f /var/log/postgresql/postgresql-15-main.log

Restart Services

sudo systemctl restart orion
sudo systemctl restart orion-celery
sudo systemctl restart nginx

Database Access

# Connect as orion user
sudo -u postgres psql orion_db

# Or with password
psql -h localhost -U orion_user -d orion_db

Deploy Updates

sudo su - orion
cd ~/app
git pull origin main
source .venv/bin/activate
pip install -r requirements.txt
alembic upgrade head
exit
sudo systemctl restart orion orion-celery

---

Backups

!!! tip "Docker deployment" For Docker-based deployments, use the automated backup scripts (scripts/backup.sh and scripts/restore.sh) with systemd timer. See Hetzner Server Setup — Step 17.

Database Backup Script (VPS without Docker)

sudo nano /home/orion/backup.sh

#!/bin/bash
BACKUP_DIR=/home/orion/backups
DATE=$(date +%Y%m%d_%H%M%S)

# Create backup directory
mkdir -p $BACKUP_DIR

# Backup database
pg_dump -U orion_user orion_db | gzip > $BACKUP_DIR/db_$DATE.sql.gz

# Backup uploads
tar -czf $BACKUP_DIR/uploads_$DATE.tar.gz -C /home/orion/app uploads/

# Keep last 7 days
find $BACKUP_DIR -name "*.gz" -mtime +7 -delete

echo "Backup completed: $DATE"

chmod +x /home/orion/backup.sh

Cron Job

sudo -u orion crontab -e

# Daily backup at 2 AM
0 2 * * * /home/orion/backup.sh >> /home/orion/backup.log 2>&1

---

Monitoring

!!! tip "Docker deployment" For Docker-based deployments, a full Prometheus + Grafana + node-exporter + cAdvisor stack is included in docker-compose.yml. See Hetzner Server Setup — Step 18 and Observability Framework.

Basic Health Check

curl -s http://localhost:8000/health | jq

Process Monitoring

# Check all services
systemctl status orion orion-celery postgresql redis nginx

# Resource usage
htop
df -h
free -h

Set Up Sentry (Error Tracking)

Sentry provides real-time error tracking and performance monitoring.

1. Create a Sentry account at sentry.io (free tier available)
2. Create a new project (Python/FastAPI)
3. Add to .env:

   SENTRY_DSN=https://your-key@sentry.io/project-id
   SENTRY_ENVIRONMENT=production
   SENTRY_TRACES_SAMPLE_RATE=0.1
   

4. Restart services:

   sudo systemctl restart orion orion-celery
   

Sentry will now capture:

• Unhandled exceptions
• API errors with request context
• Celery task failures
• Performance traces (10% sample rate)

---

Cloudflare R2 Storage

For production, use Cloudflare R2 instead of local storage for scalability and CDN integration.

Setup

1. Create R2 bucket in CloudFlare dashboard
2. Create API token with Object Read/Write permissions
3. Add to .env:

   STORAGE_BACKEND=r2
   R2_ACCOUNT_ID=your_account_id
   R2_ACCESS_KEY_ID=your_access_key
   R2_SECRET_ACCESS_KEY=your_secret_key
   R2_BUCKET_NAME=orion-media
   R2_PUBLIC_URL=https://media.yourdomain.com
   

See CloudFlare Setup Guide for detailed instructions.

---

CloudFlare CDN & Proxy

For production, proxy your domain through CloudFlare for:

• Global CDN caching
• DDoS protection
• Free SSL certificates
• WAF (Web Application Firewall)

Enable CloudFlare Headers

Add to .env:

CLOUDFLARE_ENABLED=true

This enables proper handling of CF-Connecting-IP for real client IPs.

See CloudFlare Setup Guide for complete configuration.

---

Troubleshooting

See Infrastructure Guide - Troubleshooting for detailed diagnostics.

Quick Checks

# Is the app running?
systemctl status orion

# Can we connect to the database?
pg_isready -h localhost -U orion_user

# Is Redis running?
redis-cli ping

# Check open ports
ss -tlnp | grep -E '(8000|5432|6379|80|443)'

# View recent errors
journalctl -u orion --since "1 hour ago" | grep -i error