docs(deployment): update memory limits and celery concurrency across all guides
Some checks failed
Some checks failed
Sync all deployment docs with actual docker-compose.yml values: - celery-worker: 512→768MB, concurrency 4→2 - db: 512→256MB, celery-beat: 256→128MB, flower: 256→192MB - Redis maxmemory: 256mb→100mb (matches container mem_limit 128m) - Add redis-exporter to scaling guide memory budget Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -94,11 +94,10 @@ services:
|
|||||||
celery:
|
celery:
|
||||||
build: .
|
build: .
|
||||||
restart: always
|
restart: always
|
||||||
command: celery -A app.celery worker --loglevel=info --concurrency=4
|
command: celery -A app.core.celery_config worker --loglevel=info -Q default,long_running,scheduled
|
||||||
environment:
|
environment:
|
||||||
DATABASE_URL: postgresql://orion_user:${DB_PASSWORD}@db:5432/orion_db
|
DATABASE_URL: postgresql://orion_user:${DB_PASSWORD}@db:5432/orion_db
|
||||||
REDIS_URL: redis://redis:6379/0
|
REDIS_URL: redis://redis:6379/0
|
||||||
CELERY_BROKER_URL: redis://redis:6379/1
|
|
||||||
env_file:
|
env_file:
|
||||||
- .env
|
- .env
|
||||||
depends_on:
|
depends_on:
|
||||||
@@ -109,7 +108,7 @@ services:
|
|||||||
deploy:
|
deploy:
|
||||||
resources:
|
resources:
|
||||||
limits:
|
limits:
|
||||||
memory: 512M
|
memory: 768M
|
||||||
|
|
||||||
celery-beat:
|
celery-beat:
|
||||||
build: .
|
build: .
|
||||||
@@ -148,7 +147,7 @@ services:
|
|||||||
redis:
|
redis:
|
||||||
image: redis:7-alpine
|
image: redis:7-alpine
|
||||||
restart: always
|
restart: always
|
||||||
command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
|
command: redis-server --maxmemory 100mb --maxmemory-policy allkeys-lru
|
||||||
volumes:
|
volumes:
|
||||||
- redis_data:/data
|
- redis_data:/data
|
||||||
healthcheck:
|
healthcheck:
|
||||||
|
|||||||
@@ -161,7 +161,7 @@ Complete step-by-step guide for deploying Orion on a Hetzner Cloud VPS.
|
|||||||
!!! success "Progress — 2026-02-17"
|
!!! success "Progress — 2026-02-17"
|
||||||
**Launch readiness — fully deployed and verified (44/44 checks pass):**
|
**Launch readiness — fully deployed and verified (44/44 checks pass):**
|
||||||
|
|
||||||
- **Memory limits** on all 6 app containers (db: 512m, redis: 128m, api: 512m, celery-worker: 512m, celery-beat: 256m, flower: 256m) — beat/flower bumped from 128m after OOM kills
|
- **Memory limits** on all 6 app containers (db: 256m, redis: 128m, api: 512m, celery-worker: 768m, celery-beat: 128m, flower: 192m) — rebalanced after celery-worker OOM kills (concurrency reduced from 4 to 2)
|
||||||
- **Flower port** restricted to localhost only (`127.0.0.1:5555:5555`) — access via Caddy reverse proxy
|
- **Flower port** restricted to localhost only (`127.0.0.1:5555:5555`) — access via Caddy reverse proxy
|
||||||
- **Flower password** changed from default
|
- **Flower password** changed from default
|
||||||
- **Infrastructure health checks** — `/health/ready` now checks PostgreSQL (`SELECT 1`) and Redis (`ping`) with individual check details and latency
|
- **Infrastructure health checks** — `/health/ready` now checks PostgreSQL (`SELECT 1`) and Redis (`ping`) with individual check details and latency
|
||||||
|
|||||||
@@ -409,7 +409,7 @@ services:
|
|||||||
celery:
|
celery:
|
||||||
build: .
|
build: .
|
||||||
restart: always
|
restart: always
|
||||||
command: celery -A app.celery worker --loglevel=info --concurrency=4
|
command: celery -A app.core.celery_config worker --loglevel=info --concurrency=2
|
||||||
environment:
|
environment:
|
||||||
DATABASE_URL: postgresql://orion_user:${DB_PASSWORD}@db:5432/orion_db
|
DATABASE_URL: postgresql://orion_user:${DB_PASSWORD}@db:5432/orion_db
|
||||||
REDIS_URL: redis://redis:6379/0
|
REDIS_URL: redis://redis:6379/0
|
||||||
@@ -676,7 +676,7 @@ find $BACKUP_DIR -name "*.sql.gz" -mtime +7 -delete
|
|||||||
**Configuration (`redis.conf`):**
|
**Configuration (`redis.conf`):**
|
||||||
|
|
||||||
```ini
|
```ini
|
||||||
maxmemory 256mb
|
maxmemory 100mb
|
||||||
maxmemory-policy allkeys-lru
|
maxmemory-policy allkeys-lru
|
||||||
appendonly yes
|
appendonly yes
|
||||||
appendfsync everysec
|
appendfsync everysec
|
||||||
|
|||||||
@@ -98,7 +98,7 @@ Group=orion
|
|||||||
WorkingDirectory=/home/orion/app
|
WorkingDirectory=/home/orion/app
|
||||||
Environment="PATH=/home/orion/app/.venv/bin"
|
Environment="PATH=/home/orion/app/.venv/bin"
|
||||||
EnvironmentFile=/home/orion/app/.env
|
EnvironmentFile=/home/orion/app/.env
|
||||||
ExecStart=/home/orion/app/.venv/bin/celery -A app.core.celery_config worker --loglevel=info -Q default,long_running,scheduled --concurrency=4
|
ExecStart=/home/orion/app/.venv/bin/celery -A app.core.celery_config worker --loglevel=info -Q default,long_running,scheduled --concurrency=2
|
||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=3
|
RestartSec=3
|
||||||
StandardOutput=journal
|
StandardOutput=journal
|
||||||
|
|||||||
@@ -18,21 +18,22 @@ Practical playbook for scaling Orion from a single CAX11 server to a multi-serve
|
|||||||
|
|
||||||
| Container | Limit | Purpose |
|
| Container | Limit | Purpose |
|
||||||
|-----------|-------|---------|
|
|-----------|-------|---------|
|
||||||
| db | 512 MB | PostgreSQL 15 |
|
| db | 256 MB | PostgreSQL 15 |
|
||||||
| redis | 128 MB | Task broker + cache |
|
| redis | 128 MB | Task broker (maxmemory 100mb, allkeys-lru) |
|
||||||
| api | 512 MB | FastAPI (Uvicorn) |
|
| api | 512 MB | FastAPI (Uvicorn) |
|
||||||
| celery-worker | 512 MB | Background tasks |
|
| celery-worker | 768 MB | Background tasks (concurrency=2) |
|
||||||
| celery-beat | 256 MB | Task scheduler |
|
| celery-beat | 128 MB | Task scheduler |
|
||||||
| flower | 256 MB | Celery monitoring |
|
| flower | 192 MB | Celery monitoring |
|
||||||
| **App subtotal** | **2,176 MB** | |
|
| **App subtotal** | **1,984 MB** | |
|
||||||
| prometheus | 256 MB | Metrics (15-day retention) |
|
| prometheus | 256 MB | Metrics (15-day retention) |
|
||||||
| grafana | 192 MB | Dashboards |
|
| grafana | 192 MB | Dashboards |
|
||||||
| node-exporter | 64 MB | Host metrics |
|
| node-exporter | 64 MB | Host metrics |
|
||||||
| cadvisor | 128 MB | Container metrics |
|
| cadvisor | 128 MB | Container metrics |
|
||||||
|
| redis-exporter | 32 MB | Redis metrics |
|
||||||
| alertmanager | 32 MB | Alert routing |
|
| alertmanager | 32 MB | Alert routing |
|
||||||
| **Monitoring subtotal** | **672 MB** | |
|
| **Monitoring subtotal** | **704 MB** | |
|
||||||
| **Total containers** | **2,848 MB** | |
|
| **Total containers** | **2,688 MB** | |
|
||||||
| OS + Caddy + Gitea + CI | ~1,150 MB | Remaining headroom |
|
| OS + Caddy + Gitea + CI | ~1,300 MB | Remaining headroom |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -164,14 +165,15 @@ Update `DATABASE_URL` in API and Celery to point to PgBouncer instead of `db` di
|
|||||||
|
|
||||||
### 3. Redis Hardening
|
### 3. Redis Hardening
|
||||||
|
|
||||||
Set a `maxmemory` policy to prevent OOM:
|
Redis `maxmemory` is already configured in `docker-compose.yml`:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# In docker-compose.yml, add command to redis service
|
|
||||||
redis:
|
redis:
|
||||||
command: redis-server --maxmemory 100mb --maxmemory-policy allkeys-lru
|
command: redis-server --maxmemory 100mb --maxmemory-policy allkeys-lru
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If Redis usage grows beyond 80%, the `RedisHighMemoryUsage` alert fires. Increase `maxmemory` and `mem_limit` together.
|
||||||
|
|
||||||
### 4. Separate Database Server
|
### 4. Separate Database Server
|
||||||
|
|
||||||
When the database needs its own resources (typically >50 stores):
|
When the database needs its own resources (typically >50 stores):
|
||||||
|
|||||||
Reference in New Issue
Block a user