# Hetzner Cloud Server Setup Complete step-by-step guide for deploying Wizamart on a Hetzner Cloud VPS. !!! info "Server Details" - **Provider**: Hetzner Cloud - **OS**: Ubuntu 24.04.3 LTS (upgraded to 24.04.4 after updates) - **Architecture**: aarch64 (ARM64) - **IP**: `91.99.65.229` - **IPv6**: `2a01:4f8:1c1a:b39c::1` - **Disk**: 37 GB - **RAM**: 4 GB - **Auth**: SSH key (configured via Hetzner Console) - **Setup date**: 2026-02-11 !!! success "Progress — 2026-02-12" **Completed (Steps 1–15):** - Non-root user `samir` with SSH key - Server hardened (UFW firewall, SSH root login disabled, fail2ban) - Docker 29.2.1 & Docker Compose 5.0.2 installed - Gitea running at `https://git.wizard.lu` (user: `sboulahtit`, repo: `orion`) - Repository cloned to `~/apps/orion` - Production `.env` configured with generated secrets - Full Docker stack deployed (API, PostgreSQL, Redis, Celery worker/beat, Flower) - Database migrated (76 tables) and seeded (admin, platforms, CMS, email templates) - API verified at `https://api.wizard.lu/health` - DNS A records configured and propagated for `wizard.lu` and subdomains - Caddy 2.10.2 reverse proxy with auto-SSL (Let's Encrypt) - Temporary firewall rules removed (ports 3000, 8001) - Gitea Actions runner v0.2.13 registered and running as systemd service - SSH key added to Gitea for local push via SSH - Git remote updated: `ssh://git@git.wizard.lu:2222/sboulahtit/orion.git` - CI ruff (lint) job passing - ProxyHeadersMiddleware added for correct HTTPS behind Caddy - Fixed TierLimitExceededException import and Pydantic @field_validator bugs - `wizard.lu` serving frontend with CSS over HTTPS (mixed content fixed) - `/merchants` and `/admin` redirect fix (CMS catch-all was intercepting) **Remaining for next session:** - [ ] CI pytest job — fails due to pydantic version mismatch (requirements need pinning) - [ ] DNS A records for platform domains (`oms.lu`, `loyaltyplus.lu`) - [ ] Uncomment platform domains in Caddyfile after DNS propagation - [ ] AAAA (IPv6) records for all domains - [ ] Update `platforms` table `domain` column to match production domains ## Installed Software Versions | Software | Version | |---|---| | Ubuntu | 24.04.4 LTS | | Kernel | 6.8.0-100-generic (aarch64) | | Docker | 29.2.1 | | Docker Compose | 5.0.2 | | PostgreSQL | 15 (container) | | Redis | 7-alpine (container) | | Python | 3.11-slim (container) | | Gitea | latest (container) | | Caddy | 2.10.2 | | act_runner | 0.2.13 | --- ## Step 1: Initial Server Access ```bash ssh root@91.99.65.229 ``` ## Step 2: Create Non-Root User Create a dedicated user with sudo privileges and copy the SSH key: ```bash # Create user adduser samir usermod -aG sudo samir # Copy SSH keys to new user rsync --archive --chown=samir:samir ~/.ssh /home/samir ``` Verify by connecting as the new user (from a **new terminal**): ```bash ssh samir@91.99.65.229 ``` ## Step 3: System Update & Essential Packages ```bash sudo apt update && sudo apt upgrade -y sudo apt install -y \ curl \ git \ wget \ ufw \ fail2ban \ htop \ unzip \ make ``` Reboot if a kernel upgrade is pending: ```bash sudo reboot ``` ## Step 4: Firewall Configuration (UFW) ```bash sudo ufw allow OpenSSH sudo ufw allow 80/tcp sudo ufw allow 443/tcp sudo ufw enable ``` Verify: ```bash sudo ufw status ``` Expected output: ``` Status: active To Action From -- ------ ---- OpenSSH ALLOW Anywhere 80/tcp ALLOW Anywhere 443/tcp ALLOW Anywhere ``` ## Step 5: Harden SSH !!! warning "Before doing this step" Make sure you can SSH as `samir` from another terminal first! If you lock yourself out, you'll need to use Hetzner's console rescue mode. ```bash sudo sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config sudo sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config sudo systemctl restart ssh # Note: Ubuntu 24.04 uses 'ssh' not 'sshd' ``` ## Step 6: Install Docker & Docker Compose ```bash curl -fsSL https://get.docker.com | sh sudo usermod -aG docker samir ``` Log out and back in for the group change: ```bash exit # Then: ssh samir@91.99.65.229 ``` Verify: ```bash docker --version docker compose version ``` ## Step 7: Gitea (Self-Hosted Git) Create the Gitea directory and compose file: ```bash mkdir -p ~/gitea && cd ~/gitea ``` Create `docker-compose.yml` with `nano ~/gitea/docker-compose.yml`: ```yaml services: gitea: image: gitea/gitea:latest container_name: gitea restart: always environment: - USER_UID=1000 - USER_GID=1000 - GITEA__database__DB_TYPE=postgres - GITEA__database__HOST=gitea-db:5432 - GITEA__database__NAME=gitea - GITEA__database__USER=gitea - GITEA__database__PASSWD= - GITEA__server__ROOT_URL=http://91.99.65.229:3000/ - GITEA__server__SSH_DOMAIN=91.99.65.229 - GITEA__server__DOMAIN=91.99.65.229 - GITEA__actions__ENABLED=true volumes: - gitea-data:/data - /etc/timezone:/etc/timezone:ro - /etc/localtime:/etc/localtime:ro ports: - "3000:3000" - "2222:22" depends_on: gitea-db: condition: service_healthy gitea-db: image: postgres:15 container_name: gitea-db restart: always environment: POSTGRES_DB: gitea POSTGRES_USER: gitea POSTGRES_PASSWORD: volumes: - gitea-db-data:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U gitea"] interval: 10s timeout: 5s retries: 5 volumes: gitea-data: gitea-db-data: ``` Generate the database password with `openssl rand -hex 16` and replace `` in both places. Open the firewall for Gitea and start: ```bash sudo ufw allow 3000/tcp docker compose up -d docker compose ps ``` Visit `http://91.99.65.229:3000` and complete the setup wizard. Create an admin account (e.g. `sboulahtit`). Then create a repository (e.g. `orion`). ## Step 8: Push Repository to Gitea ### Add SSH Key to Gitea Before pushing via SSH, add your local machine's public key to Gitea: 1. Copy your public key: ```bash cat ~/.ssh/id_ed25519.pub # Or if using RSA: cat ~/.ssh/id_rsa.pub ``` 2. In the Gitea web UI: click your avatar → **Settings** → **SSH / GPG Keys** → **Add Key** → paste the key. 3. Add the Gitea SSH host to known hosts: ```bash ssh-keyscan -p 2222 git.wizard.lu >> ~/.ssh/known_hosts ``` ### Add Remote and Push From your **local machine**: ```bash cd /home/samir/Documents/PycharmProjects/letzshop-product-import git remote add gitea ssh://git@git.wizard.lu:2222/sboulahtit/orion.git git push gitea master ``` !!! note "Remote URL updated" The remote was initially set to `http://91.99.65.229:3000/...` during setup. After Caddy was configured, it was updated to use the domain with SSH: `ssh://git@git.wizard.lu:2222/sboulahtit/orion.git` To update an existing remote: ```bash git remote set-url gitea ssh://git@git.wizard.lu:2222/sboulahtit/orion.git ``` ## Step 9: Clone Repository on Server ```bash mkdir -p ~/apps cd ~/apps git clone http://localhost:3000/sboulahtit/orion.git cd orion ``` ## Step 10: Configure Production Environment ```bash cp .env.example .env nano .env ``` ### Critical Production Values Generate secrets: ```bash openssl rand -hex 32 # For JWT_SECRET_KEY openssl rand -hex 16 # For database password ``` | Variable | How to Generate / What to Set | |---|---| | `DEBUG` | `False` | | `DATABASE_URL` | `postgresql://wizamart_user:YOUR_DB_PW@db:5432/wizamart_db` | | `JWT_SECRET_KEY` | Output of `openssl rand -hex 32` | | `ADMIN_PASSWORD` | Strong password | | `USE_CELERY` | `true` | | `REDIS_URL` | `redis://redis:6379/0` | | `STRIPE_SECRET_KEY` | Your Stripe secret key (configure later) | | `STRIPE_PUBLISHABLE_KEY` | Your Stripe publishable key (configure later) | | `STRIPE_WEBHOOK_SECRET` | Your Stripe webhook secret (configure later) | | `STORAGE_BACKEND` | `r2` (if using Cloudflare R2, configure later) | Also update the PostgreSQL password in `docker-compose.yml` (lines 9 and 40) to match. ## Step 11: Deploy with Docker Compose ```bash cd ~/apps/orion # Create directories with correct permissions for the container user mkdir -p logs uploads exports sudo chown -R 1000:1000 logs uploads exports # Start infrastructure first docker compose up -d db redis # Wait for health checks to pass docker compose ps # Build and start the full stack docker compose --profile full up -d --build ``` Verify all services are running: ```bash docker compose --profile full ps ``` Expected: `api` (healthy), `db` (healthy), `redis` (healthy), `celery-worker` (healthy), `celery-beat` (running), `flower` (running). ## Step 12: Initialize Database !!! note "PYTHONPATH required" The seed scripts need `PYTHONPATH=/app` set explicitly when running inside the container. ```bash # Run migrations (use 'heads' for multi-branch Alembic) docker compose --profile full exec -e PYTHONPATH=/app api python -m alembic upgrade heads # Seed production data docker compose --profile full exec -e PYTHONPATH=/app api python scripts/seed/init_production.py docker compose --profile full exec -e PYTHONPATH=/app api python scripts/seed/init_log_settings.py docker compose --profile full exec -e PYTHONPATH=/app api python scripts/seed/create_default_content_pages.py docker compose --profile full exec -e PYTHONPATH=/app api python scripts/seed/create_platform_pages.py docker compose --profile full exec -e PYTHONPATH=/app api python scripts/seed/seed_email_templates.py ``` ### Seeded Data Summary | Data | Count | |---|---| | Admin users | 1 (`admin@wizamart.com`) | | Platforms | 3 (OMS, Main, Loyalty+) | | Admin settings | 15 | | Subscription tiers | 4 (Essential, Professional, Business, Enterprise) | | Log settings | 6 | | CMS pages | 8 (About, Contact, FAQ, Shipping, Returns, Privacy, Terms, Homepage) | | Email templates | 17 (4 languages: en, fr, de, lb) | --- ## Step 13: DNS Configuration Before setting up Caddy, point your domain's DNS to the server. ### wizard.lu (Main Platform) — Completed | Type | Name | Value | TTL | |---|---|---|---| | A | `@` | `91.99.65.229` | 300 | | A | `www` | `91.99.65.229` | 300 | | A | `api` | `91.99.65.229` | 300 | | A | `git` | `91.99.65.229` | 300 | | A | `flower` | `91.99.65.229` | 300 | ### oms.lu (OMS Platform) — TODO | Type | Name | Value | TTL | |---|---|---|---| | A | `@` | `91.99.65.229` | 300 | | A | `www` | `91.99.65.229` | 300 | ### loyaltyplus.lu (Loyalty+ Platform) — TODO | Type | Name | Value | TTL | |---|---|---|---| | A | `@` | `91.99.65.229` | 300 | | A | `www` | `91.99.65.229` | 300 | ### IPv6 (AAAA) Records — TODO Optional but recommended. Add AAAA records for all domains above, pointing to the server's IPv6 address. Verify your IPv6 address first: ```bash ip -6 addr show eth0 | grep 'scope global' ``` It should match the value in the Hetzner Cloud Console (Networking tab). Then create AAAA records mirroring each A record above, e.g.: | Type | Name (wizard.lu) | Value | TTL | |---|---|---|---| | AAAA | `@` | `2a01:4f8:1c1a:b39c::1` | 300 | | AAAA | `www` | `2a01:4f8:1c1a:b39c::1` | 300 | | AAAA | `api` | `2a01:4f8:1c1a:b39c::1` | 300 | | AAAA | `git` | `2a01:4f8:1c1a:b39c::1` | 300 | | AAAA | `flower` | `2a01:4f8:1c1a:b39c::1` | 300 | Repeat for `oms.lu` and `loyaltyplus.lu`. !!! tip "DNS propagation" Set TTL to 300 (5 minutes) initially. DNS changes can take up to 24 hours to propagate globally, but usually complete within 30 minutes. Verify with: `dig api.wizard.lu +short` ## Step 14: Reverse Proxy with Caddy Install Caddy: ```bash sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \ | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \ | sudo tee /etc/apt/sources.list.d/caddy-stable.list sudo apt update && sudo apt install caddy ``` ### Caddyfile Configuration Edit `/etc/caddy/Caddyfile`: ```caddy # ─── Platform 1: Main (wizard.lu) ─────────────────────────── wizard.lu { reverse_proxy localhost:8001 } www.wizard.lu { redir https://wizard.lu{uri} permanent } # ─── Platform 2: OMS (oms.lu) ─────────────────────────────── # Uncomment after DNS is configured for oms.lu # oms.lu { # reverse_proxy localhost:8001 # } # # www.oms.lu { # redir https://oms.lu{uri} permanent # } # ─── Platform 3: Loyalty+ (loyaltyplus.lu) ────────────────── # Uncomment after DNS is configured for loyaltyplus.lu # loyaltyplus.lu { # reverse_proxy localhost:8001 # } # # www.loyaltyplus.lu { # redir https://loyaltyplus.lu{uri} permanent # } # ─── Services ─────────────────────────────────────────────── api.wizard.lu { reverse_proxy localhost:8001 } git.wizard.lu { reverse_proxy localhost:3000 } flower.wizard.lu { reverse_proxy localhost:5555 } ``` !!! info "How multi-platform routing works" All platform domains (`wizard.lu`, `oms.lu`, `loyaltyplus.lu`) point to the **same FastAPI backend** on port 8001. The `PlatformContextMiddleware` reads the `Host` header to detect which platform the request is for. Caddy preserves the Host header by default, so no extra configuration is needed. The `domain` column in the `platforms` database table must match: | Platform | code | domain | |---|---|---| | Main | `main` | `wizard.lu` | | OMS | `oms` | `oms.lu` | | Loyalty+ | `loyalty` | `loyaltyplus.lu` | Start Caddy: ```bash sudo systemctl restart caddy ``` Caddy automatically provisions Let's Encrypt SSL certificates for all configured domains. Verify: ```bash curl -I https://wizard.lu curl -I https://api.wizard.lu/health curl -I https://git.wizard.lu ``` After Caddy is working, remove the temporary firewall rules: ```bash sudo ufw delete allow 3000/tcp sudo ufw delete allow 8001/tcp ``` Update Gitea's configuration to use its new domain. In `~/gitea/docker-compose.yml`, change: ```yaml - GITEA__server__ROOT_URL=https://git.wizard.lu/ - GITEA__server__SSH_DOMAIN=git.wizard.lu - GITEA__server__DOMAIN=git.wizard.lu ``` Then restart Gitea: ```bash cd ~/gitea && docker compose up -d gitea ``` ### Future: Multi-Tenant Store Routing Stores on each platform use two routing modes: - **Standard (subdomain)**: `acme.oms.lu` — included in the base subscription - **Premium (custom domain)**: `acme.lu` — available with premium subscription tiers Both modes are handled by the `StoreContextMiddleware` which reads the `Host` header, so Caddy just needs to forward requests and preserve the header. #### Wildcard Subdomains (for store subdomains) When stores start using subdomains like `acme.oms.lu`, add wildcard blocks: ```caddy *.oms.lu { reverse_proxy localhost:8001 } *.loyaltyplus.lu { reverse_proxy localhost:8001 } *.wizard.lu { reverse_proxy localhost:8001 } ``` !!! warning "Wildcard SSL requires DNS challenge" Let's Encrypt cannot issue wildcard certificates via HTTP challenge. Wildcard certs require a **DNS challenge**, which means installing a Caddy DNS provider plugin (e.g. `caddy-dns/cloudflare`) and configuring API credentials for your DNS provider. See [Caddy DNS challenge docs](https://caddyserver.com/docs/automatic-https#dns-challenge). #### Custom Store Domains (for premium stores) When premium stores bring their own domains (e.g. `acme.lu`), use Caddy's **on-demand TLS**: ```caddy https:// { tls { on_demand } reverse_proxy localhost:8001 } ``` On-demand TLS auto-provisions SSL certificates when a new domain connects. Add an `ask` endpoint to validate that the domain is registered in the `store_domains` table, preventing abuse: ```caddy tls { on_demand ask http://localhost:8001/api/v1/internal/verify-domain } ``` !!! note "Not needed yet" Wildcard subdomains and custom domains are future work. The current Caddyfile handles all platform root domains and service subdomains. ## Step 15: Gitea Actions Runner !!! warning "ARM64 architecture" This server is ARM64. Download the `arm64` binary, not `amd64`. Download and install: ```bash mkdir -p ~/gitea-runner && cd ~/gitea-runner # Download act_runner v0.2.13 (ARM64) wget https://gitea.com/gitea/act_runner/releases/download/v0.2.13/act_runner-0.2.13-linux-arm64 chmod +x act_runner-0.2.13-linux-arm64 ln -s act_runner-0.2.13-linux-arm64 act_runner ``` Register the runner (get token from **Site Administration > Actions > Runners > Create new Runner**): ```bash ./act_runner register \ --instance https://git.wizard.lu \ --token YOUR_RUNNER_TOKEN ``` Accept the default runner name and labels when prompted. Create a systemd service for persistent operation: ```bash sudo nano /etc/systemd/system/gitea-runner.service ``` ```ini [Unit] Description=Gitea Actions Runner After=network.target [Service] Type=simple User=samir WorkingDirectory=/home/samir/gitea-runner ExecStart=/home/samir/gitea-runner/act_runner daemon Restart=always RestartSec=10 [Install] WantedBy=multi-user.target ``` Enable and start: ```bash sudo systemctl daemon-reload sudo systemctl enable --now gitea-runner sudo systemctl status gitea-runner ``` Verify the runner shows as **Online** in Gitea: **Site Administration > Actions > Runners**. ## Step 16: Verify Full Deployment ```bash # All app containers running cd ~/apps/orion && docker compose --profile full ps # API health (via Caddy with SSL) curl https://api.wizard.lu/health # Main platform curl -I https://wizard.lu # Gitea curl -I https://git.wizard.lu # Flower curl -I https://flower.wizard.lu # Gitea runner status sudo systemctl status gitea-runner ``` --- ## Domain & Port Reference | Service | Internal Port | External Port | Domain (via Caddy) | |---|---|---|---| | Wizamart API | 8000 | 8001 | `api.wizard.lu` | | Main Platform | 8000 | 8001 | `wizard.lu` | | OMS Platform | 8000 | 8001 | `oms.lu` (TODO) | | Loyalty+ Platform | 8000 | 8001 | `loyaltyplus.lu` (TODO) | | PostgreSQL | 5432 | 5432 | (internal only) | | Redis | 6379 | 6380 | (internal only) | | Flower | 5555 | 5555 | `flower.wizard.lu` | | Gitea | 3000 | 3000 | `git.wizard.lu` | | Caddy | — | 80, 443 | (reverse proxy) | !!! note "Single backend, multiple domains" All platform domains route to the same FastAPI backend. The `PlatformContextMiddleware` identifies the platform from the `Host` header. See [Multi-Platform Architecture](../architecture/multi-platform-cms.md) for details. ## Directory Structure on Server ``` ~/ ├── gitea/ │ └── docker-compose.yml # Gitea + PostgreSQL ├── apps/ │ └── orion/ # Wizamart application │ ├── .env # Production environment │ ├── docker-compose.yml # App stack (API, DB, Redis, Celery) │ ├── logs/ # Application logs │ ├── uploads/ # User uploads │ └── exports/ # Export files └── gitea-runner/ # CI/CD runner (act_runner v0.2.13) ├── act_runner # symlink → act_runner-0.2.13-linux-arm64 ├── act_runner-0.2.13-linux-arm64 └── .runner # registration config ``` ## Troubleshooting ### Permission denied on logs The Docker container runs as `appuser` (UID 1000). Host-mounted volumes need matching ownership: ```bash sudo chown -R 1000:1000 logs uploads exports ``` ### Celery workers restarting Check logs for import errors: ```bash docker compose --profile full logs celery-worker --tail 30 ``` Common cause: stale task module references in `app/core/celery_config.py`. ### SSH service name on Ubuntu 24.04 Ubuntu 24.04 uses `ssh` not `sshd`: ```bash sudo systemctl restart ssh # correct sudo systemctl restart sshd # will fail ``` ### git pull fails with local changes If `docker-compose.yml` was edited on the server (e.g. passwords), stash before pulling: ```bash git stash git pull git stash pop ``` ## Maintenance ### Deploy updates (pull & rebuild) After pushing code to Gitea from local: ```bash cd ~/apps/orion && git pull && docker compose --profile full up -d --build ``` If there are local changes on the server (e.g. `.env` edits), stash first: ```bash cd ~/apps/orion git stash # Save local config changes git pull git stash pop # Re-apply local config docker compose --profile full up -d --build ``` If the update includes database migrations: ```bash docker compose --profile full exec -e PYTHONPATH=/app api python -m alembic upgrade heads ``` ### View logs ```bash # Follow all logs in real-time docker compose --profile full logs -f # Follow a specific service docker compose --profile full logs -f api docker compose --profile full logs -f celery-worker docker compose --profile full logs -f celery-beat docker compose --profile full logs -f flower # View last N lines (useful for debugging crashes) docker compose --profile full logs --tail=50 api docker compose --profile full logs --tail=100 celery-worker # Filter logs for errors docker compose --profile full logs api | grep -i "error\|exception\|failed" ``` ### Check container status ```bash # Overview of all containers (health, uptime, ports) docker compose --profile full ps # Watch for containers stuck in "Restarting" — indicates a crash loop # Healthy containers show: Up Xs (healthy) ``` ### Restart services ```bash # Restart a single service docker compose --profile full restart api # Restart everything docker compose --profile full restart # Full rebuild (after code changes) docker compose --profile full up -d --build ``` ### Quick access URLs After Caddy is configured: | Service | URL | |---|---| | Main Platform | `https://wizard.lu` | | API Swagger docs | `https://api.wizard.lu/docs` | | API ReDoc | `https://api.wizard.lu/redoc` | | Admin panel | `https://wizard.lu/admin/login` | | Health check | `https://api.wizard.lu/health` | | Gitea | `https://git.wizard.lu` | | Flower | `https://flower.wizard.lu` | | OMS Platform | `https://oms.lu` (after DNS) | | Loyalty+ Platform | `https://loyaltyplus.lu` (after DNS) | Direct IP access (temporary, until firewall rules are removed): | Service | URL | |---|---| | API | `http://91.99.65.229:8001/docs` | | Gitea | `http://91.99.65.229:3000` | | Flower | `http://91.99.65.229:5555` |