orion/docs/getting-started/database-setup-guide.md

Database Setup & Initialization Guide

Overview

This guide walks you through setting up the Orion database from scratch. Whether you're a new developer joining the team or need to reset your local database, this document covers everything you need to know.

---

Table of Contents

1. Prerequisites
2. First-Time Setup
3. Database Reset (Clean Slate)
4. Verifying Your Setup
5. Common Issues & Solutions
6. Database Migrations
7. Seeding Data

---

Prerequisites

Required Software

• Python 3.11+
• Alembic (installed via requirements.txt)
• SQLite (built into Python) or PostgreSQL (for production-like setup)

Environment Setup

# 1. Clone the repository
    git clone <orion-repo>
    cd orion-repo

# 2. Create virtual environment
    python -m venv venv

# 3. Activate virtual environment
# Windows:
    venv\Scripts\activate
# Linux/Mac:
    source venv/bin/activate

# 4. Install dependencies
    pip install -r requirements.txt

---

First-Time Setup

Step 1: Configure Database Connection

Create or update .env file in project root:

# Database Configuration
DATABASE_URL=sqlite:///./orion.db

# For PostgreSQL (production):
# DATABASE_URL=postgresql://user:password@localhost:5432/orion

# Other required settings
SECRET_KEY=your-secret-key-here-change-in-production
ALGORITHM=HS256
ACCESS_TOKEN_EXPIRE_MINUTES=30

---

Step 2: Initialize Alembic (First Time Only)

If alembic/ folder doesn't exist:

# Initialize Alembic
alembic init alembic

Configure alembic.ini:

# alembic.ini
# Find this line and update:
sqlalchemy.url = sqlite:///./orion.db

# Or for PostgreSQL:
# sqlalchemy.url = postgresql://user:password@localhost:5432/orion

Configure alembic/env.py:

# alembic/env.py

# Add this near the top (after imports)
import sys
from pathlib import Path

# Add project root to Python path
sys.path.append(str(Path(__file__).parents[1]))

# ============================================================================
# CRITICAL: Import ALL database models here
# ============================================================================
from models.database.user import User
from models.database.store import Store
from models.database.store_domain import StoreDomain
from models.database.store_theme import StoreTheme
from models.database.customer import Customer
from models.database.team import Team, TeamMember, TeamInvitation
from models.database.product import Product
from models.database.marketplace_product import MarketplaceProduct
from models.database.inventory import Inventory
from models.database.order import Order
from models.database.marketplace_import_job import MarketplaceImportJob

# Import Base
from app.core.database import Base

# Set target metadata
target_metadata = Base.metadata

---

Step 3: Generate Initial Migration

# Generate migration from your models
alembic revision --autogenerate -m "Initial migration - all tables"

# You should see output listing all detected tables:
# INFO  [alembic.autogenerate.compare] Detected added table 'users'
# INFO  [alembic.autogenerate.compare] Detected added table 'stores'
# INFO  [alembic.autogenerate.compare] Detected added table 'store_themes'
# etc.

---

Step 4: Apply Migration

# Apply the migration to create all tables
alembic upgrade head

# You should see:
# INFO  [alembic.runtime.migration] Running upgrade  -> abc123def456, Initial migration - all tables

---

Step 5: Seed Initial Data

Create an admin user and test store:

python scripts/seed_database.py

Or manually:

# Run Python interactive shell
python

# Then execute:
from app.core.database import SessionLocal
from models.database.user import User
from models.database.store import Store
from middleware.auth import AuthManager
from datetime import datetime, timezone

db = SessionLocal()
auth_manager = AuthManager()

# Create admin user
admin = User(
    username="admin",
    email="admin@orion.lu",
    hashed_password=auth_manager.hash_password("admin123"),
    role="admin",
    is_active=True,
    created_at=datetime.now(timezone.utc),
    updated_at=datetime.now(timezone.utc)
)
db.add(admin)
db.flush()

# Create test store
store = Store(
    store_code="TESTSTORE",
    subdomain="teststore",
    name="Test Store",
    description="Development test store",
    owner_user_id=admin.id,
    contact_email="contact@teststore.com",
    is_active=True,
    is_verified=True,
    created_at=datetime.now(timezone.utc),
    updated_at=datetime.now(timezone.utc)
)
db.add(store)

db.commit()
print("✅ Admin user and test store created!")
db.close()
exit()

---

Step 6: Start the Application

# Start FastAPI server
python -m uvicorn app.main:app --reload

# Server should start on http://localhost:8000

Test the setup:

• Admin login: http://localhost:8000/admin/login
• Username: admin
• Password: admin123

---

Database Reset (Clean Slate)

Use this when you want to completely reset your database and start fresh.

⚠️ WARNING: This will delete ALL data! Make a backup if needed.

---

Quick Reset Script (Windows PowerShell)

Create scripts/reset_database.ps1:

# scripts/reset_database.ps1
Write-Host "🗑️  Database Reset Script" -ForegroundColor Cyan
Write-Host "This will delete ALL data!" -ForegroundColor Red
$confirm = Read-Host "Type 'YES' to continue"

if ($confirm -ne "YES") {
    Write-Host "Aborted." -ForegroundColor Yellow
    exit
}

Write-Host ""
Write-Host "Step 1: Backing up current database..." -ForegroundColor Yellow
if (Test-Path orion.db) {
    $backupName = "orion_backup_$(Get-Date -Format 'yyyyMMdd_HHmmss').db"
    Copy-Item orion.db $backupName
    Write-Host "✓ Backup created: $backupName" -ForegroundColor Green
}

Write-Host ""
Write-Host "Step 2: Removing old migrations..." -ForegroundColor Yellow
Remove-Item alembic\versions\*.py -Exclude __init__.py -ErrorAction SilentlyContinue
Remove-Item -Recurse -Force alembic\versions\__pycache__ -ErrorAction SilentlyContinue

# Ensure __init__.py exists
if (-not (Test-Path alembic\versions\__init__.py)) {
    New-Item -Path alembic\versions\__init__.py -ItemType File | Out-Null
}
Write-Host "✓ Old migrations removed" -ForegroundColor Green

Write-Host ""
Write-Host "Step 3: Deleting database..." -ForegroundColor Yellow
Remove-Item orion.db -ErrorAction SilentlyContinue
Write-Host "✓ Database deleted" -ForegroundColor Green

Write-Host ""
Write-Host "Step 4: Generating new migration..." -ForegroundColor Yellow
alembic revision --autogenerate -m "Initial migration - all tables"
Write-Host "✓ Migration generated" -ForegroundColor Green

Write-Host ""
Write-Host "Step 5: Applying migration..." -ForegroundColor Yellow
alembic upgrade head
Write-Host "✓ Migration applied" -ForegroundColor Green

Write-Host ""
Write-Host "Step 6: Verifying tables..." -ForegroundColor Yellow
python -c @"
import sqlite3
conn = sqlite3.connect('orion.db')
cursor = conn.cursor()
cursor.execute('SELECT name FROM sqlite_master WHERE type=\"table\" ORDER BY name;')
tables = [t[0] for t in cursor.fetchall()]
print('\n📊 Tables created:')
for t in tables:
    print(f'  ✓ {t}')
print(f'\n✅ Total: {len(tables)} tables')
if 'store_themes' in tables:
    print('✅ store_themes table confirmed!')
conn.close()
"@

Write-Host ""
Write-Host "Step 7: Seeding initial data..." -ForegroundColor Yellow
python scripts/seed_database.py

Write-Host ""
Write-Host "✅ Database reset complete!" -ForegroundColor Green
Write-Host ""
Write-Host "Next steps:" -ForegroundColor Cyan
Write-Host "  1. Start server: python -m uvicorn app.main:app --reload" -ForegroundColor White
Write-Host "  2. Login: http://localhost:8000/admin/login" -ForegroundColor White
Write-Host "  3. Username: admin | Password: admin123" -ForegroundColor White

Run it:

# First time only - allow script execution
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

# Run the reset
.\scripts\reset_database.ps1

---

Quick Reset Script (Linux/Mac Bash)

Create scripts/reset_database.sh:

#!/bin/bash
# scripts/reset_database.sh

echo "🗑️  Database Reset Script"
echo "⚠️  This will delete ALL data!"
read -p "Type 'YES' to continue: " confirm

if [ "$confirm" != "YES" ]; then
    echo "Aborted."
    exit 1
fi

echo ""
echo "Step 1: Backing up current database..."
if [ -f orion.db ]; then
    backup_name="orion_backup_$(date +%Y%m%d_%H%M%S).db"
    cp orion.db "$backup_name"
    echo "✓ Backup created: $backup_name"
fi

echo ""
echo "Step 2: Removing old migrations..."
find alembic/versions -type f -name "*.py" ! -name "__init__.py" -delete
rm -rf alembic/versions/__pycache__
touch alembic/versions/__init__.py
echo "✓ Old migrations removed"

echo ""
echo "Step 3: Deleting database..."
rm -f orion.db
echo "✓ Database deleted"

echo ""
echo "Step 4: Generating new migration..."
alembic revision --autogenerate -m "Initial migration - all tables"
echo "✓ Migration generated"

echo ""
echo "Step 5: Applying migration..."
alembic upgrade head
echo "✓ Migration applied"

echo ""
echo "Step 6: Verifying tables..."
python -c "
import sqlite3
conn = sqlite3.connect('orion.db')
cursor = conn.cursor()
cursor.execute('SELECT name FROM sqlite_master WHERE type=\"table\" ORDER BY name;')
tables = [t[0] for t in cursor.fetchall()]
print('\n📊 Tables created:')
for t in tables:
    print(f'  ✓ {t}')
print(f'\n✅ Total: {len(tables)} tables')
if 'store_themes' in tables:
    print('✅ store_themes table confirmed!')
conn.close()
"

echo ""
echo "Step 7: Seeding initial data..."
python scripts/seed_database.py

echo ""
echo "✅ Database reset complete!"
echo ""
echo "Next steps:"
echo "  1. Start server: python -m uvicorn app.main:app --reload"
echo "  2. Login: http://localhost:8000/admin/login"
echo "  3. Username: admin | Password: admin123"

Run it:

chmod +x scripts/reset_database.sh
./scripts/reset_database.sh

---

Manual Reset Steps

If you prefer to run commands manually:

# 1. Backup (optional)
cp orion.db orion_backup.db  # Windows: copy orion.db orion_backup.db

# 2. Remove migrations
rm alembic/versions/*.py  # Windows: Remove-Item alembic\versions\*.py -Exclude __init__.py
touch alembic/versions/__init__.py  # Windows: New-Item -Path alembic\versions\__init__.py

# 3. Delete database
rm orion.db  # Windows: Remove-Item orion.db

# 4. Generate migration
alembic revision --autogenerate -m "Initial migration - all tables"

# 5. Apply migration
alembic upgrade head

# 6. Seed data
python scripts/seed_database.py

---

Verifying Your Setup

Check Alembic Status

# Check current migration version
alembic current

# Should show:
# <revision_id> (head)

Verify Tables Exist

Using Python:

import sqlite3

conn = sqlite3.connect('orion.db')
cursor = conn.cursor()

# List all tables
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name;")
tables = cursor.fetchall()

print("📊 Database Tables:")
for table in tables:
    print(f"  ✓ {table[0]}")

# Check specific table structure
cursor.execute("PRAGMA table_info(store_themes);")
columns = cursor.fetchall()

print("\n📋 store_themes columns:")
for col in columns:
    print(f"  - {col[1]} ({col[2]})")

conn.close()

Test Database Connection

# Start Python shell
python

# Test connection
from app.core.database import SessionLocal, engine
from sqlalchemy import text

db = SessionLocal()
result = db.execute(text("SELECT name FROM sqlite_master WHERE type='table';"))
tables = [row[0] for row in result]

print("✅ Database connection successful!")
print(f"📊 Tables found: {len(tables)}")
for table in tables:
    print(f"  - {table}")

db.close()
exit()

Test Application Startup

# Start server
python -m uvicorn app.main:app --reload

# Should see:
# INFO:     Uvicorn running on http://127.0.0.1:8000
# INFO:     Application startup complete.

No errors = successful setup!

---

Common Issues & Solutions

Issue 1: "No such table" Error

Symptom:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: store_themes

Solution:

# 1. Check if model is imported in alembic/env.py
# Open alembic/env.py and verify:
from models.database.store_theme import StoreTheme

# 2. Regenerate migration
alembic revision --autogenerate -m "Add missing tables"

# 3. Apply migration
alembic upgrade head

---

Issue 2: "Target database is not up to date"

Symptom:

ERROR [alembic.util.messaging] Target database is not up to date.

Solution:

# Option 1: Stamp to current head
alembic stamp head

# Option 2: Check and upgrade
alembic current  # See current version
alembic upgrade head  # Upgrade to latest

# Option 3: Full reset (see Database Reset section)

---

Issue 3: Database Locked

Symptom:

sqlite3.OperationalError: database is locked

Solution:

# 1. Stop the FastAPI server (Ctrl+C)

# 2. Kill any Python processes
# Windows:
taskkill /F /IM python.exe

# Linux/Mac:
pkill python

# 3. Close any database browser applications

# 4. Try again
alembic upgrade head

---

Issue 4: Migration File Conflicts

Symptom:

Multiple head revisions are present

Solution:

# Option 1: Merge migrations
alembic merge heads -m "Merge migrations"
alembic upgrade head

# Option 2: Clean reset (recommended for development)
# See Database Reset section above

---

Issue 5: Import Errors in alembic/env.py

Symptom:

ModuleNotFoundError: No module named 'models'

Solution:

Check alembic/env.py has correct path setup:

# alembic/env.py (top of file)
import sys
from pathlib import Path

# Add project root to Python path
sys.path.append(str(Path(__file__).parents[1]))

---

Database Migrations

Creating New Migrations

When you add/modify models:

# 1. Update your model file (e.g., models/database/store_theme.py)

# 2. Generate migration
alembic revision --autogenerate -m "Add new field to store_themes"

# 3. Review the generated migration
# Check: alembic/versions/<revision>_add_new_field_to_store_themes.py

# 4. Apply migration
alembic upgrade head

---

Migration Best Practices

✅ DO:

• Review generated migrations before applying
• Use descriptive migration messages
• Test migrations on development database first
• Keep migrations small and focused
• Commit migrations to version control

❌ DON'T:

• Edit applied migrations (create new ones instead)
• Delete migration files from version control
• Skip migration review
• Combine unrelated changes in one migration

---

Rolling Back Migrations

# Downgrade one version
alembic downgrade -1

# Downgrade to specific revision
alembic downgrade <revision_id>

# Downgrade to base (empty database)
alembic downgrade base

---

Seeding Data

Development Data Seed

Create scripts/seed_database.py:

# scripts/seed_database.py
"""
Seed the database with initial development data.
"""

from app.core.database import SessionLocal
from models.database.user import User
from models.database.store import Store
from app.core.security import get_password_hash
from datetime import datetime, timezone


def seed_database():
    """Seed database with initial data."""
    db = SessionLocal()

    try:
        print("🌱 Seeding database...")

        # Check if admin already exists
        existing_admin = db.query(User).filter(User.username == "admin").first()
        if existing_admin:
            print("⚠️  Admin user already exists, skipping...")
            return

        # Create admin user
        admin = User(
            username="admin",
            email="admin@orion.lu",
            hashed_password=get_password_hash("admin123"),
            is_admin=True,
            is_active=True,
            created_at=datetime.now(timezone.utc),
            updated_at=datetime.now(timezone.utc)
        )
        db.add(admin)
        db.flush()
        print(f"✓ Admin user created (ID: {admin.id})")

        # Create test store
        store = Store(
            store_code="TESTSTORE",
            subdomain="teststore",
            name="Test Store",
            description="Development test store",
            owner_user_id=admin.id,
            contact_email="contact@teststore.com",
            is_active=True,
            is_verified=True,
            created_at=datetime.now(timezone.utc),
            updated_at=datetime.now(timezone.utc)
        )
        db.add(store)
        db.flush()
        print(f"✓ Test store created: {store.store_code}")

        db.commit()

        print("\n✅ Database seeded successfully!")
        print("\n📝 Login Credentials:")
        print("   URL: http://localhost:8000/admin/login")
        print("   Username: admin")
        print("   Password: admin123")

    except Exception as e:
        db.rollback()
        print(f"❌ Error seeding database: {e}")
        raise
    finally:
        db.close()


if __name__ == "__main__":
    seed_database()

Run it:

python scripts/seed_database.py

---

Production Data Migration

For production, create separate seed scripts:

# scripts/seed_production.py
# - Create only essential admin user
# - No test data
# - Strong passwords from environment variables

---

Quick Reference

Essential Commands

# Check migration status
alembic current

# Generate new migration
alembic revision --autogenerate -m "Description"

# Apply migrations
alembic upgrade head

# Rollback one migration
alembic downgrade -1

# View migration history
alembic history

# Seed database
python scripts/seed_database.py

# Start server
python -m uvicorn app.main:app --reload

---

File Locations

project/
├── alembic/
│   ├── versions/           # Migration files here
│   │   └── __init__.py
│   ├── env.py             # Alembic configuration (import models here!)
│   └── script.py.mako
├── app/
│   └── core/
│       └── database.py    # Database connection
├── models/
│   └── database/          # SQLAlchemy models
│       ├── user.py
│       ├── store.py
│       ├── store_theme.py
│       └── ...
├── scripts/
│   ├── seed_database.py   # Development seed
│   └── reset_database.ps1 # Reset script
├── alembic.ini            # Alembic config
├── orion.db           # SQLite database (gitignored)
└── .env                  # Environment variables (gitignored)

---

Getting Help

Internal Resources

• Architecture Guide: docs/PROPER_ARCHITECTURE_GUIDE.md
• Exception Handling: docs/EXCEPTION_PATTERN_EXPLAINED.md
• Admin Feature Guide: docs/ADMIN_FEATURE_INTEGRATION_GUIDE.md

External Resources

• Alembic Documentation
• SQLAlchemy Documentation
• FastAPI Documentation

---

Contributing

When adding new models:

1. ✅ Create the model in models/database/
2. ✅ Import it in alembic/env.py
3. ✅ Generate migration: alembic revision --autogenerate -m "Add XYZ model"
4. ✅ Review the migration file
5. ✅ Test locally: alembic upgrade head
6. ✅ Commit both model and migration files

---

Last Updated: 2025-10-27 Maintainer: Development Team Questions? Contact the team lead or check internal documentation.