sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: PostgreSQL migration compatibility and infrastructure improvements

Browse Source 
Database & Migrations:
- Update all Alembic migrations for PostgreSQL compatibility
- Remove SQLite-specific syntax (AUTOINCREMENT, etc.)
- Add database utility helpers for PostgreSQL operations
- Fix services to use PostgreSQL-compatible queries

Documentation:
- Add comprehensive Docker deployment guide
- Add production deployment documentation
- Add infrastructure architecture documentation
- Update database setup guide for PostgreSQL-only
- Expand troubleshooting guide

Architecture & Validation:
- Add migration.yaml rules for SQL compatibility checking
- Enhance validate_architecture.py with migration validation
- Update architecture rules to validate Alembic migrations

Development:
- Fix duplicate install-all target in Makefile
- Add Celery/Redis validation to install.py script
- Add docker-compose.test.yml for CI testing
- Add squash_migrations.py utility script
- Update tests for PostgreSQL compatibility
- Improve test fixtures in conftest.py

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Samir Boulahtit
2026-01-11 17:52:28 +01:00
parent 2792414395
commit 3614d448e4
45 changed files with 3179 additions and 507 deletions
Download Patch File Download Diff File Expand all files Collapse all files

94
docs/development/migration/database-migrations.md
View File

@@ -54,6 +54,100 @@ make backup-db
make verify-setup
```
## Alembic Commands Explained
Understanding what each Alembic command does is essential for managing database migrations effectively.
### Core Commands Reference
| Command | Description |
|---------|-------------|
| `alembic upgrade head` | Apply **all** pending migrations to get to the latest version |
| `alembic upgrade +1` | Apply only the **next** single migration (one step forward) |
| `alembic downgrade -1` | Roll back the **last** applied migration (one step back) |
| `alembic downgrade base` | Roll back **all** migrations (returns to empty schema) |
| `alembic current` | Show which migration the database is currently at |
| `alembic history` | List all migrations in the chain |
| `alembic heads` | Show the latest migration revision(s) |
### Visual Example
```
Migration Chain: [base] → [A] → [B] → [C] → [D] (head)
↑
└── current database state (at revision B)
Command Result
─────────────────────────────────────────────────────────
alembic upgrade head → Database moves to [D]
alembic upgrade +1 → Database moves to [C]
alembic downgrade -1 → Database moves to [A]
alembic downgrade base → Database returns to [base] (empty schema)
alembic current → Shows "B" (current revision)
```
### Makefile Shortcuts
| Make Command | Alembic Equivalent | Description |
|--------------|-------------------|-------------|
| `make migrate-up` | `alembic upgrade head` | Apply all pending migrations |
| `make migrate-down` | `alembic downgrade -1` | Roll back last migration |
| `make migrate-status` | `alembic current` + `alembic history` | Show current state and history |
### Additional Useful Commands
```bash
# Show detailed migration history
alembic history --verbose
# Upgrade/downgrade to a specific revision
alembic upgrade abc123def456
alembic downgrade abc123def456
# Show what SQL would be generated (without executing)
alembic upgrade head --sql
# Mark database as being at a specific revision (without running migrations)
alembic stamp head
alembic stamp abc123def456
# Show the current revision ID only
alembic current --verbose
```
### Database Initialization Workflow
For setting up a new database:
```bash
# Option 1: Empty schema only (just tables, no data)
make migrate-up
# Option 2: Schema + production essentials (admin user, settings, CMS, email templates)
make init-prod
# Option 3: Full development setup (schema + production data + demo data)
make db-setup
# Or step by step:
make init-prod
make seed-demo
```
### Reset Workflow
For completely resetting the database:
```bash
# Nuclear reset: deletes database file, recreates schema, seeds all data
make db-reset
```
> **Note**: `make db-reset` rolls back all migrations and recreates the schema from scratch.
!!! note "PostgreSQL Required"
This project uses PostgreSQL exclusively. SQLite is not supported.
Start the development database with: `make docker-up`
## Development Workflows
### Adding New Database Fields

149
docs/development/troubleshooting.md
View File

@@ -1,11 +1,12 @@
# PyCharm Troubleshooting Guide
# Development Troubleshooting Guide
Common PyCharm issues and their solutions for the development team.
Common development issues and their solutions for the team.
---
## Table of Contents
- [Docker DNS/WiFi Issues (Digital Nomads)](#docker-dnswifi-issues-digital-nomads)
- [Go to Declaration Not Working (Ctrl+B)](#go-to-declaration-not-working-ctrlb)
- [Import Errors and Red Underlines](#import-errors-and-red-underlines)
- [Python Interpreter Issues](#python-interpreter-issues)
@@ -17,6 +18,150 @@ Common PyCharm issues and their solutions for the development team.
---
## Docker DNS/WiFi Issues (Digital Nomads)
### Problem
When using Docker on Linux (Ubuntu/Xubuntu), the `docker0` bridge network interface can interfere with DNS resolution, causing:
- Hotel/public WiFi captive portals not loading
- DNS lookups failing or timing out
- Internet connectivity issues when Docker is running
- WiFi works fine until Docker starts
This happens because Docker's bridge network (`docker0`) can take priority over your WiFi connection for DNS resolution.
### Solution: Lower docker0 Priority (Recommended)
This solution lets Docker work normally while ensuring your WiFi DNS takes priority. Perfect for digital nomads working from hotels, cafes, and co-working spaces.
**Step 1: Create the NetworkManager dispatcher script**
```bash
sudo tee /etc/NetworkManager/dispatcher.d/99-docker-dns-fix << 'EOF'
#!/bin/bash
# Lower docker0 metric so WiFi takes priority for DNS
# This prevents Docker from interfering with hotel/public WiFi captive portals
if [ "$1" = "docker0" ]; then
ip route del default via 172.17.0.1 dev docker0 2>/dev/null || true
fi
EOF
```
**Step 2: Make it executable**
```bash
sudo chmod +x /etc/NetworkManager/dispatcher.d/99-docker-dns-fix
```
**Step 3: Restart NetworkManager**
```bash
sudo systemctl restart NetworkManager
```
**Step 4: Verify it works**
```bash
# Start Docker
sudo systemctl start docker
# Check routes - docker0 should NOT have a default route
ip route | grep docker0
# Should return nothing or non-default routes only
# Your WiFi should still work
ping -c 3 google.com
```
### Alternative Solutions
#### Option A: Use Google DNS (Simple but less flexible)
If you don't need hotel captive portal DNS:
```bash
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json << 'EOF'
{
"dns": ["8.8.8.8", "8.8.4.4"],
"dns-opts": ["ndots:1"]
}
EOF
sudo systemctl restart docker
```
!!! warning "Captive Portals"
This option uses hardcoded Google DNS, which means hotel WiFi captive portals
(login pages) may not work properly since they rely on DNS hijacking.
#### Option B: Stop Docker when not in use
```bash
# When you need WiFi without Docker interference
sudo systemctl stop docker
sudo ip link set docker0 down
# When you need Docker again
sudo systemctl start docker
```
### Installing Docker on Xubuntu/Ubuntu
If you haven't installed Docker yet:
```bash
# Update package index
sudo apt update
# Install prerequisites
sudo apt install -y ca-certificates curl gnupg
# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
# Add the repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
# Add your user to docker group (run docker without sudo)
sudo usermod -aG docker $USER
# Apply the DNS fix BEFORE using Docker
sudo tee /etc/NetworkManager/dispatcher.d/99-docker-dns-fix << 'EOF'
#!/bin/bash
if [ "$1" = "docker0" ]; then
ip route del default via 172.17.0.1 dev docker0 2>/dev/null || true
fi
EOF
sudo chmod +x /etc/NetworkManager/dispatcher.d/99-docker-dns-fix
# Log out and back in for docker group, then verify
docker run hello-world
```
### Using Docker with this Project
After installing Docker with the DNS fix:
```bash
# Start PostgreSQL for development
make docker-up
# Run tests (auto-starts test database)
make test
# Stop when done
make docker-down
```
---
## Go to Declaration Not Working (Ctrl+B)
### Problem