Database & Migrations: - Update all Alembic migrations for PostgreSQL compatibility - Remove SQLite-specific syntax (AUTOINCREMENT, etc.) - Add database utility helpers for PostgreSQL operations - Fix services to use PostgreSQL-compatible queries Documentation: - Add comprehensive Docker deployment guide - Add production deployment documentation - Add infrastructure architecture documentation - Update database setup guide for PostgreSQL-only - Expand troubleshooting guide Architecture & Validation: - Add migration.yaml rules for SQL compatibility checking - Enhance validate_architecture.py with migration validation - Update architecture rules to validate Alembic migrations Development: - Fix duplicate install-all target in Makefile - Add Celery/Redis validation to install.py script - Add docker-compose.test.yml for CI testing - Add squash_migrations.py utility script - Update tests for PostgreSQL compatibility - Improve test fixtures in conftest.py Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
345 lines
8.7 KiB
Markdown
345 lines
8.7 KiB
Markdown
# Database Setup Guide
|
|
|
|
This guide will help new team members set up and understand the database system used in this project.
|
|
|
|
## Quick Setup (New Team Members)
|
|
|
|
After cloning the repository, follow these steps to get your database ready:
|
|
|
|
### 1. Install Dependencies
|
|
```bash
|
|
# Install all dependencies including Alembic
|
|
make install-all
|
|
```
|
|
|
|
### 2. Start PostgreSQL Database
|
|
```bash
|
|
# Start the development database with Docker
|
|
make docker-up
|
|
|
|
# Or manually:
|
|
docker-compose up -d db
|
|
```
|
|
|
|
### 3. Set Up Environment
|
|
```bash
|
|
# Copy the example environment file
|
|
cp .env.example .env
|
|
|
|
# Edit .env with your database configuration
|
|
# The default works with docker-compose:
|
|
DATABASE_URL=postgresql://wizamart_user:secure_password@localhost:5432/wizamart_db
|
|
```
|
|
|
|
!!! note "PostgreSQL Required"
|
|
This project requires PostgreSQL. SQLite is not supported.
|
|
Docker Compose provides the easiest way to run PostgreSQL locally.
|
|
|
|
### 3. Run Database Migrations
|
|
```bash
|
|
# Apply all migrations to create the database schema
|
|
make migrate-up
|
|
```
|
|
|
|
### 4. Seed Test Data (Optional)
|
|
```bash
|
|
# Create test vendors, products, and inventory
|
|
python scripts/create_test_data.py
|
|
|
|
# Create inventory entries for existing products
|
|
python scripts/create_inventory.py
|
|
|
|
# Create landing pages for vendors
|
|
python scripts/create_landing_page.py
|
|
```
|
|
|
|
### 5. Verify Setup
|
|
```bash
|
|
# Check that everything is working
|
|
make verify-setup
|
|
```
|
|
|
|
### 6. Start Development
|
|
```bash
|
|
# Start the development server
|
|
make dev
|
|
```
|
|
|
|
## Understanding Our Database System
|
|
|
|
### What is Alembic?
|
|
Alembic is a database migration tool that helps us:
|
|
- **Version control our database schema** - Every database change is tracked
|
|
- **Share schema changes with the team** - When you pull code, you get database updates too
|
|
- **Deploy safely** - Production deployments include database updates
|
|
- **Roll back if needed** - We can undo problematic database changes
|
|
|
|
### Key Concepts
|
|
|
|
**Migrations**: Files that describe how to change the database schema
|
|
- Located in `alembic/versions/`
|
|
- Each migration has a unique ID and timestamp
|
|
- Migrations run in order to build up the complete schema
|
|
|
|
**Migration Status**: Alembic tracks which migrations have been applied
|
|
- `alembic current` - Shows the current migration
|
|
- `alembic history` - Shows all available migrations
|
|
|
|
## Daily Workflow
|
|
|
|
### When You Pull Code
|
|
```bash
|
|
# After pulling changes from git, check for new migrations
|
|
make migrate-status
|
|
|
|
# If there are pending migrations, apply them
|
|
make migrate-up
|
|
```
|
|
|
|
### When Working with Models
|
|
```bash
|
|
# After modifying SQLAlchemy models, create a migration
|
|
make migrate-create message="add_user_profile_table"
|
|
|
|
# Review the generated migration file in alembic/versions/
|
|
# Then apply it
|
|
make migrate-up
|
|
```
|
|
|
|
## Common Scenarios
|
|
|
|
### First Time Setup
|
|
```bash
|
|
make setup # This handles everything automatically
|
|
```
|
|
|
|
### Database is Out of Sync
|
|
```bash
|
|
# Check current status
|
|
make migrate-status
|
|
|
|
# Apply any missing migrations
|
|
make migrate-up
|
|
```
|
|
|
|
### Something Went Wrong
|
|
```bash
|
|
# Create a backup first
|
|
make backup-db
|
|
|
|
# Check what migrations are available
|
|
make migrate-status
|
|
|
|
# If you need to rollback the last migration
|
|
make migrate-down
|
|
```
|
|
|
|
### Starting Fresh (Development Only)
|
|
```bash
|
|
# Backup first (just in case)
|
|
make backup-db
|
|
|
|
# Delete database and recreate from scratch
|
|
del wizamart.db # or drop PostgreSQL database
|
|
make migrate-up
|
|
```
|
|
|
|
## Environment-Specific Setup
|
|
|
|
### Development (Docker - Recommended)
|
|
```env
|
|
DATABASE_URL=postgresql://wizamart_user:secure_password@localhost:5432/wizamart_db
|
|
```
|
|
- Start with `make docker-up`
|
|
- Consistent environment matching production
|
|
- Easy to reset with `make db-reset`
|
|
|
|
### Development (Local PostgreSQL)
|
|
```env
|
|
DATABASE_URL=postgresql://user:password@localhost:5432/wizamart_dev
|
|
```
|
|
- Use if you prefer a local PostgreSQL installation
|
|
- Ensure PostgreSQL 15+ is installed
|
|
|
|
### Production
|
|
```env
|
|
DATABASE_URL=postgresql://user:password@production-host:5432/wizamart_prod
|
|
```
|
|
- PostgreSQL is required for production
|
|
- Migrations are applied automatically during deployment
|
|
|
|
## Troubleshooting
|
|
|
|
### "No module named 'models'"
|
|
```bash
|
|
# Make sure you're in the project root and have activated the virtual environment
|
|
cd /path/to/project
|
|
source venv/bin/activate # or venv\Scripts\activate on Windows
|
|
```
|
|
|
|
### "Database connection failed"
|
|
```bash
|
|
# Check your DATABASE_URL in .env
|
|
# For SQLite, make sure the directory exists and is writable
|
|
# For PostgreSQL, ensure the server is running and credentials are correct
|
|
```
|
|
|
|
### "Migration conflicts"
|
|
```bash
|
|
# Check migration status
|
|
make migrate-status
|
|
|
|
# If there are conflicts, contact the team lead
|
|
# We may need to merge or reorder migrations
|
|
```
|
|
|
|
### Need Help?
|
|
- Check `make help-db` for database commands
|
|
- Use `make verify-setup` to diagnose issues
|
|
- Ask in the team chat if you're stuck
|
|
|
|
## Best Practices
|
|
|
|
1. **Always run migrations after pulling code**
|
|
2. **Never edit migration files after they're committed**
|
|
3. **Test your migrations on a copy of production data**
|
|
4. **Include meaningful messages when creating migrations**
|
|
5. **Review generated migrations before applying them**
|
|
|
|
## Data Seeding for Development
|
|
|
|
### Overview
|
|
|
|
After setting up your database schema with migrations, you'll need test data to develop and test features. We provide several scripts to populate your database with sample data.
|
|
|
|
### Available Seeding Scripts
|
|
|
|
#### 1. Create Test Data
|
|
**Script:** `scripts/create_test_data.py`
|
|
|
|
Creates complete test data including vendors, marketplace products, vendor products, themes, and content pages.
|
|
|
|
```bash
|
|
python scripts/create_test_data.py
|
|
```
|
|
|
|
**What it creates:**
|
|
- 3 test vendors (WizaMart, Fashion Hub, The Book Store)
|
|
- 20 marketplace products per vendor
|
|
- Vendor-specific products with pricing
|
|
- Vendor themes with custom colors
|
|
- Sample CMS content pages (About, Contact, FAQ, etc.)
|
|
|
|
#### 2. Create Inventory Entries
|
|
**Script:** `scripts/create_inventory.py`
|
|
|
|
Creates inventory entries for products that don't have inventory. This is **required** for the shop to function properly, as products with 0 inventory cannot be added to cart.
|
|
|
|
```bash
|
|
python scripts/create_inventory.py
|
|
```
|
|
|
|
**What it does:**
|
|
- Finds all products without inventory entries
|
|
- Creates inventory record for each product
|
|
- Sets 100 units available in "Main Warehouse"
|
|
- Can be run multiple times (only creates missing entries)
|
|
|
|
**Why it's needed:**
|
|
- Product `available_inventory` is calculated from inventory entries
|
|
- Empty inventory table = 0 inventory for all products
|
|
- Add to cart functionality requires `available_inventory > 0`
|
|
|
|
#### 3. Create Landing Pages
|
|
**Script:** `scripts/create_landing_page.py`
|
|
|
|
Creates or updates landing pages for vendors with different templates.
|
|
|
|
```bash
|
|
# Interactive mode
|
|
python scripts/create_landing_page.py
|
|
|
|
# Or programmatically in Python
|
|
from scripts.create_landing_page import create_landing_page
|
|
create_landing_page('wizamart', template='modern')
|
|
```
|
|
|
|
**Available templates:**
|
|
- `default` - Clean professional layout with 3-column quick links
|
|
- `minimal` - Ultra-simple centered design with single CTA
|
|
- `modern` - Full-screen hero with animations and features
|
|
- `full` - Maximum features with split-screen hero and stats
|
|
|
|
### Complete Setup Workflow
|
|
|
|
For a fresh development environment with test data:
|
|
|
|
```bash
|
|
# 1. Setup database schema
|
|
make migrate-up
|
|
|
|
# 2. Create test vendors and products
|
|
python scripts/create_test_data.py
|
|
|
|
# 3. Create inventory (REQUIRED for shop to work)
|
|
python scripts/create_inventory.py
|
|
|
|
# 4. Create landing pages
|
|
python scripts/create_landing_page.py
|
|
# Follow interactive prompts to create landing pages
|
|
|
|
# 5. Start the server
|
|
make dev
|
|
|
|
# 6. Test the shop
|
|
# Visit: http://localhost:8000/vendors/wizamart/shop/
|
|
```
|
|
|
|
### Common Issues
|
|
|
|
**Problem: Products can't be added to cart**
|
|
```bash
|
|
# Solution: Create inventory entries
|
|
python scripts/create_inventory.py
|
|
```
|
|
|
|
**Problem: Landing page shows 404**
|
|
```bash
|
|
# Solution: Create landing page for the vendor
|
|
python scripts/create_landing_page.py
|
|
# OR the vendor will auto-redirect to /shop/
|
|
```
|
|
|
|
**Problem: No products showing in shop**
|
|
```bash
|
|
# Solution: Create test data
|
|
python scripts/create_test_data.py
|
|
```
|
|
|
|
### Database Reset (Development Only)
|
|
|
|
To start completely fresh:
|
|
|
|
```bash
|
|
# Complete reset (drops all data and recreates)
|
|
make db-reset
|
|
|
|
# Or step by step:
|
|
# 1. Rollback all migrations
|
|
make migrate-down # Run multiple times or use: alembic downgrade base
|
|
|
|
# 2. Recreate schema
|
|
make migrate-up
|
|
|
|
# 3. Seed data
|
|
make init-prod
|
|
make seed-demo
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
- [Shop Setup Guide](../guides/shop-setup.md) - Configure vendor storefronts
|
|
- [Landing Pages Guide](../features/vendor-landing-pages.md) - Customize landing pages
|
|
- [Database Migrations Guide](../development/migration/database-migrations.md) - Advanced migration workflows
|
|
- [API Documentation](../api/index.md) - Start building features
|