Files
orion/docs/development/troubleshooting.md
Samir Boulahtit 3614d448e4 chore: PostgreSQL migration compatibility and infrastructure improvements
Database & Migrations:
- Update all Alembic migrations for PostgreSQL compatibility
- Remove SQLite-specific syntax (AUTOINCREMENT, etc.)
- Add database utility helpers for PostgreSQL operations
- Fix services to use PostgreSQL-compatible queries

Documentation:
- Add comprehensive Docker deployment guide
- Add production deployment documentation
- Add infrastructure architecture documentation
- Update database setup guide for PostgreSQL-only
- Expand troubleshooting guide

Architecture & Validation:
- Add migration.yaml rules for SQL compatibility checking
- Enhance validate_architecture.py with migration validation
- Update architecture rules to validate Alembic migrations

Development:
- Fix duplicate install-all target in Makefile
- Add Celery/Redis validation to install.py script
- Add docker-compose.test.yml for CI testing
- Add squash_migrations.py utility script
- Update tests for PostgreSQL compatibility
- Improve test fixtures in conftest.py

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:52:28 +01:00

684 lines
17 KiB
Markdown

# Development Troubleshooting Guide
Common development issues and their solutions for the team.
---
## Table of Contents
- [Docker DNS/WiFi Issues (Digital Nomads)](#docker-dnswifi-issues-digital-nomads)
- [Go to Declaration Not Working (Ctrl+B)](#go-to-declaration-not-working-ctrlb)
- [Import Errors and Red Underlines](#import-errors-and-red-underlines)
- [Python Interpreter Issues](#python-interpreter-issues)
- [Debugging Not Working](#debugging-not-working)
- [Performance Issues](#performance-issues)
- [Git Integration Issues](#git-integration-issues)
- [Database Tool Issues](#database-tool-issues)
- [Frontend Asset Loading Issues](#frontend-asset-loading-issues)
---
## Docker DNS/WiFi Issues (Digital Nomads)
### Problem
When using Docker on Linux (Ubuntu/Xubuntu), the `docker0` bridge network interface can interfere with DNS resolution, causing:
- Hotel/public WiFi captive portals not loading
- DNS lookups failing or timing out
- Internet connectivity issues when Docker is running
- WiFi works fine until Docker starts
This happens because Docker's bridge network (`docker0`) can take priority over your WiFi connection for DNS resolution.
### Solution: Lower docker0 Priority (Recommended)
This solution lets Docker work normally while ensuring your WiFi DNS takes priority. Perfect for digital nomads working from hotels, cafes, and co-working spaces.
**Step 1: Create the NetworkManager dispatcher script**
```bash
sudo tee /etc/NetworkManager/dispatcher.d/99-docker-dns-fix << 'EOF'
#!/bin/bash
# Lower docker0 metric so WiFi takes priority for DNS
# This prevents Docker from interfering with hotel/public WiFi captive portals
if [ "$1" = "docker0" ]; then
ip route del default via 172.17.0.1 dev docker0 2>/dev/null || true
fi
EOF
```
**Step 2: Make it executable**
```bash
sudo chmod +x /etc/NetworkManager/dispatcher.d/99-docker-dns-fix
```
**Step 3: Restart NetworkManager**
```bash
sudo systemctl restart NetworkManager
```
**Step 4: Verify it works**
```bash
# Start Docker
sudo systemctl start docker
# Check routes - docker0 should NOT have a default route
ip route | grep docker0
# Should return nothing or non-default routes only
# Your WiFi should still work
ping -c 3 google.com
```
### Alternative Solutions
#### Option A: Use Google DNS (Simple but less flexible)
If you don't need hotel captive portal DNS:
```bash
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json << 'EOF'
{
"dns": ["8.8.8.8", "8.8.4.4"],
"dns-opts": ["ndots:1"]
}
EOF
sudo systemctl restart docker
```
!!! warning "Captive Portals"
This option uses hardcoded Google DNS, which means hotel WiFi captive portals
(login pages) may not work properly since they rely on DNS hijacking.
#### Option B: Stop Docker when not in use
```bash
# When you need WiFi without Docker interference
sudo systemctl stop docker
sudo ip link set docker0 down
# When you need Docker again
sudo systemctl start docker
```
### Installing Docker on Xubuntu/Ubuntu
If you haven't installed Docker yet:
```bash
# Update package index
sudo apt update
# Install prerequisites
sudo apt install -y ca-certificates curl gnupg
# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
# Add the repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
# Add your user to docker group (run docker without sudo)
sudo usermod -aG docker $USER
# Apply the DNS fix BEFORE using Docker
sudo tee /etc/NetworkManager/dispatcher.d/99-docker-dns-fix << 'EOF'
#!/bin/bash
if [ "$1" = "docker0" ]; then
ip route del default via 172.17.0.1 dev docker0 2>/dev/null || true
fi
EOF
sudo chmod +x /etc/NetworkManager/dispatcher.d/99-docker-dns-fix
# Log out and back in for docker group, then verify
docker run hello-world
```
### Using Docker with this Project
After installing Docker with the DNS fix:
```bash
# Start PostgreSQL for development
make docker-up
# Run tests (auto-starts test database)
make test
# Stop when done
make docker-down
```
---
## Go to Declaration Not Working (Ctrl+B)
### Problem
When you press **Ctrl+B** (or Cmd+B on Mac) on a function, class, or variable, PyCharm shows:
- "Cannot find declaration to go to"
- No navigation occurs
- Imports are marked as unresolved even though code runs fine
**Example:**
```python
from app.services.admin_audit_service import admin_audit_service
# Ctrl+B on get_audit_logs doesn't work
logs = admin_audit_service.get_audit_logs(db, filters)
```
### Solution 1: Configure Source Roots (Most Common Fix)
PyCharm needs to know your project structure:
1. **Mark project root as Sources Root:**
- Right-click on project root folder (`letzshop-product-import`)
- Select **"Mark Directory as"** → **"Sources Root"**
- The folder icon should turn blue
2. **Via Project Structure settings:**
- **File** → **Settings** (Ctrl+Alt+S)
- Navigate to **Project: letzshop-product-import****Project Structure**
- Select your project root directory
- Click the **"Sources"** button at the top
- Click **Apply****OK**
3. **Verify the change:**
- Your project root should now have a blue folder icon
- The `.idea` folder should show the source root is configured
### Solution 2: Verify Python Interpreter
Ensure PyCharm is using the correct virtual environment:
1. **Check current interpreter:**
- **File** → **Settings****Project****Python Interpreter**
- Should show: `/home/samir/PycharmProjects/letzshop-product-import/.venv/bin/python`
2. **If wrong interpreter:**
- Click the gear icon → **Add**
- Select **"Existing environment"**
- Navigate to: `[PROJECT_ROOT]/.venv/bin/python`
- Click **OK**
3. **Wait for indexing:**
- Bottom right corner will show "Indexing..."
- Wait for it to complete before testing
### Solution 3: Invalidate Caches
PyCharm's index might be corrupted:
1. **File****Invalidate Caches...**
2. Select all checkboxes:
- ✅ Invalidate and Restart
- ✅ Clear file system cache and Local History
- ✅ Clear downloaded shared indexes
3. Click **"Invalidate and Restart"**
4. **Wait 2-5 minutes** for re-indexing after restart
### Solution 4: Reimport Project
If nothing else works, rebuild the project structure:
1. **Close PyCharm**
2. **Delete PyCharm's cache folder:**
```bash
rm -rf .idea/
```
3. **Reopen project in PyCharm:**
- **File** → **Open**
- Select project folder
- Let PyCharm recreate `.idea/` folder
4. **Reconfigure:**
- Set Python interpreter
- Mark source roots
- Wait for indexing
### Solution 5: Check .idea/workspace.xml
Sometimes workspace configuration gets corrupted:
```bash
# Close PyCharm first
rm .idea/workspace.xml
# Reopen PyCharm - it will regenerate this file
```
---
## Import Errors and Red Underlines
### Problem
Imports show red underlines and errors like:
- "Unresolved reference"
- "No module named 'app'"
- Code runs fine but PyCharm shows errors
### Solution
1. **Check Sources Root** (see above)
2. **Verify `__init__.py` files exist:**
```bash
# All package directories need __init__.py
ls -la app/__init__.py
ls -la app/services/__init__.py
ls -la app/api/__init__.py
```
3. **Add project root to PYTHONPATH:**
- **File** → **Settings** → **Project** → **Project Structure**
- Right-click project root → **"Mark as Sources Root"**
4. **Enable namespace packages** (if using PEP 420):
- **File** → **Settings** → **Editor** → **Code Style** → **Python**
- Check "Support namespace packages"
---
## Python Interpreter Issues
### Problem
- "No Python interpreter configured"
- Wrong Python version
- Missing packages even though they're installed
### Solution
1. **Verify virtual environment:**
```bash
# In terminal
which python # Should show .venv/bin/python
python --version # Should show Python 3.13.x
```
2. **Configure in PyCharm:**
- **File** → **Settings** → **Project** → **Python Interpreter**
- Click gear icon → **Add**
- Select **"Existing Environment"**
- Choose: `[PROJECT_ROOT]/.venv/bin/python`
3. **Refresh interpreter packages:**
- In Python Interpreter settings
- Click refresh icon (⟳)
- Wait for packages to reload
4. **Reinstall packages if needed:**
```bash
source .venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-dev.txt
```
---
## Debugging Not Working
### Problem
- Breakpoints are grayed out
- Debugger doesn't stop at breakpoints
- "Debugger is not attached"
### Solution 1: Check Run Configuration
1. **Edit configuration:**
- Top right → Edit Configurations
- Should use "Python" configuration type
- Script path should point to your main file
- Interpreter should be your virtual environment
2. **For FastAPI/Uvicorn:**
- Module: `uvicorn`
- Parameters: `main:app --reload`
- Working directory: Project root
### Solution 2: Enable Gevent/Asyncio Support
For async code (FastAPI):
1. **File** → **Settings** → **Build, Execution, Deployment** → **Python Debugger**
2. Check **"Gevent compatible"** or **"Asyncio compatible"**
3. Restart debugger
### Solution 3: Check Breakpoint Settings
Right-click on breakpoint (red dot):
- Ensure "Enabled" is checked
- Check "Suspend" is set to "All"
- Remove any conditions if not needed
---
## Performance Issues
### Problem
- PyCharm is slow or freezing
- High CPU/memory usage
- Indexing takes forever
### Solution 1: Exclude Unnecessary Directories
Exclude directories PyCharm doesn't need to index:
1. **File** → **Settings** → **Project** → **Project Structure**
2. **Mark as "Excluded":**
- `.venv` (if not already)
- `node_modules`
- `.pytest_cache`
- `__pycache__`
- `htmlcov`
- `site` (mkdocs build output)
3. **Right-click these folders in Project view:**
- **Mark Directory as** → **Excluded**
### Solution 2: Increase Memory
1. **Help** → **Change Memory Settings**
2. Increase heap size to **2048 MB** or higher
3. Restart PyCharm
### Solution 3: Disable Unused Plugins
1. **File** → **Settings** → **Plugins**
2. Disable plugins you don't use
3. Keep essential: Python, Git, Database, Markdown
4. Restart PyCharm
### Solution 4: Pause Indexing
If you need to work immediately:
1. **File** → **Pause Indexing**
2. Do your work
3. **Resume Indexing** when convenient
---
## Git Integration Issues
### Problem
- Git operations fail
- "Cannot run git" errors
- Changes not detected
### Solution 1: Configure Git Executable
1. **File** → **Settings** → **Version Control** → **Git**
2. **Path to Git executable:**
- Linux/Mac: `/usr/bin/git`
- Windows: `C:\Program Files\Git\bin\git.exe`
3. Click **"Test"** to verify
4. Click **OK**
### Solution 2: Check Git Root
1. **File** → **Settings** → **Version Control**
2. Verify your project root is listed
3. If missing, click **+** and add it
### Solution 3: Refresh VCS
1. **VCS** → **Refresh File Status**
2. Or: **Git** → **Refresh**
3. Or press: **Ctrl+F5**
---
## Database Tool Issues
### Problem
- Can't connect to PostgreSQL database
- "Connection refused" errors
- Tables not showing
### Solution 1: Verify Database Connection
1. **Database** tool window (usually right side)
2. Click **+** → **Data Source** → **PostgreSQL**
3. **Configure:**
```
Host: localhost
Port: 5432
Database: wizamart_db
User: [from .env file]
Password: [from .env file]
```
4. Click **"Test Connection"**
5. Download drivers if prompted
### Solution 2: Check Database is Running
```bash
# Check if PostgreSQL is running
sudo systemctl status postgresql
# Start if not running
sudo systemctl start postgresql
```
### Solution 3: Verify .env Configuration
```bash
# Check your .env file
cat .env | grep DATABASE
# Should show something like:
DATABASE_URL=postgresql://user:password@localhost:5432/letzshop_db
```
---
## Frontend Asset Loading Issues
### Problem
- Tailwind CSS styles not loading
- Alpine.js not initializing
- Frontend pages look unstyled
- Console errors about missing CSS/JS files
- App doesn't work offline
### Solution 1: Verify CDN Fallback Files Exist
All frontends use CDN with local fallback. Check that local copies exist:
```bash
# From project root
ls -lh static/shared/css/tailwind.min.css
ls -lh static/shared/js/vendor/alpine.min.js
```
**Expected output:**
```
-rw-r--r-- 1 user user 2.9M static/shared/css/tailwind.min.css
-rw-r--r-- 1 user user 44K static/shared/js/vendor/alpine.min.js
```
**If files are missing:**
```bash
# Download Tailwind CSS
cd static/shared/css
curl -o tailwind.min.css https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css
# Download Alpine.js
cd ../js/vendor
curl -o alpine.min.js https://cdn.jsdelivr.net/npm/alpinejs@3.13.3/dist/cdn.min.js
```
### Solution 2: Check Browser Console
Open DevTools (F12) and check the Console tab for errors:
**Common errors:**
- `GET http://localhost:8000/static/shared/css/tailwind.min.css [404 Not Found]` → File missing
- `Alpine.js CDN failed, loading local copy...` → Normal fallback behavior
- `Uncaught ReferenceError: Alpine is not defined` → Alpine.js failed to load
### Solution 3: Test CDN Fallback Behavior
To verify fallback works correctly:
1. **Start application:**
```bash
uvicorn app.main:app --reload
```
2. **Test with CDN blocked:**
- Open browser DevTools (F12)
- Go to Network tab
- Add request blocking rule for `cdn.jsdelivr.net`
- Reload page
3. **Verify fallback:**
- Check Console for: `⚠️ Alpine.js CDN failed, loading local copy...`
- Check Network tab shows local files loaded
- Page should still work normally
### Solution 4: Verify Static Files Mounting
Check that FastAPI is serving static files correctly:
**In `app/main.py`:**
```python
from fastapi.staticfiles import StaticFiles
app.mount("/static", StaticFiles(directory="static"), name="static")
```
**Test static file serving:**
```bash
# With app running, test directly:
curl http://localhost:8000/static/shared/css/tailwind.min.css | head -n 5
curl http://localhost:8000/static/shared/js/vendor/alpine.min.js | head -n 5
```
### Solution 5: File Permissions
If files exist but return 403 Forbidden:
```bash
# Fix permissions
chmod 644 static/shared/css/tailwind.min.css
chmod 644 static/shared/js/vendor/alpine.min.js
chmod 755 static/shared/css
chmod 755 static/shared/js/vendor
```
### Solution 6: For Offline Development
If working completely offline, the CDN will always fail. Verify:
1. **Console shows fallback message** (expected behavior)
2. **Local files load successfully**
3. **Page functions normally**
**To suppress CDN warnings when offline:**
- Accept that CDN will fail (expected)
- Fallback mechanism handles it automatically
- Or modify templates to use local-only (see [CDN Fallback Strategy](../frontend/cdn-fallback-strategy.md#optimization-strategies))
### Solution 7: Network Issues
If both CDN and local files fail intermittently:
1. **Check browser cache:**
- Open DevTools → Network tab
- Check "Disable cache" option
- Hard reload: Ctrl+Shift+R
2. **Clear browser cache completely:**
- Browser Settings → Privacy → Clear browsing data
- Select "Cached images and files"
- Clear data
3. **Check for firewall/proxy issues:**
```bash
# Test CDN accessibility
curl https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css | head -n 5
```
### Related Documentation
See [CDN Fallback Strategy](../frontend/cdn-fallback-strategy.md) for complete details on:
- How the fallback mechanism works
- Updating local asset files
- Production deployment considerations
- Air-gapped deployment setup
---
## Quick Checklist for New Developers
When setting up PyCharm for the first time:
- [ ] Clone repository
- [ ] Create virtual environment: `python -m venv .venv`
- [ ] Activate: `source .venv/bin/activate`
- [ ] Install dependencies: `pip install -r requirements.txt`
- [ ] Open project in PyCharm
- [ ] Configure Python interpreter (`.venv/bin/python`)
- [ ] Mark project root as Sources Root
- [ ] Exclude `.venv`, `.pytest_cache`, `__pycache__` directories
- [ ] Configure Git executable
- [ ] Configure database connection
- [ ] Wait for indexing to complete
- [ ] Test: Ctrl+B on an import should work
---
## Still Having Issues?
### Check PyCharm Logs
1. **Help** → **Show Log in Explorer/Finder**
2. Look for errors in `idea.log`
3. Search for "ERROR" or "Exception"
### Enable Debug Logging
1. **Help** → **Diagnostic Tools** → **Debug Log Settings**
2. Add: `#com.intellij.python`
3. Reproduce the issue
4. Check logs
### Report to Team
If the issue persists:
1. Screenshot the error
2. Share PyCharm version: **Help** → **About**
3. Share Python version: `python --version`
4. Share OS: `uname -a` (Linux/Mac) or `ver` (Windows)
5. Post in team chat with:
- What you were trying to do
- What error you got
- What you've tried so far
---
## Related Documentation
- [PyCharm Make Configuration](pycharm-configuration-make.md) - Configure Make with virtual environment
- [Contributing Guide](contributing.md) - Development workflow
- [Database Migrations](migration/database-migrations.md) - Working with Alembic