sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
orion/docs/archive/module-migration-plan.md
Samir Boulahtit e9253fbd84 refactor: rename Wizamart to Orion across entire codebase 
Replace all ~1,086 occurrences of Wizamart/wizamart/WIZAMART/WizaMart
with Orion/orion/ORION across 184 files. This includes database
identifiers, email addresses, domain references, R2 bucket names,
DNS prefixes, encryption salt, Celery app name, config defaults,
Docker configs, CI configs, documentation, seed data, and templates.

Renames homepage-wizamart.html template to homepage-orion.html.
Fixes duplicate file_pattern key in api.yaml architecture rule.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 16:46:56 +01:00

610 lines
21 KiB
Markdown
Raw Permalink Blame History

# Module Migration Plan
**Status:** In Progress
**Started:** 2026-01-25
**Last Updated:** 2026-01-28
**Target:** v1.0.0
This is the unified migration plan for transforming Orion into a fully modular architecture.
---
## Executive Summary
Transform the platform from a monolithic structure to self-contained modules where each module owns its:
- Services (business logic)
- Models (database entities)
- Schemas (Pydantic DTOs)
- Tasks (Celery background jobs)
- Templates (UI if applicable)
- Migrations (database changes)
---
## Current State Assessment
### Module Migration Status
| Module | Classification | Current State | Services | Models | Tasks | Target |
|--------|---------------|---------------|----------|--------|-------|--------|
| `cms` | Core | ✅ **Complete** | ✅ | ✅ | - | Done |
| `payments` | Optional | 🟡 Partial | ✅ | ✅ | - | Done |
| `billing` | Optional | ✅ **Complete** | ✅ | ✅ | ✅ | Done |
| `marketplace` | Optional | ✅ **Complete** | ✅ | ✅ | ✅ | Done |
| `orders` | Optional | ✅ **Complete** | ✅ | ✅ | - | Done |
| `inventory` | Optional | ✅ **Complete** | ✅ | ✅ | - | Done |
| `customers` | Core | ✅ **Complete** | ✅ | ✅ | - | Done |
| `analytics` | Optional | ✅ **Complete** | ✅ | - | - | Done |
| `messaging` | Optional | ✅ **Complete** | ✅ | ✅ | - | Done |
| `monitoring` | Internal | ✅ **Complete** | ✅ | ✅ | - | Done |
| `dev-tools` | Internal | ✅ **Complete** | ✅ | ✅ | ✅ | Done |
| `tenancy` | Core | 🔴 Shell | ❌ | ❌ | - | Full |
| `core` | Core | 🔴 Shell | ❌ | ❌ | - | Minimal |
### Current Code Locations (To Be Migrated)
```
app/services/ # → Move to respective modules
├── subscription_service.py # → billing/services/
├── stripe_service.py # → billing/services/ (or payments)
├── customer_service.py # → customers/services/
├── order_service.py # → orders/services/
├── inventory_service.py # → inventory/services/
├── letzshop_service.py # → marketplace/services/
├── marketplace_import_service.py # → marketplace/services/
├── messaging_service.py # → messaging/services/
└── ...
models/database/ # → Move to respective modules
├── subscription.py # → billing/models/
├── customer.py # → customers/models/
├── order.py # → orders/models/
├── inventory.py # → inventory/models/
├── letzshop_*.py # → marketplace/models/
└── ...
app/tasks/celery_tasks/ # → Move to respective modules
├── subscription.py # → billing/tasks/
├── marketplace.py # → marketplace/tasks/
├── letzshop.py # → marketplace/tasks/
├── export.py # → marketplace/tasks/
├── code_quality.py # → dev_tools/tasks/
└── test_runner.py # → dev_tools/tasks/
```
### Celery Task Mapping
| Current Task | Current Location | Target Module |
|--------------|------------------|---------------|
| `reset_period_counters` | subscription.py | `billing` |
| `check_trial_expirations` | subscription.py | `billing` |
| `sync_stripe_status` | subscription.py | `billing` |
| `cleanup_stale_subscriptions` | subscription.py | `billing` |
| `capture_capacity_snapshot` | subscription.py | `monitoring` |
| `process_marketplace_import` | marketplace.py | `marketplace` |
| `process_historical_import` | letzshop.py | `marketplace` |
| `sync_store_directory` | letzshop.py | `marketplace` |
| `export_store_products_to_folder` | export.py | `marketplace` |
| `export_marketplace_products` | export.py | `marketplace` |
| `execute_code_quality_scan` | code_quality.py | `dev-tools` |
| `execute_test_run` | test_runner.py | `dev-tools` |
---
## Infrastructure Work (Completed)
### ✅ Phase 1: Module Classification
- Three-tier system: Core, Optional, Internal
- Renamed `platform-admin` → `tenancy`
- Module registry with CORE_MODULES, OPTIONAL_MODULES, INTERNAL_MODULES
### ✅ Phase 2: Module Infrastructure
- `ModuleDefinition` with lifecycle hooks
- Module events system (`events.py`)
- Module migration discovery (`migrations.py`)
- Observability framework (`observability.py`)
### ✅ Phase 3: Documentation
- Module system architecture docs
- Menu management docs
- Observability docs
- Creating modules developer guide
### ✅ Phase 4: Celery Task Infrastructure
- Added `ScheduledTask` dataclass to `ModuleDefinition`
- Added `tasks_path` and `scheduled_tasks` fields
- Created `app/modules/task_base.py` with `ModuleTask` base class
- Created `app/modules/tasks.py` with discovery utilities
- Updated `celery_config.py` to use module discovery
### ✅ Phase 5: Billing Module Migration
- Created `app/modules/billing/services/` with subscription, stripe, admin services
- Created `app/modules/billing/models/` re-exporting from central location
- Created `app/modules/billing/schemas/` re-exporting from central location
- Created `app/modules/billing/tasks/` with 4 scheduled tasks
- Created `app/modules/billing/exceptions.py`
- Updated `definition.py` with self-contained configuration
- Created backward-compatible re-exports in `app/services/`
- Updated legacy celery_config.py to not duplicate scheduled tasks
### ✅ Phase 6: Marketplace Module Migration
- Created `app/modules/marketplace/services/` re-exporting from existing locations
- Created `app/modules/marketplace/models/` re-exporting marketplace & letzshop models
- Created `app/modules/marketplace/schemas/` re-exporting marketplace schemas
- Created `app/modules/marketplace/tasks/` with:
- `import_tasks.py` - process_marketplace_import, process_historical_import
- `sync_tasks.py` - sync_store_directory (scheduled daily)
- `export_tasks.py` - export_store_products_to_folder, export_marketplace_products
- Created `app/modules/marketplace/exceptions.py`
- Updated `definition.py` with self-contained configuration
- Updated legacy task files to re-export from new location
- Removed marketplace/letzshop/export from LEGACY_TASK_MODULES
### ✅ Phase 7: Dev-Tools Module Migration
- Created `app/modules/dev_tools/models/` with:
- `architecture_scan.py` - ArchitectureScan, ArchitectureViolation, ArchitectureRule, etc.
- `test_run.py` - TestRun, TestResult, TestCollection
- Created `app/modules/dev_tools/services/` re-exporting code_quality_service, test_runner_service
- Created `app/modules/dev_tools/schemas/` (placeholder for future schemas)
- Created `app/modules/dev_tools/tasks/` with:
- `code_quality.py` - execute_code_quality_scan
- `test_runner.py` - execute_test_run
- Created `app/modules/dev_tools/exceptions.py` with test runner exceptions
- Created `app/modules/dev_tools/routes/api/` with admin router
- Updated `definition.py` with self-contained configuration
- Updated legacy model files to re-export from module location
- Updated legacy task files to re-export from module location
- Used lazy imports in `__init__.py` to avoid circular import issues
### ✅ Phase 8: Monitoring Module Migration
- Created `app/modules/monitoring/models/` re-exporting CapacitySnapshot, AdminNotification
- Created `app/modules/monitoring/services/` re-exporting BackgroundTasksService
- Created `app/modules/monitoring/schemas/` (placeholder)
- Created `app/modules/monitoring/exceptions.py` with monitoring-specific exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues
### ✅ Phase 9: Customers Module Migration
- Created `app/modules/customers/models/` re-exporting Customer, CustomerAddress, PasswordResetToken
- Created `app/modules/customers/services/` re-exporting customer services
- Created `app/modules/customers/schemas/` re-exporting customer schemas
- Created `app/modules/customers/exceptions.py` re-exporting customer exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues
### ✅ Phase 10: Orders Module Migration
- Created `app/modules/orders/models/` re-exporting Order, OrderItem, Invoice, etc.
- Created `app/modules/orders/services/` re-exporting order and invoice services
- Created `app/modules/orders/schemas/` re-exporting order and invoice schemas
- Created `app/modules/orders/exceptions.py` re-exporting order exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues
### ✅ Phase 11: Inventory Module Migration
- Created `app/modules/inventory/models/` re-exporting Inventory, InventoryTransaction
- Created `app/modules/inventory/services/` re-exporting inventory services
- Created `app/modules/inventory/schemas/` re-exporting inventory schemas
- Created `app/modules/inventory/exceptions.py` re-exporting inventory exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues
### ✅ Phase 12: Analytics & Messaging Module Migration
- **Analytics Module:**
- Created `app/modules/analytics/models/` (empty - uses data from other modules)
- Created `app/modules/analytics/services/` re-exporting StatsService, UsageService
- Created `app/modules/analytics/schemas/` re-exporting stats schemas
- Created `app/modules/analytics/exceptions.py` with reporting exceptions
- Updated `definition.py` with self-contained configuration
- **Messaging Module:**
- Created `app/modules/messaging/models/` re-exporting Conversation, Message, etc.
- Created `app/modules/messaging/services/` re-exporting messaging services
- Created `app/modules/messaging/schemas/` re-exporting messaging schemas
- Created `app/modules/messaging/exceptions.py` re-exporting message exceptions
- Updated `definition.py` with self-contained configuration
---
## Module Migration Phases
Each module migration includes: services, models, schemas, tasks, templates (if any).
### Phase 5: Billing Module (High Priority)
**Why first:** Has most Celery tasks, critical business logic.
#### Components to Move
| From | To | Type |
|------|-----|------|
| `app/services/subscription_service.py` | `app/modules/billing/services/` | Service |
| `app/services/stripe_service.py` | `app/modules/billing/services/` | Service |
| `models/database/subscription.py` | `app/modules/billing/models/` | Model |
| `models/database/subscription_tier.py` | `app/modules/billing/models/` | Model |
| `models/schema/subscription.py` | `app/modules/billing/schemas/` | Schema |
| `app/tasks/celery_tasks/subscription.py` | `app/modules/billing/tasks/` | Tasks |
#### Target Structure
```
app/modules/billing/
├── __init__.py
├── definition.py # Update with tasks_path, scheduled_tasks
├── config.py # BillingConfig schema
├── exceptions.py # BillingException, SubscriptionError, etc.
├── services/
│ ├── __init__.py
│ ├── subscription_service.py
│ └── stripe_service.py
├── models/
│ ├── __init__.py
│ ├── subscription.py
│ └── subscription_tier.py
├── schemas/
│ ├── __init__.py
│ └── subscription.py
├── tasks/
│ ├── __init__.py
│ └── subscription.py # 4 scheduled tasks
├── routes/
│ ├── admin.py
│ └── store.py
└── migrations/
└── versions/
```
#### Tasks
| Task | Schedule | Queue |
|------|----------|-------|
| `reset_period_counters` | Daily 00:05 | scheduled |
| `check_trial_expirations` | Daily 01:00 | scheduled |
| `sync_stripe_status` | Hourly :30 | scheduled |
| `cleanup_stale_subscriptions` | Weekly Sun 03:00 | scheduled |
#### Migration Checklist
- [ ] Create `billing/services/` directory
- [ ] Move `subscription_service.py` → update imports
- [ ] Move `stripe_service.py` → update imports
- [ ] Create `billing/models/` directory
- [ ] Move subscription models → update imports
- [ ] Create `billing/schemas/` directory
- [ ] Move subscription schemas → update imports
- [ ] Create `billing/tasks/` directory
- [ ] Move subscription tasks → update task paths
- [ ] Create `billing/exceptions.py`
- [ ] Update `definition.py` with `tasks_path`, `scheduled_tasks`
- [ ] Create module migrations
- [ ] Update all imports across codebase
- [ ] Test subscription flows
- [ ] Test scheduled tasks
---
### Phase 6: Marketplace Module (High Priority)
**Why second:** Has most tasks, complex import logic.
#### Components to Move
| From | To | Type |
|------|-----|------|
| `app/services/letzshop_service.py` | `app/modules/marketplace/services/` | Service |
| `app/services/marketplace_import_service.py` | `app/modules/marketplace/services/` | Service |
| `app/services/letzshop_credentials_service.py` | `app/modules/marketplace/services/` | Service |
| `models/database/letzshop_*.py` | `app/modules/marketplace/models/` | Models |
| `models/database/marketplace_import_job.py` | `app/modules/marketplace/models/` | Model |
| `app/tasks/celery_tasks/marketplace.py` | `app/modules/marketplace/tasks/` | Tasks |
| `app/tasks/celery_tasks/letzshop.py` | `app/modules/marketplace/tasks/` | Tasks |
| `app/tasks/celery_tasks/export.py` | `app/modules/marketplace/tasks/` | Tasks |
#### Target Structure
```
app/modules/marketplace/
├── __init__.py
├── definition.py
├── exceptions.py
├── services/
│ ├── __init__.py
│ ├── letzshop_service.py
│ ├── letzshop_credentials_service.py
│ ├── import_service.py
│ └── export_service.py
├── models/
│ ├── __init__.py
│ ├── letzshop_order.py
│ ├── letzshop_credentials.py
│ └── import_job.py
├── schemas/
│ └── ...
├── tasks/
│ ├── __init__.py
│ ├── import_tasks.py # process_marketplace_import
│ ├── letzshop.py # process_historical_import, sync_store_directory
│ └── export.py # export tasks
└── routes/
```
#### Tasks
| Task | Type | Queue |
|------|------|-------|
| `process_marketplace_import` | On-demand | long_running |
| `process_historical_import` | On-demand | long_running |
| `sync_store_directory` | Scheduled (optional) | long_running |
| `export_store_products_to_folder` | On-demand | default |
| `export_marketplace_products` | On-demand | default |
---
### Phase 7: Dev-Tools Module (Medium Priority)
**Why:** Internal module with code quality and test runner tasks.
#### Components to Move
| From | To | Type |
|------|-----|------|
| `app/services/code_quality_service.py` | `app/modules/dev_tools/services/` | Service |
| `app/services/test_runner_service.py` | `app/modules/dev_tools/services/` | Service |
| `models/database/architecture_scan.py` | `app/modules/dev_tools/models/` | Model |
| `models/database/test_run.py` | `app/modules/dev_tools/models/` | Model |
| `app/tasks/celery_tasks/code_quality.py` | `app/modules/dev_tools/tasks/` | Tasks |
| `app/tasks/celery_tasks/test_runner.py` | `app/modules/dev_tools/tasks/` | Tasks |
#### Target Structure
```
app/modules/dev_tools/
├── __init__.py
├── definition.py
├── services/
│ ├── code_quality_service.py
│ └── test_runner_service.py
├── models/
│ ├── architecture_scan.py
│ ├── architecture_violation.py
│ └── test_run.py
├── tasks/
│ ├── code_quality.py
│ └── test_runner.py
└── routes/
```
---
### Phase 8: Monitoring Module (Medium Priority)
#### Components to Move
| From | To | Type |
|------|-----|------|
| `app/services/background_tasks_service.py` | `app/modules/monitoring/services/` | Service |
| `capture_capacity_snapshot` task | `app/modules/monitoring/tasks/` | Task |
| `models/database/capacity_snapshot.py` | `app/modules/monitoring/models/` | Model |
#### Target Structure
```
app/modules/monitoring/
├── __init__.py
├── definition.py
├── services/
│ └── background_tasks_service.py
├── models/
│ └── capacity_snapshot.py
├── tasks/
│ └── capacity.py # capture_capacity_snapshot
└── routes/
```
---
### Phase 9: Customers Module (Core)
#### Components to Move
| From | To |
|------|-----|
| `app/services/customer_service.py` | `app/modules/customers/services/` |
| `models/database/customer.py` | `app/modules/customers/models/` |
| Customer-related schemas | `app/modules/customers/schemas/` |
---
### Phase 10: Orders Module
#### Components to Move
| From | To |
|------|-----|
| `app/services/order_service.py` | `app/modules/orders/services/` |
| `models/database/order.py` | `app/modules/orders/models/` |
| Order-related schemas | `app/modules/orders/schemas/` |
---
### Phase 11: Inventory Module
#### Components to Move
| From | To |
|------|-----|
| `app/services/inventory_service.py` | `app/modules/inventory/services/` |
| `models/database/inventory*.py` | `app/modules/inventory/models/` |
---
### Phase 12: Remaining Modules
- **Analytics** - Move analytics services
- **Messaging** - Move messaging service, notification models
- **Tenancy** - Move platform, merchant, store services
---
## Phase 13: Cleanup & Full Celery Migration
After all modules are migrated:
### 13.1 Remove Fallback Tasks
Delete:
```
app/tasks/background_tasks.py
app/tasks/letzshop_tasks.py
app/tasks/subscription_tasks.py
app/tasks/code_quality_tasks.py
app/tasks/test_runner_tasks.py
```
### 13.2 Remove USE_CELERY Flag
```python
# app/core/config.py
# DELETE: USE_CELERY: bool = False
```
### 13.3 Simplify Dispatcher
Rewrite `app/tasks/dispatcher.py` to Celery-only.
### 13.4 Delete Old Task Directory
```
app/tasks/celery_tasks/ # DELETE - moved to modules
```
### 13.5 Clean Up Old Services/Models
Remove files from:
- `app/services/` (moved to modules)
- `models/database/` (moved to modules)
Leave re-exports for backward compatibility if needed.
---
## Queue Configuration
### Final Queue Structure
| Queue | Purpose | Modules |
|-------|---------|---------|
| `default` | Fast tasks | marketplace (exports) |
| `long_running` | CPU-intensive | marketplace (imports), dev-tools |
| `scheduled` | Celery Beat | billing, monitoring |
### Task Routing
```python
task_routes = {
"app.modules.billing.tasks.*": {"queue": "scheduled"},
"app.modules.marketplace.tasks.import_tasks.*": {"queue": "long_running"},
"app.modules.marketplace.tasks.letzshop.*": {"queue": "long_running"},
"app.modules.marketplace.tasks.export.*": {"queue": "default"},
"app.modules.monitoring.tasks.*": {"queue": "scheduled"},
"app.modules.dev_tools.tasks.*": {"queue": "long_running"},
}
```
---
## Migration Strategy
### For Each Module
1. **Create directory structure**
2. **Move models first** (fewest dependencies)
3. **Move services** (depend on models)
4. **Move schemas** (depend on models)
5. **Move tasks** (depend on services)
6. **Update all imports** across codebase
7. **Create re-exports** in old locations (temporary)
8. **Test thoroughly**
9. **Remove re-exports** in next phase
### Import Update Strategy
Use IDE refactoring or script:
```bash
# Example: Update subscription_service imports
find . -name "*.py" -exec sed -i 's/from app.services.subscription_service/from app.modules.billing.services.subscription_service/g' {} \;
```
---
## Verification Checklist
### Per-Module Verification
- [ ] Module directory has all components
- [ ] All imports updated
- [ ] Services work
- [ ] API routes work
- [ ] Tasks execute correctly
- [ ] Scheduled tasks run (Celery Beat)
- [ ] No circular imports
- [ ] Tests pass
### Final Verification
- [ ] All 12 tasks in modules
- [ ] `USE_CELERY` flag removed
- [ ] Fallback tasks deleted
- [ ] Old service files removed
- [ ] Old model files removed
- [ ] All tests pass
- [ ] Production smoke test
---
## Priority Order
| Priority | Phase | Module | Reason |
|----------|-------|--------|--------|
| 1 | 4 | Infrastructure | Required for task migration |
| 2 | 5 | `billing` | Most tasks, critical business logic |
| 3 | 6 | `marketplace` | Most complex, many tasks |
| 4 | 7 | `dev-tools` | Internal, low risk |
| 5 | 8 | `monitoring` | Internal, low risk |
| 6 | 9-12 | Others | Lower complexity |
| 7 | 13 | Cleanup | Final step |
---
## Estimated Effort
| Phase | Sessions | Notes |
|-------|----------|-------|
| 4 (Infrastructure) | 1 | Task base class, discovery |
| 5 (Billing) | 2-3 | Complex, many components |
| 6 (Marketplace) | 3-4 | Most complex module |
| 7-8 (Internal) | 1-2 | Simpler modules |
| 9-12 (Others) | 4-6 | Medium complexity each |
| 13 (Cleanup) | 1 | Removing old code |
**Total:** ~12-17 sessions
---
## Rollback Plan
Each phase can be rolled back independently:
1. Restore moved files to original locations
2. Revert import changes
3. Restore celery_config.py
No database schema changes during code migration = no DB rollback needed.
---
## Related Documents
- [Module System Architecture](../architecture/module-system.md)
- [Creating Modules](../development/creating-modules.md)
- [Observability](../architecture/observability.md)
- [SESSION_NOTE_2026-01-27](SESSION_NOTE_2026-01-27_module-reclassification.md)
Reference in New Issue View Git Blame Copy Permalink