# Module Migration Plan

**Status:** In Progress
**Started:** 2026-01-25
**Last Updated:** 2026-01-28
**Target:** v1.0.0

This is the unified migration plan for transforming Orion into a fully modular architecture.

---

## Executive Summary

Transform the platform from a monolithic structure to self-contained modules where each module owns its:
- Services (business logic)
- Models (database entities)
- Schemas (Pydantic DTOs)
- Tasks (Celery background jobs)
- Templates (UI if applicable)
- Migrations (database changes)

---

## Current State Assessment

### Module Migration Status

| Module | Classification | Current State | Services | Models | Tasks | Target |
|--------|---------------|---------------|----------|--------|-------|--------|
| `cms` | Core | ✅ **Complete** | ✅ | ✅ | - | Done |
| `payments` | Optional | 🟡 Partial | ✅ | ✅ | - | Done |
| `billing` | Optional | ✅ **Complete** | ✅ | ✅ | ✅ | Done |
| `marketplace` | Optional | ✅ **Complete** | ✅ | ✅ | ✅ | Done |
| `orders` | Optional | ✅ **Complete** | ✅ | ✅ | - | Done |
| `inventory` | Optional | ✅ **Complete** | ✅ | ✅ | - | Done |
| `customers` | Core | ✅ **Complete** | ✅ | ✅ | - | Done |
| `analytics` | Optional | ✅ **Complete** | ✅ | - | - | Done |
| `messaging` | Optional | ✅ **Complete** | ✅ | ✅ | - | Done |
| `monitoring` | Internal | ✅ **Complete** | ✅ | ✅ | - | Done |
| `dev-tools` | Internal | ✅ **Complete** | ✅ | ✅ | ✅ | Done |
| `tenancy` | Core | 🔴 Shell | ❌ | ❌ | - | Full |
| `core` | Core | 🔴 Shell | ❌ | ❌ | - | Minimal |

### Current Code Locations (To Be Migrated)

```
app/services/                    # → Move to respective modules
├── subscription_service.py      # → billing/services/
├── stripe_service.py            # → billing/services/ (or payments)
├── customer_service.py          # → customers/services/
├── order_service.py             # → orders/services/
├── inventory_service.py         # → inventory/services/
├── letzshop_service.py          # → marketplace/services/
├── marketplace_import_service.py # → marketplace/services/
├── messaging_service.py         # → messaging/services/
└── ...

models/database/                 # → Move to respective modules
├── subscription.py              # → billing/models/
├── customer.py                  # → customers/models/
├── order.py                     # → orders/models/
├── inventory.py                 # → inventory/models/
├── letzshop_*.py               # → marketplace/models/
└── ...

app/tasks/celery_tasks/          # → Move to respective modules
├── subscription.py              # → billing/tasks/
├── marketplace.py               # → marketplace/tasks/
├── letzshop.py                  # → marketplace/tasks/
├── export.py                    # → marketplace/tasks/
├── code_quality.py              # → dev_tools/tasks/
└── test_runner.py               # → dev_tools/tasks/
```

### Celery Task Mapping

| Current Task | Current Location | Target Module |
|--------------|------------------|---------------|
| `reset_period_counters` | subscription.py | `billing` |
| `check_trial_expirations` | subscription.py | `billing` |
| `sync_stripe_status` | subscription.py | `billing` |
| `cleanup_stale_subscriptions` | subscription.py | `billing` |
| `capture_capacity_snapshot` | subscription.py | `monitoring` |
| `process_marketplace_import` | marketplace.py | `marketplace` |
| `process_historical_import` | letzshop.py | `marketplace` |
| `sync_store_directory` | letzshop.py | `marketplace` |
| `export_store_products_to_folder` | export.py | `marketplace` |
| `export_marketplace_products` | export.py | `marketplace` |
| `execute_code_quality_scan` | code_quality.py | `dev-tools` |
| `execute_test_run` | test_runner.py | `dev-tools` |

---

## Infrastructure Work (Completed)

### ✅ Phase 1: Module Classification
- Three-tier system: Core, Optional, Internal
- Renamed `platform-admin` → `tenancy`
- Module registry with CORE_MODULES, OPTIONAL_MODULES, INTERNAL_MODULES

### ✅ Phase 2: Module Infrastructure
- `ModuleDefinition` with lifecycle hooks
- Module events system (`events.py`)
- Module migration discovery (`migrations.py`)
- Observability framework (`observability.py`)

### ✅ Phase 3: Documentation
- Module system architecture docs
- Menu management docs
- Observability docs
- Creating modules developer guide

### ✅ Phase 4: Celery Task Infrastructure
- Added `ScheduledTask` dataclass to `ModuleDefinition`
- Added `tasks_path` and `scheduled_tasks` fields
- Created `app/modules/task_base.py` with `ModuleTask` base class
- Created `app/modules/tasks.py` with discovery utilities
- Updated `celery_config.py` to use module discovery

### ✅ Phase 5: Billing Module Migration
- Created `app/modules/billing/services/` with subscription, stripe, admin services
- Created `app/modules/billing/models/` re-exporting from central location
- Created `app/modules/billing/schemas/` re-exporting from central location
- Created `app/modules/billing/tasks/` with 4 scheduled tasks
- Created `app/modules/billing/exceptions.py`
- Updated `definition.py` with self-contained configuration
- Created backward-compatible re-exports in `app/services/`
- Updated legacy celery_config.py to not duplicate scheduled tasks

### ✅ Phase 6: Marketplace Module Migration
- Created `app/modules/marketplace/services/` re-exporting from existing locations
- Created `app/modules/marketplace/models/` re-exporting marketplace & letzshop models
- Created `app/modules/marketplace/schemas/` re-exporting marketplace schemas
- Created `app/modules/marketplace/tasks/` with:
  - `import_tasks.py` - process_marketplace_import, process_historical_import
  - `sync_tasks.py` - sync_store_directory (scheduled daily)
  - `export_tasks.py` - export_store_products_to_folder, export_marketplace_products
- Created `app/modules/marketplace/exceptions.py`
- Updated `definition.py` with self-contained configuration
- Updated legacy task files to re-export from new location
- Removed marketplace/letzshop/export from LEGACY_TASK_MODULES

### ✅ Phase 7: Dev-Tools Module Migration
- Created `app/modules/dev_tools/models/` with:
  - `architecture_scan.py` - ArchitectureScan, ArchitectureViolation, ArchitectureRule, etc.
  - `test_run.py` - TestRun, TestResult, TestCollection
- Created `app/modules/dev_tools/services/` re-exporting code_quality_service, test_runner_service
- Created `app/modules/dev_tools/schemas/` (placeholder for future schemas)
- Created `app/modules/dev_tools/tasks/` with:
  - `code_quality.py` - execute_code_quality_scan
  - `test_runner.py` - execute_test_run
- Created `app/modules/dev_tools/exceptions.py` with test runner exceptions
- Created `app/modules/dev_tools/routes/api/` with admin router
- Updated `definition.py` with self-contained configuration
- Updated legacy model files to re-export from module location
- Updated legacy task files to re-export from module location
- Used lazy imports in `__init__.py` to avoid circular import issues

### ✅ Phase 8: Monitoring Module Migration
- Created `app/modules/monitoring/models/` re-exporting CapacitySnapshot, AdminNotification
- Created `app/modules/monitoring/services/` re-exporting BackgroundTasksService
- Created `app/modules/monitoring/schemas/` (placeholder)
- Created `app/modules/monitoring/exceptions.py` with monitoring-specific exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues

### ✅ Phase 9: Customers Module Migration
- Created `app/modules/customers/models/` re-exporting Customer, CustomerAddress, PasswordResetToken
- Created `app/modules/customers/services/` re-exporting customer services
- Created `app/modules/customers/schemas/` re-exporting customer schemas
- Created `app/modules/customers/exceptions.py` re-exporting customer exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues

### ✅ Phase 10: Orders Module Migration
- Created `app/modules/orders/models/` re-exporting Order, OrderItem, Invoice, etc.
- Created `app/modules/orders/services/` re-exporting order and invoice services
- Created `app/modules/orders/schemas/` re-exporting order and invoice schemas
- Created `app/modules/orders/exceptions.py` re-exporting order exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues

### ✅ Phase 11: Inventory Module Migration
- Created `app/modules/inventory/models/` re-exporting Inventory, InventoryTransaction
- Created `app/modules/inventory/services/` re-exporting inventory services
- Created `app/modules/inventory/schemas/` re-exporting inventory schemas
- Created `app/modules/inventory/exceptions.py` re-exporting inventory exceptions
- Updated `definition.py` with self-contained configuration
- Used lazy imports in `__init__.py` to avoid circular import issues

### ✅ Phase 12: Analytics & Messaging Module Migration
- **Analytics Module:**
  - Created `app/modules/analytics/models/` (empty - uses data from other modules)
  - Created `app/modules/analytics/services/` re-exporting StatsService, UsageService
  - Created `app/modules/analytics/schemas/` re-exporting stats schemas
  - Created `app/modules/analytics/exceptions.py` with reporting exceptions
  - Updated `definition.py` with self-contained configuration

- **Messaging Module:**
  - Created `app/modules/messaging/models/` re-exporting Conversation, Message, etc.
  - Created `app/modules/messaging/services/` re-exporting messaging services
  - Created `app/modules/messaging/schemas/` re-exporting messaging schemas
  - Created `app/modules/messaging/exceptions.py` re-exporting message exceptions
  - Updated `definition.py` with self-contained configuration

---

## Module Migration Phases

Each module migration includes: services, models, schemas, tasks, templates (if any).

### Phase 5: Billing Module (High Priority)

**Why first:** Has most Celery tasks, critical business logic.

#### Components to Move

| From | To | Type |
|------|-----|------|
| `app/services/subscription_service.py` | `app/modules/billing/services/` | Service |
| `app/services/stripe_service.py` | `app/modules/billing/services/` | Service |
| `models/database/subscription.py` | `app/modules/billing/models/` | Model |
| `models/database/subscription_tier.py` | `app/modules/billing/models/` | Model |
| `models/schema/subscription.py` | `app/modules/billing/schemas/` | Schema |
| `app/tasks/celery_tasks/subscription.py` | `app/modules/billing/tasks/` | Tasks |

#### Target Structure

```
app/modules/billing/
├── __init__.py
├── definition.py          # Update with tasks_path, scheduled_tasks
├── config.py              # BillingConfig schema
├── exceptions.py          # BillingException, SubscriptionError, etc.
├── services/
│   ├── __init__.py
│   ├── subscription_service.py
│   └── stripe_service.py
├── models/
│   ├── __init__.py
│   ├── subscription.py
│   └── subscription_tier.py
├── schemas/
│   ├── __init__.py
│   └── subscription.py
├── tasks/
│   ├── __init__.py
│   └── subscription.py    # 4 scheduled tasks
├── routes/
│   ├── admin.py
│   └── store.py
└── migrations/
    └── versions/
```

#### Tasks

| Task | Schedule | Queue |
|------|----------|-------|
| `reset_period_counters` | Daily 00:05 | scheduled |
| `check_trial_expirations` | Daily 01:00 | scheduled |
| `sync_stripe_status` | Hourly :30 | scheduled |
| `cleanup_stale_subscriptions` | Weekly Sun 03:00 | scheduled |

#### Migration Checklist

- [ ] Create `billing/services/` directory
- [ ] Move `subscription_service.py` → update imports
- [ ] Move `stripe_service.py` → update imports
- [ ] Create `billing/models/` directory
- [ ] Move subscription models → update imports
- [ ] Create `billing/schemas/` directory
- [ ] Move subscription schemas → update imports
- [ ] Create `billing/tasks/` directory
- [ ] Move subscription tasks → update task paths
- [ ] Create `billing/exceptions.py`
- [ ] Update `definition.py` with `tasks_path`, `scheduled_tasks`
- [ ] Create module migrations
- [ ] Update all imports across codebase
- [ ] Test subscription flows
- [ ] Test scheduled tasks

---

### Phase 6: Marketplace Module (High Priority)

**Why second:** Has most tasks, complex import logic.

#### Components to Move

| From | To | Type |
|------|-----|------|
| `app/services/letzshop_service.py` | `app/modules/marketplace/services/` | Service |
| `app/services/marketplace_import_service.py` | `app/modules/marketplace/services/` | Service |
| `app/services/letzshop_credentials_service.py` | `app/modules/marketplace/services/` | Service |
| `models/database/letzshop_*.py` | `app/modules/marketplace/models/` | Models |
| `models/database/marketplace_import_job.py` | `app/modules/marketplace/models/` | Model |
| `app/tasks/celery_tasks/marketplace.py` | `app/modules/marketplace/tasks/` | Tasks |
| `app/tasks/celery_tasks/letzshop.py` | `app/modules/marketplace/tasks/` | Tasks |
| `app/tasks/celery_tasks/export.py` | `app/modules/marketplace/tasks/` | Tasks |

#### Target Structure

```
app/modules/marketplace/
├── __init__.py
├── definition.py
├── exceptions.py
├── services/
│   ├── __init__.py
│   ├── letzshop_service.py
│   ├── letzshop_credentials_service.py
│   ├── import_service.py
│   └── export_service.py
├── models/
│   ├── __init__.py
│   ├── letzshop_order.py
│   ├── letzshop_credentials.py
│   └── import_job.py
├── schemas/
│   └── ...
├── tasks/
│   ├── __init__.py
│   ├── import_tasks.py    # process_marketplace_import
│   ├── letzshop.py        # process_historical_import, sync_store_directory
│   └── export.py          # export tasks
└── routes/
```

#### Tasks

| Task | Type | Queue |
|------|------|-------|
| `process_marketplace_import` | On-demand | long_running |
| `process_historical_import` | On-demand | long_running |
| `sync_store_directory` | Scheduled (optional) | long_running |
| `export_store_products_to_folder` | On-demand | default |
| `export_marketplace_products` | On-demand | default |

---

### Phase 7: Dev-Tools Module (Medium Priority)

**Why:** Internal module with code quality and test runner tasks.

#### Components to Move

| From | To | Type |
|------|-----|------|
| `app/services/code_quality_service.py` | `app/modules/dev_tools/services/` | Service |
| `app/services/test_runner_service.py` | `app/modules/dev_tools/services/` | Service |
| `models/database/architecture_scan.py` | `app/modules/dev_tools/models/` | Model |
| `models/database/test_run.py` | `app/modules/dev_tools/models/` | Model |
| `app/tasks/celery_tasks/code_quality.py` | `app/modules/dev_tools/tasks/` | Tasks |
| `app/tasks/celery_tasks/test_runner.py` | `app/modules/dev_tools/tasks/` | Tasks |

#### Target Structure

```
app/modules/dev_tools/
├── __init__.py
├── definition.py
├── services/
│   ├── code_quality_service.py
│   └── test_runner_service.py
├── models/
│   ├── architecture_scan.py
│   ├── architecture_violation.py
│   └── test_run.py
├── tasks/
│   ├── code_quality.py
│   └── test_runner.py
└── routes/
```

---

### Phase 8: Monitoring Module (Medium Priority)

#### Components to Move

| From | To | Type |
|------|-----|------|
| `app/services/background_tasks_service.py` | `app/modules/monitoring/services/` | Service |
| `capture_capacity_snapshot` task | `app/modules/monitoring/tasks/` | Task |
| `models/database/capacity_snapshot.py` | `app/modules/monitoring/models/` | Model |

#### Target Structure

```
app/modules/monitoring/
├── __init__.py
├── definition.py
├── services/
│   └── background_tasks_service.py
├── models/
│   └── capacity_snapshot.py
├── tasks/
│   └── capacity.py    # capture_capacity_snapshot
└── routes/
```

---

### Phase 9: Customers Module (Core)

#### Components to Move

| From | To |
|------|-----|
| `app/services/customer_service.py` | `app/modules/customers/services/` |
| `models/database/customer.py` | `app/modules/customers/models/` |
| Customer-related schemas | `app/modules/customers/schemas/` |

---

### Phase 10: Orders Module

#### Components to Move

| From | To |
|------|-----|
| `app/services/order_service.py` | `app/modules/orders/services/` |
| `models/database/order.py` | `app/modules/orders/models/` |
| Order-related schemas | `app/modules/orders/schemas/` |

---

### Phase 11: Inventory Module

#### Components to Move

| From | To |
|------|-----|
| `app/services/inventory_service.py` | `app/modules/inventory/services/` |
| `models/database/inventory*.py` | `app/modules/inventory/models/` |

---

### Phase 12: Remaining Modules

- **Analytics** - Move analytics services
- **Messaging** - Move messaging service, notification models
- **Tenancy** - Move platform, merchant, store services

---

## Phase 13: Cleanup & Full Celery Migration

After all modules are migrated:

### 13.1 Remove Fallback Tasks

Delete:
```
app/tasks/background_tasks.py
app/tasks/letzshop_tasks.py
app/tasks/subscription_tasks.py
app/tasks/code_quality_tasks.py
app/tasks/test_runner_tasks.py
```

### 13.2 Remove USE_CELERY Flag

```python
# app/core/config.py
# DELETE: USE_CELERY: bool = False
```

### 13.3 Simplify Dispatcher

Rewrite `app/tasks/dispatcher.py` to Celery-only.

### 13.4 Delete Old Task Directory

```
app/tasks/celery_tasks/  # DELETE - moved to modules
```

### 13.5 Clean Up Old Services/Models

Remove files from:
- `app/services/` (moved to modules)
- `models/database/` (moved to modules)

Leave re-exports for backward compatibility if needed.

---

## Queue Configuration

### Final Queue Structure

| Queue | Purpose | Modules |
|-------|---------|---------|
| `default` | Fast tasks | marketplace (exports) |
| `long_running` | CPU-intensive | marketplace (imports), dev-tools |
| `scheduled` | Celery Beat | billing, monitoring |

### Task Routing

```python
task_routes = {
    "app.modules.billing.tasks.*": {"queue": "scheduled"},
    "app.modules.marketplace.tasks.import_tasks.*": {"queue": "long_running"},
    "app.modules.marketplace.tasks.letzshop.*": {"queue": "long_running"},
    "app.modules.marketplace.tasks.export.*": {"queue": "default"},
    "app.modules.monitoring.tasks.*": {"queue": "scheduled"},
    "app.modules.dev_tools.tasks.*": {"queue": "long_running"},
}
```

---

## Migration Strategy

### For Each Module

1. **Create directory structure**
2. **Move models first** (fewest dependencies)
3. **Move services** (depend on models)
4. **Move schemas** (depend on models)
5. **Move tasks** (depend on services)
6. **Update all imports** across codebase
7. **Create re-exports** in old locations (temporary)
8. **Test thoroughly**
9. **Remove re-exports** in next phase

### Import Update Strategy

Use IDE refactoring or script:
```bash
# Example: Update subscription_service imports
find . -name "*.py" -exec sed -i 's/from app.services.subscription_service/from app.modules.billing.services.subscription_service/g' {} \;
```

---

## Verification Checklist

### Per-Module Verification

- [ ] Module directory has all components
- [ ] All imports updated
- [ ] Services work
- [ ] API routes work
- [ ] Tasks execute correctly
- [ ] Scheduled tasks run (Celery Beat)
- [ ] No circular imports
- [ ] Tests pass

### Final Verification

- [ ] All 12 tasks in modules
- [ ] `USE_CELERY` flag removed
- [ ] Fallback tasks deleted
- [ ] Old service files removed
- [ ] Old model files removed
- [ ] All tests pass
- [ ] Production smoke test

---

## Priority Order

| Priority | Phase | Module | Reason |
|----------|-------|--------|--------|
| 1 | 4 | Infrastructure | Required for task migration |
| 2 | 5 | `billing` | Most tasks, critical business logic |
| 3 | 6 | `marketplace` | Most complex, many tasks |
| 4 | 7 | `dev-tools` | Internal, low risk |
| 5 | 8 | `monitoring` | Internal, low risk |
| 6 | 9-12 | Others | Lower complexity |
| 7 | 13 | Cleanup | Final step |

---

## Estimated Effort

| Phase | Sessions | Notes |
|-------|----------|-------|
| 4 (Infrastructure) | 1 | Task base class, discovery |
| 5 (Billing) | 2-3 | Complex, many components |
| 6 (Marketplace) | 3-4 | Most complex module |
| 7-8 (Internal) | 1-2 | Simpler modules |
| 9-12 (Others) | 4-6 | Medium complexity each |
| 13 (Cleanup) | 1 | Removing old code |

**Total:** ~12-17 sessions

---

## Rollback Plan

Each phase can be rolled back independently:
1. Restore moved files to original locations
2. Revert import changes
3. Restore celery_config.py

No database schema changes during code migration = no DB rollback needed.

---

## Related Documents

- [Module System Architecture](../architecture/module-system.md)
- [Creating Modules](../development/creating-modules.md)
- [Observability](../architecture/observability.md)
- [SESSION_NOTE_2026-01-27](SESSION_NOTE_2026-01-27_module-reclassification.md)