orion/docs/archive/module-migration-plan.md

Module Migration Plan

Status: In Progress Started: 2026-01-25 Last Updated: 2026-01-28 Target: v1.0.0

This is the unified migration plan for transforming Orion into a fully modular architecture.

---

Executive Summary

Transform the platform from a monolithic structure to self-contained modules where each module owns its:

• Services (business logic)
• Models (database entities)
• Schemas (Pydantic DTOs)
• Tasks (Celery background jobs)
• Templates (UI if applicable)
• Migrations (database changes)

---

Current State Assessment

Module Migration Status

Module	Classification	Current State	Services	Models	Tasks	Target
cms	Core	✅ Complete	✅	✅	-	Done
payments	Optional	🟡 Partial	✅	✅	-	Done
billing	Optional	✅ Complete	✅	✅	✅	Done
marketplace	Optional	✅ Complete	✅	✅	✅	Done
orders	Optional	✅ Complete	✅	✅	-	Done
inventory	Optional	✅ Complete	✅	✅	-	Done
customers	Core	✅ Complete	✅	✅	-	Done
analytics	Optional	✅ Complete	✅	-	-	Done
messaging	Optional	✅ Complete	✅	✅	-	Done
monitoring	Internal	✅ Complete	✅	✅	-	Done
dev-tools	Internal	✅ Complete	✅	✅	✅	Done
tenancy	Core	🔴 Shell	❌	❌	-	Full
core	Core	🔴 Shell	❌	❌	-	Minimal

Current Code Locations (To Be Migrated)

app/services/                    # → Move to respective modules
├── subscription_service.py      # → billing/services/
├── stripe_service.py            # → billing/services/ (or payments)
├── customer_service.py          # → customers/services/
├── order_service.py             # → orders/services/
├── inventory_service.py         # → inventory/services/
├── letzshop_service.py          # → marketplace/services/
├── marketplace_import_service.py # → marketplace/services/
├── messaging_service.py         # → messaging/services/
└── ...

models/database/                 # → Move to respective modules
├── subscription.py              # → billing/models/
├── customer.py                  # → customers/models/
├── order.py                     # → orders/models/
├── inventory.py                 # → inventory/models/
├── letzshop_*.py               # → marketplace/models/
└── ...

app/tasks/celery_tasks/          # → Move to respective modules
├── subscription.py              # → billing/tasks/
├── marketplace.py               # → marketplace/tasks/
├── letzshop.py                  # → marketplace/tasks/
├── export.py                    # → marketplace/tasks/
├── code_quality.py              # → dev_tools/tasks/
└── test_runner.py               # → dev_tools/tasks/

Celery Task Mapping

Current Task	Current Location	Target Module
reset_period_counters	subscription.py	billing
check_trial_expirations	subscription.py	billing
sync_stripe_status	subscription.py	billing
cleanup_stale_subscriptions	subscription.py	billing
capture_capacity_snapshot	subscription.py	monitoring
process_marketplace_import	marketplace.py	marketplace
process_historical_import	letzshop.py	marketplace
sync_store_directory	letzshop.py	marketplace
export_store_products_to_folder	export.py	marketplace
export_marketplace_products	export.py	marketplace
execute_code_quality_scan	code_quality.py	dev-tools
execute_test_run	test_runner.py	dev-tools

---

Infrastructure Work (Completed)

✅ Phase 1: Module Classification

• Three-tier system: Core, Optional, Internal
• Renamed platform-admin → tenancy
• Module registry with CORE_MODULES, OPTIONAL_MODULES, INTERNAL_MODULES

✅ Phase 2: Module Infrastructure

• ModuleDefinition with lifecycle hooks
• Module events system (events.py)
• Module migration discovery (migrations.py)
• Observability framework (observability.py)

✅ Phase 3: Documentation

• Module system architecture docs
• Menu management docs
• Observability docs
• Creating modules developer guide

✅ Phase 4: Celery Task Infrastructure

• Added ScheduledTask dataclass to ModuleDefinition
• Added tasks_path and scheduled_tasks fields
• Created app/modules/task_base.py with ModuleTask base class
• Created app/modules/tasks.py with discovery utilities
• Updated celery_config.py to use module discovery

✅ Phase 5: Billing Module Migration

• Created app/modules/billing/services/ with subscription, stripe, admin services
• Created app/modules/billing/models/ re-exporting from central location
• Created app/modules/billing/schemas/ re-exporting from central location
• Created app/modules/billing/tasks/ with 4 scheduled tasks
• Created app/modules/billing/exceptions.py
• Updated definition.py with self-contained configuration
• Created backward-compatible re-exports in app/services/
• Updated legacy celery_config.py to not duplicate scheduled tasks

✅ Phase 6: Marketplace Module Migration

• Created app/modules/marketplace/services/ re-exporting from existing locations
• Created app/modules/marketplace/models/ re-exporting marketplace & letzshop models
• Created app/modules/marketplace/schemas/ re-exporting marketplace schemas
• Created app/modules/marketplace/tasks/ with:
  • import_tasks.py - process_marketplace_import, process_historical_import
  • sync_tasks.py - sync_store_directory (scheduled daily)
  • export_tasks.py - export_store_products_to_folder, export_marketplace_products
• Created app/modules/marketplace/exceptions.py
• Updated definition.py with self-contained configuration
• Updated legacy task files to re-export from new location
• Removed marketplace/letzshop/export from LEGACY_TASK_MODULES

✅ Phase 7: Dev-Tools Module Migration

• Created app/modules/dev_tools/models/ with:
  • architecture_scan.py - ArchitectureScan, ArchitectureViolation, ArchitectureRule, etc.
  • test_run.py - TestRun, TestResult, TestCollection
• Created app/modules/dev_tools/services/ re-exporting code_quality_service, test_runner_service
• Created app/modules/dev_tools/schemas/ (placeholder for future schemas)
• Created app/modules/dev_tools/tasks/ with:
  • code_quality.py - execute_code_quality_scan
  • test_runner.py - execute_test_run
• Created app/modules/dev_tools/exceptions.py with test runner exceptions
• Created app/modules/dev_tools/routes/api/ with admin router
• Updated definition.py with self-contained configuration
• Updated legacy model files to re-export from module location
• Updated legacy task files to re-export from module location
• Used lazy imports in __init__.py to avoid circular import issues

✅ Phase 8: Monitoring Module Migration

• Created app/modules/monitoring/models/ re-exporting CapacitySnapshot, AdminNotification
• Created app/modules/monitoring/services/ re-exporting BackgroundTasksService
• Created app/modules/monitoring/schemas/ (placeholder)
• Created app/modules/monitoring/exceptions.py with monitoring-specific exceptions
• Updated definition.py with self-contained configuration
• Used lazy imports in __init__.py to avoid circular import issues

✅ Phase 9: Customers Module Migration

• Created app/modules/customers/models/ re-exporting Customer, CustomerAddress, PasswordResetToken
• Created app/modules/customers/services/ re-exporting customer services
• Created app/modules/customers/schemas/ re-exporting customer schemas
• Created app/modules/customers/exceptions.py re-exporting customer exceptions
• Updated definition.py with self-contained configuration
• Used lazy imports in __init__.py to avoid circular import issues

✅ Phase 10: Orders Module Migration

• Created app/modules/orders/models/ re-exporting Order, OrderItem, Invoice, etc.
• Created app/modules/orders/services/ re-exporting order and invoice services
• Created app/modules/orders/schemas/ re-exporting order and invoice schemas
• Created app/modules/orders/exceptions.py re-exporting order exceptions
• Updated definition.py with self-contained configuration
• Used lazy imports in __init__.py to avoid circular import issues

✅ Phase 11: Inventory Module Migration

• Created app/modules/inventory/models/ re-exporting Inventory, InventoryTransaction
• Created app/modules/inventory/services/ re-exporting inventory services
• Created app/modules/inventory/schemas/ re-exporting inventory schemas
• Created app/modules/inventory/exceptions.py re-exporting inventory exceptions
• Updated definition.py with self-contained configuration
• Used lazy imports in __init__.py to avoid circular import issues

✅ Phase 12: Analytics & Messaging Module Migration

• Analytics Module:

  • Created app/modules/analytics/models/ (empty - uses data from other modules)
  • Created app/modules/analytics/services/ re-exporting StatsService, UsageService
  • Created app/modules/analytics/schemas/ re-exporting stats schemas
  • Created app/modules/analytics/exceptions.py with reporting exceptions
  • Updated definition.py with self-contained configuration

• Messaging Module:

  • Created app/modules/messaging/models/ re-exporting Conversation, Message, etc.
  • Created app/modules/messaging/services/ re-exporting messaging services
  • Created app/modules/messaging/schemas/ re-exporting messaging schemas
  • Created app/modules/messaging/exceptions.py re-exporting message exceptions
  • Updated definition.py with self-contained configuration

---

Module Migration Phases

Each module migration includes: services, models, schemas, tasks, templates (if any).

Phase 5: Billing Module (High Priority)

Why first: Has most Celery tasks, critical business logic.

Components to Move

From	To	Type
app/services/subscription_service.py	app/modules/billing/services/	Service
app/services/stripe_service.py	app/modules/billing/services/	Service
models/database/subscription.py	app/modules/billing/models/	Model
models/database/subscription_tier.py	app/modules/billing/models/	Model
models/schema/subscription.py	app/modules/billing/schemas/	Schema
app/tasks/celery_tasks/subscription.py	app/modules/billing/tasks/	Tasks

Target Structure

app/modules/billing/
├── __init__.py
├── definition.py          # Update with tasks_path, scheduled_tasks
├── config.py              # BillingConfig schema
├── exceptions.py          # BillingException, SubscriptionError, etc.
├── services/
│   ├── __init__.py
│   ├── subscription_service.py
│   └── stripe_service.py
├── models/
│   ├── __init__.py
│   ├── subscription.py
│   └── subscription_tier.py
├── schemas/
│   ├── __init__.py
│   └── subscription.py
├── tasks/
│   ├── __init__.py
│   └── subscription.py    # 4 scheduled tasks
├── routes/
│   ├── admin.py
│   └── store.py
└── migrations/
    └── versions/

Tasks

Task	Schedule	Queue
reset_period_counters	Daily 00:05	scheduled
check_trial_expirations	Daily 01:00	scheduled
sync_stripe_status	Hourly :30	scheduled
cleanup_stale_subscriptions	Weekly Sun 03:00	scheduled

Migration Checklist

• Create billing/services/ directory
• Move subscription_service.py → update imports
• Move stripe_service.py → update imports
• Create billing/models/ directory
• Move subscription models → update imports
• Create billing/schemas/ directory
• Move subscription schemas → update imports
• Create billing/tasks/ directory
• Move subscription tasks → update task paths
• Create billing/exceptions.py
• Update definition.py with tasks_path, scheduled_tasks
• Create module migrations
• Update all imports across codebase
• Test subscription flows
• Test scheduled tasks

---

Phase 6: Marketplace Module (High Priority)

Why second: Has most tasks, complex import logic.

Components to Move

From	To	Type
app/services/letzshop_service.py	app/modules/marketplace/services/	Service
app/services/marketplace_import_service.py	app/modules/marketplace/services/	Service
app/services/letzshop_credentials_service.py	app/modules/marketplace/services/	Service
models/database/letzshop_*.py	app/modules/marketplace/models/	Models
models/database/marketplace_import_job.py	app/modules/marketplace/models/	Model
app/tasks/celery_tasks/marketplace.py	app/modules/marketplace/tasks/	Tasks
app/tasks/celery_tasks/letzshop.py	app/modules/marketplace/tasks/	Tasks
app/tasks/celery_tasks/export.py	app/modules/marketplace/tasks/	Tasks

Target Structure

app/modules/marketplace/
├── __init__.py
├── definition.py
├── exceptions.py
├── services/
│   ├── __init__.py
│   ├── letzshop_service.py
│   ├── letzshop_credentials_service.py
│   ├── import_service.py
│   └── export_service.py
├── models/
│   ├── __init__.py
│   ├── letzshop_order.py
│   ├── letzshop_credentials.py
│   └── import_job.py
├── schemas/
│   └── ...
├── tasks/
│   ├── __init__.py
│   ├── import_tasks.py    # process_marketplace_import
│   ├── letzshop.py        # process_historical_import, sync_store_directory
│   └── export.py          # export tasks
└── routes/

Tasks

Task	Type	Queue
process_marketplace_import	On-demand	long_running
process_historical_import	On-demand	long_running
sync_store_directory	Scheduled (optional)	long_running
export_store_products_to_folder	On-demand	default
export_marketplace_products	On-demand	default

---

Phase 7: Dev-Tools Module (Medium Priority)

Why: Internal module with code quality and test runner tasks.

Components to Move

From	To	Type
app/services/code_quality_service.py	app/modules/dev_tools/services/	Service
app/services/test_runner_service.py	app/modules/dev_tools/services/	Service
models/database/architecture_scan.py	app/modules/dev_tools/models/	Model
models/database/test_run.py	app/modules/dev_tools/models/	Model
app/tasks/celery_tasks/code_quality.py	app/modules/dev_tools/tasks/	Tasks
app/tasks/celery_tasks/test_runner.py	app/modules/dev_tools/tasks/	Tasks

Target Structure

app/modules/dev_tools/
├── __init__.py
├── definition.py
├── services/
│   ├── code_quality_service.py
│   └── test_runner_service.py
├── models/
│   ├── architecture_scan.py
│   ├── architecture_violation.py
│   └── test_run.py
├── tasks/
│   ├── code_quality.py
│   └── test_runner.py
└── routes/

---

Phase 8: Monitoring Module (Medium Priority)

Components to Move

From	To	Type
app/services/background_tasks_service.py	app/modules/monitoring/services/	Service
capture_capacity_snapshot task	app/modules/monitoring/tasks/	Task
models/database/capacity_snapshot.py	app/modules/monitoring/models/	Model

Target Structure

app/modules/monitoring/
├── __init__.py
├── definition.py
├── services/
│   └── background_tasks_service.py
├── models/
│   └── capacity_snapshot.py
├── tasks/
│   └── capacity.py    # capture_capacity_snapshot
└── routes/

---

Phase 9: Customers Module (Core)

Components to Move

From	To
app/services/customer_service.py	app/modules/customers/services/
models/database/customer.py	app/modules/customers/models/
Customer-related schemas	app/modules/customers/schemas/

---

Phase 10: Orders Module

Components to Move

From	To
app/services/order_service.py	app/modules/orders/services/
models/database/order.py	app/modules/orders/models/
Order-related schemas	app/modules/orders/schemas/

---

Phase 11: Inventory Module

Components to Move

From	To
app/services/inventory_service.py	app/modules/inventory/services/
models/database/inventory*.py	app/modules/inventory/models/

---

Phase 12: Remaining Modules

• Analytics - Move analytics services
• Messaging - Move messaging service, notification models
• Tenancy - Move platform, merchant, store services

---

Phase 13: Cleanup & Full Celery Migration

After all modules are migrated:

13.1 Remove Fallback Tasks

Delete:

app/tasks/background_tasks.py
app/tasks/letzshop_tasks.py
app/tasks/subscription_tasks.py
app/tasks/code_quality_tasks.py
app/tasks/test_runner_tasks.py

13.2 Remove USE_CELERY Flag

# app/core/config.py
# DELETE: USE_CELERY: bool = False

13.3 Simplify Dispatcher

Rewrite app/tasks/dispatcher.py to Celery-only.

13.4 Delete Old Task Directory

app/tasks/celery_tasks/  # DELETE - moved to modules

13.5 Clean Up Old Services/Models

Remove files from:

• app/services/ (moved to modules)
• models/database/ (moved to modules)

Leave re-exports for backward compatibility if needed.

---

Queue Configuration

Final Queue Structure

Queue	Purpose	Modules
default	Fast tasks	marketplace (exports)
long_running	CPU-intensive	marketplace (imports), dev-tools
scheduled	Celery Beat	billing, monitoring

Task Routing

task_routes = {
    "app.modules.billing.tasks.*": {"queue": "scheduled"},
    "app.modules.marketplace.tasks.import_tasks.*": {"queue": "long_running"},
    "app.modules.marketplace.tasks.letzshop.*": {"queue": "long_running"},
    "app.modules.marketplace.tasks.export.*": {"queue": "default"},
    "app.modules.monitoring.tasks.*": {"queue": "scheduled"},
    "app.modules.dev_tools.tasks.*": {"queue": "long_running"},
}

---

Migration Strategy

For Each Module

1. Create directory structure
2. Move models first (fewest dependencies)
3. Move services (depend on models)
4. Move schemas (depend on models)
5. Move tasks (depend on services)
6. Update all imports across codebase
7. Create re-exports in old locations (temporary)
8. Test thoroughly
9. Remove re-exports in next phase

Import Update Strategy

Use IDE refactoring or script:

# Example: Update subscription_service imports
find . -name "*.py" -exec sed -i 's/from app.services.subscription_service/from app.modules.billing.services.subscription_service/g' {} \;

---

Verification Checklist

Per-Module Verification

• Module directory has all components
• All imports updated
• Services work
• API routes work
• Tasks execute correctly
• Scheduled tasks run (Celery Beat)
• No circular imports
• Tests pass

Final Verification

• All 12 tasks in modules
• USE_CELERY flag removed
• Fallback tasks deleted
• Old service files removed
• Old model files removed
• All tests pass
• Production smoke test

---

Priority Order

Priority	Phase	Module	Reason
1	4	Infrastructure	Required for task migration
2	5	billing	Most tasks, critical business logic
3	6	marketplace	Most complex, many tasks
4	7	dev-tools	Internal, low risk
5	8	monitoring	Internal, low risk
6	9-12	Others	Lower complexity
7	13	Cleanup	Final step

---

Estimated Effort

Phase	Sessions	Notes
4 (Infrastructure)	1	Task base class, discovery
5 (Billing)	2-3	Complex, many components
6 (Marketplace)	3-4	Most complex module
7-8 (Internal)	1-2	Simpler modules
9-12 (Others)	4-6	Medium complexity each
13 (Cleanup)	1	Removing old code

Total: ~12-17 sessions

---

Rollback Plan

Each phase can be rolled back independently:

1. Restore moved files to original locations
2. Revert import changes
3. Restore celery_config.py

No database schema changes during code migration = no DB rollback needed.

---

Related Documents

• Module System Architecture
• Creating Modules
• Observability
• SESSION_NOTE_2026-01-27