orion/docs/proposals/module-migration-plan.md

Module Migration Plan

Status: In Progress Started: 2026-01-25 Last Updated: 2026-01-27 Target: v1.0.0

This is the unified migration plan for transforming Wizamart into a fully modular architecture.

---

Executive Summary

Transform the platform from a monolithic structure to self-contained modules where each module owns its:

• Services (business logic)
• Models (database entities)
• Schemas (Pydantic DTOs)
• Tasks (Celery background jobs)
• Templates (UI if applicable)
• Migrations (database changes)

---

Current State Assessment

Module Migration Status

Module	Classification	Current State	Services	Models	Tasks	Target
cms	Core	✅ Complete	✅	✅	-	Done
payments	Optional	🟡 Partial	✅	✅	-	Done
billing	Optional	🔴 Shell	❌	❌	❌	Full
marketplace	Optional	🔴 Shell	❌	❌	❌	Full
orders	Optional	🔴 Shell	❌	❌	-	Full
inventory	Optional	🔴 Shell	❌	❌	-	Full
customers	Core	🔴 Shell	❌	❌	-	Full
analytics	Optional	🔴 Shell	❌	❌	-	Full
messaging	Optional	🔴 Shell	❌	❌	-	Full
monitoring	Internal	🔴 Shell	❌	❌	❌	Full
dev-tools	Internal	🔴 Shell	❌	❌	❌	Full
tenancy	Core	🔴 Shell	❌	❌	-	Full
core	Core	🔴 Shell	❌	❌	-	Minimal

Current Code Locations (To Be Migrated)

app/services/                    # → Move to respective modules
├── subscription_service.py      # → billing/services/
├── stripe_service.py            # → billing/services/ (or payments)
├── customer_service.py          # → customers/services/
├── order_service.py             # → orders/services/
├── inventory_service.py         # → inventory/services/
├── letzshop_service.py          # → marketplace/services/
├── marketplace_import_service.py # → marketplace/services/
├── messaging_service.py         # → messaging/services/
└── ...

models/database/                 # → Move to respective modules
├── subscription.py              # → billing/models/
├── customer.py                  # → customers/models/
├── order.py                     # → orders/models/
├── inventory.py                 # → inventory/models/
├── letzshop_*.py               # → marketplace/models/
└── ...

app/tasks/celery_tasks/          # → Move to respective modules
├── subscription.py              # → billing/tasks/
├── marketplace.py               # → marketplace/tasks/
├── letzshop.py                  # → marketplace/tasks/
├── export.py                    # → marketplace/tasks/
├── code_quality.py              # → dev_tools/tasks/
└── test_runner.py               # → dev_tools/tasks/

Celery Task Mapping

Current Task	Current Location	Target Module
reset_period_counters	subscription.py	billing
check_trial_expirations	subscription.py	billing
sync_stripe_status	subscription.py	billing
cleanup_stale_subscriptions	subscription.py	billing
capture_capacity_snapshot	subscription.py	monitoring
process_marketplace_import	marketplace.py	marketplace
process_historical_import	letzshop.py	marketplace
sync_vendor_directory	letzshop.py	marketplace
export_vendor_products_to_folder	export.py	marketplace
export_marketplace_products	export.py	marketplace
execute_code_quality_scan	code_quality.py	dev-tools
execute_test_run	test_runner.py	dev-tools

---

Infrastructure Work (Completed)

✅ Phase 1: Module Classification

• Three-tier system: Core, Optional, Internal
• Renamed platform-admin → tenancy
• Module registry with CORE_MODULES, OPTIONAL_MODULES, INTERNAL_MODULES

✅ Phase 2: Module Infrastructure

• ModuleDefinition with lifecycle hooks
• Module events system (events.py)
• Module migration discovery (migrations.py)
• Observability framework (observability.py)

✅ Phase 3: Documentation

• Module system architecture docs
• Menu management docs
• Observability docs
• Creating modules developer guide

---

Infrastructure Work (Remaining)

Phase 4: Celery Task Infrastructure

Add task support to module system before migrating modules.

4.1 Update ModuleDefinition

# app/modules/base.py - Add fields

@dataclass
class ScheduledTask:
    """Celery Beat scheduled task definition."""
    name: str                    # e.g., "billing.reset_counters"
    task: str                    # Full task path
    schedule: str | dict         # Cron string or crontab dict
    args: tuple = ()
    kwargs: dict = field(default_factory=dict)

@dataclass
class ModuleDefinition:
    # ... existing fields ...

    # Task configuration (NEW)
    tasks_path: str | None = None
    scheduled_tasks: list[ScheduledTask] = field(default_factory=list)

4.2 Create Task Discovery

# app/modules/tasks.py (NEW)

def discover_module_tasks() -> list[str]:
    """Discover task modules from all registered modules."""
    ...

def build_beat_schedule() -> dict:
    """Build Celery Beat schedule from module definitions."""
    ...

4.3 Create Module Task Base

# app/modules/task_base.py (NEW)

class ModuleTask(Task):
    """Base Celery task with DB session management."""

    @contextmanager
    def get_db(self):
        ...

4.4 Update Celery Config

# app/core/celery_config.py

from app.modules.tasks import discover_module_tasks, build_beat_schedule

# Auto-discover from modules
celery_app.autodiscover_tasks(discover_module_tasks())

# Build schedule from modules
celery_app.conf.beat_schedule = build_beat_schedule()

Files to create:

• app/modules/tasks.py
• app/modules/task_base.py

Files to modify:

• app/modules/base.py
• app/core/celery_config.py

---

Module Migration Phases

Each module migration includes: services, models, schemas, tasks, templates (if any).

Phase 5: Billing Module (High Priority)

Why first: Has most Celery tasks, critical business logic.

Components to Move

From	To	Type
app/services/subscription_service.py	app/modules/billing/services/	Service
app/services/stripe_service.py	app/modules/billing/services/	Service
models/database/subscription.py	app/modules/billing/models/	Model
models/database/subscription_tier.py	app/modules/billing/models/	Model
models/schema/subscription.py	app/modules/billing/schemas/	Schema
app/tasks/celery_tasks/subscription.py	app/modules/billing/tasks/	Tasks

Target Structure

app/modules/billing/
├── __init__.py
├── definition.py          # Update with tasks_path, scheduled_tasks
├── config.py              # BillingConfig schema
├── exceptions.py          # BillingException, SubscriptionError, etc.
├── services/
│   ├── __init__.py
│   ├── subscription_service.py
│   └── stripe_service.py
├── models/
│   ├── __init__.py
│   ├── subscription.py
│   └── subscription_tier.py
├── schemas/
│   ├── __init__.py
│   └── subscription.py
├── tasks/
│   ├── __init__.py
│   └── subscription.py    # 4 scheduled tasks
├── routes/
│   ├── admin.py
│   └── vendor.py
└── migrations/
    └── versions/

Tasks

Task	Schedule	Queue
reset_period_counters	Daily 00:05	scheduled
check_trial_expirations	Daily 01:00	scheduled
sync_stripe_status	Hourly :30	scheduled
cleanup_stale_subscriptions	Weekly Sun 03:00	scheduled

Migration Checklist

• Create billing/services/ directory
• Move subscription_service.py → update imports
• Move stripe_service.py → update imports
• Create billing/models/ directory
• Move subscription models → update imports
• Create billing/schemas/ directory
• Move subscription schemas → update imports
• Create billing/tasks/ directory
• Move subscription tasks → update task paths
• Create billing/exceptions.py
• Update definition.py with tasks_path, scheduled_tasks
• Create module migrations
• Update all imports across codebase
• Test subscription flows
• Test scheduled tasks

---

Phase 6: Marketplace Module (High Priority)

Why second: Has most tasks, complex import logic.

Components to Move

From	To	Type
app/services/letzshop_service.py	app/modules/marketplace/services/	Service
app/services/marketplace_import_service.py	app/modules/marketplace/services/	Service
app/services/letzshop_credentials_service.py	app/modules/marketplace/services/	Service
models/database/letzshop_*.py	app/modules/marketplace/models/	Models
models/database/marketplace_import_job.py	app/modules/marketplace/models/	Model
app/tasks/celery_tasks/marketplace.py	app/modules/marketplace/tasks/	Tasks
app/tasks/celery_tasks/letzshop.py	app/modules/marketplace/tasks/	Tasks
app/tasks/celery_tasks/export.py	app/modules/marketplace/tasks/	Tasks

Target Structure

app/modules/marketplace/
├── __init__.py
├── definition.py
├── exceptions.py
├── services/
│   ├── __init__.py
│   ├── letzshop_service.py
│   ├── letzshop_credentials_service.py
│   ├── import_service.py
│   └── export_service.py
├── models/
│   ├── __init__.py
│   ├── letzshop_order.py
│   ├── letzshop_credentials.py
│   └── import_job.py
├── schemas/
│   └── ...
├── tasks/
│   ├── __init__.py
│   ├── import_tasks.py    # process_marketplace_import
│   ├── letzshop.py        # process_historical_import, sync_vendor_directory
│   └── export.py          # export tasks
└── routes/

Tasks

Task	Type	Queue
process_marketplace_import	On-demand	long_running
process_historical_import	On-demand	long_running
sync_vendor_directory	Scheduled (optional)	long_running
export_vendor_products_to_folder	On-demand	default
export_marketplace_products	On-demand	default

---

Phase 7: Dev-Tools Module (Medium Priority)

Why: Internal module with code quality and test runner tasks.

Components to Move

From	To	Type
app/services/code_quality_service.py	app/modules/dev_tools/services/	Service
app/services/test_runner_service.py	app/modules/dev_tools/services/	Service
models/database/architecture_scan.py	app/modules/dev_tools/models/	Model
models/database/test_run.py	app/modules/dev_tools/models/	Model
app/tasks/celery_tasks/code_quality.py	app/modules/dev_tools/tasks/	Tasks
app/tasks/celery_tasks/test_runner.py	app/modules/dev_tools/tasks/	Tasks

Target Structure

app/modules/dev_tools/
├── __init__.py
├── definition.py
├── services/
│   ├── code_quality_service.py
│   └── test_runner_service.py
├── models/
│   ├── architecture_scan.py
│   ├── architecture_violation.py
│   └── test_run.py
├── tasks/
│   ├── code_quality.py
│   └── test_runner.py
└── routes/

---

Phase 8: Monitoring Module (Medium Priority)

Components to Move

From	To	Type
app/services/background_tasks_service.py	app/modules/monitoring/services/	Service
capture_capacity_snapshot task	app/modules/monitoring/tasks/	Task
models/database/capacity_snapshot.py	app/modules/monitoring/models/	Model

Target Structure

app/modules/monitoring/
├── __init__.py
├── definition.py
├── services/
│   └── background_tasks_service.py
├── models/
│   └── capacity_snapshot.py
├── tasks/
│   └── capacity.py    # capture_capacity_snapshot
└── routes/

---

Phase 9: Customers Module (Core)

Components to Move

From	To
app/services/customer_service.py	app/modules/customers/services/
models/database/customer.py	app/modules/customers/models/
Customer-related schemas	app/modules/customers/schemas/

---

Phase 10: Orders Module

Components to Move

From	To
app/services/order_service.py	app/modules/orders/services/
models/database/order.py	app/modules/orders/models/
Order-related schemas	app/modules/orders/schemas/

---

Phase 11: Inventory Module

Components to Move

From	To
app/services/inventory_service.py	app/modules/inventory/services/
models/database/inventory*.py	app/modules/inventory/models/

---

Phase 12: Remaining Modules

• Analytics - Move analytics services
• Messaging - Move messaging service, notification models
• Tenancy - Move platform, company, vendor services

---

Phase 13: Cleanup & Full Celery Migration

After all modules are migrated:

13.1 Remove Fallback Tasks

Delete:

app/tasks/background_tasks.py
app/tasks/letzshop_tasks.py
app/tasks/subscription_tasks.py
app/tasks/code_quality_tasks.py
app/tasks/test_runner_tasks.py

13.2 Remove USE_CELERY Flag

# app/core/config.py
# DELETE: USE_CELERY: bool = False

13.3 Simplify Dispatcher

Rewrite app/tasks/dispatcher.py to Celery-only.

13.4 Delete Old Task Directory

app/tasks/celery_tasks/  # DELETE - moved to modules

13.5 Clean Up Old Services/Models

Remove files from:

• app/services/ (moved to modules)
• models/database/ (moved to modules)

Leave re-exports for backward compatibility if needed.

---

Queue Configuration

Final Queue Structure

Queue	Purpose	Modules
default	Fast tasks	marketplace (exports)
long_running	CPU-intensive	marketplace (imports), dev-tools
scheduled	Celery Beat	billing, monitoring

Task Routing

task_routes = {
    "app.modules.billing.tasks.*": {"queue": "scheduled"},
    "app.modules.marketplace.tasks.import_tasks.*": {"queue": "long_running"},
    "app.modules.marketplace.tasks.letzshop.*": {"queue": "long_running"},
    "app.modules.marketplace.tasks.export.*": {"queue": "default"},
    "app.modules.monitoring.tasks.*": {"queue": "scheduled"},
    "app.modules.dev_tools.tasks.*": {"queue": "long_running"},
}

---

Migration Strategy

For Each Module

1. Create directory structure
2. Move models first (fewest dependencies)
3. Move services (depend on models)
4. Move schemas (depend on models)
5. Move tasks (depend on services)
6. Update all imports across codebase
7. Create re-exports in old locations (temporary)
8. Test thoroughly
9. Remove re-exports in next phase

Import Update Strategy

Use IDE refactoring or script:

# Example: Update subscription_service imports
find . -name "*.py" -exec sed -i 's/from app.services.subscription_service/from app.modules.billing.services.subscription_service/g' {} \;

---

Verification Checklist

Per-Module Verification

• Module directory has all components
• All imports updated
• Services work
• API routes work
• Tasks execute correctly
• Scheduled tasks run (Celery Beat)
• No circular imports
• Tests pass

Final Verification

• All 12 tasks in modules
• USE_CELERY flag removed
• Fallback tasks deleted
• Old service files removed
• Old model files removed
• All tests pass
• Production smoke test

---

Priority Order

Priority	Phase	Module	Reason
1	4	Infrastructure	Required for task migration
2	5	billing	Most tasks, critical business logic
3	6	marketplace	Most complex, many tasks
4	7	dev-tools	Internal, low risk
5	8	monitoring	Internal, low risk
6	9-12	Others	Lower complexity
7	13	Cleanup	Final step

---

Estimated Effort

Phase	Sessions	Notes
4 (Infrastructure)	1	Task base class, discovery
5 (Billing)	2-3	Complex, many components
6 (Marketplace)	3-4	Most complex module
7-8 (Internal)	1-2	Simpler modules
9-12 (Others)	4-6	Medium complexity each
13 (Cleanup)	1	Removing old code

Total: ~12-17 sessions

---

Rollback Plan

Each phase can be rolled back independently:

1. Restore moved files to original locations
2. Revert import changes
3. Restore celery_config.py

No database schema changes during code migration = no DB rollback needed.

---

Related Documents

• Module System Architecture
• Creating Modules
• Observability
• SESSION_NOTE_2026-01-27