feat(arch): implement soft delete for business-critical models
Adds SoftDeleteMixin (deleted_at + deleted_by_id) with automatic query
filtering via do_orm_execute event. Soft-deleted records are invisible
by default; bypass with execution_options={"include_deleted": True}.
Models: User, Merchant, Store, StoreUser, Customer, Order, Product,
LoyaltyProgram, LoyaltyCard.
Infrastructure:
- SoftDeleteMixin in models/database/base.py
- Auto query filter registered on SessionLocal and test sessions
- soft_delete(), restore(), soft_delete_cascade() in app/core/soft_delete.py
- Alembic migration adding columns to 9 tables
- Partial unique indexes on users.email/username, stores.store_code/subdomain
Service changes:
- admin_service: delete_user, delete_store → soft_delete/soft_delete_cascade
- merchant_service: delete_merchant → soft_delete_cascade (stores→children)
- store_team_service: remove_team_member → soft_delete (fixes is_active bug)
- product_service: delete_product → soft_delete
- program_service: delete_program → soft_delete_cascade
Admin API:
- include_deleted/only_deleted query params on admin list endpoints
- PUT restore endpoints for users, merchants, stores
Tests: 9 unit tests for soft-delete infrastructure.
Docs: docs/backend/soft-delete.md + follow-up proposals.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
119
docs/backend/soft-delete.md
Normal file
119
docs/backend/soft-delete.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Soft Delete
|
||||
|
||||
## Overview
|
||||
|
||||
Business-critical records use soft delete instead of hard delete. When a record is "deleted", it gets a `deleted_at` timestamp instead of being removed from the database. This preserves data for investigation, auditing, and potential restoration.
|
||||
|
||||
## How It Works
|
||||
|
||||
### SoftDeleteMixin
|
||||
|
||||
Models opt into soft delete by inheriting `SoftDeleteMixin` (from `models/database/base.py`):
|
||||
|
||||
```python
|
||||
from models.database.base import SoftDeleteMixin, TimestampMixin
|
||||
|
||||
class MyModel(Base, TimestampMixin, SoftDeleteMixin):
|
||||
__tablename__ = "my_table"
|
||||
# ...
|
||||
```
|
||||
|
||||
This adds two columns:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `deleted_at` | DateTime (nullable, indexed) | When the record was deleted. NULL = alive. |
|
||||
| `deleted_by_id` | Integer (FK to users.id, nullable) | Who performed the deletion. |
|
||||
|
||||
### Automatic Query Filtering
|
||||
|
||||
A `do_orm_execute` event on the session automatically appends `WHERE deleted_at IS NULL` to all SELECT queries for models with `SoftDeleteMixin`. This means:
|
||||
|
||||
- **Normal queries never see deleted records** — no code changes needed
|
||||
- **Relationship lazy loads are also filtered** — e.g., `store.products` won't include deleted products
|
||||
|
||||
### Bypassing the Filter
|
||||
|
||||
To see deleted records (admin views, restore operations):
|
||||
|
||||
```python
|
||||
# Legacy query style
|
||||
db.query(User).execution_options(include_deleted=True).all()
|
||||
|
||||
# Core select style
|
||||
from sqlalchemy import select
|
||||
db.execute(
|
||||
select(User).filter(User.id == 42),
|
||||
execution_options={"include_deleted": True}
|
||||
).scalar_one_or_none()
|
||||
```
|
||||
|
||||
## Models Using Soft Delete
|
||||
|
||||
| Model | Table | Module |
|
||||
|-------|-------|--------|
|
||||
| User | users | tenancy |
|
||||
| Merchant | merchants | tenancy |
|
||||
| Store | stores | tenancy |
|
||||
| StoreUser | store_users | tenancy |
|
||||
| Customer | customers | customers |
|
||||
| Order | orders | orders |
|
||||
| Product | products | catalog |
|
||||
| LoyaltyProgram | loyalty_programs | loyalty |
|
||||
| LoyaltyCard | loyalty_cards | loyalty |
|
||||
|
||||
## Utility Functions
|
||||
|
||||
Import from `app.core.soft_delete`:
|
||||
|
||||
### `soft_delete(db, entity, deleted_by_id)`
|
||||
|
||||
Marks a single record as deleted.
|
||||
|
||||
### `restore(db, model_class, entity_id, restored_by_id)`
|
||||
|
||||
Restores a soft-deleted record. Queries with `include_deleted=True` internally.
|
||||
|
||||
### `soft_delete_cascade(db, entity, deleted_by_id, cascade_rels)`
|
||||
|
||||
Soft-deletes a record and recursively soft-deletes its children:
|
||||
|
||||
```python
|
||||
soft_delete_cascade(db, merchant, deleted_by_id=admin.id, cascade_rels=[
|
||||
("stores", [
|
||||
("products", []),
|
||||
("customers", []),
|
||||
("orders", []),
|
||||
("store_users", []),
|
||||
]),
|
||||
])
|
||||
```
|
||||
|
||||
## Partial Unique Indexes
|
||||
|
||||
Tables with unique constraints (e.g., `users.email`, `stores.store_code`) use **partial unique indexes** that only enforce uniqueness among non-deleted rows:
|
||||
|
||||
```sql
|
||||
CREATE UNIQUE INDEX uq_users_email_active ON users (email) WHERE deleted_at IS NULL;
|
||||
```
|
||||
|
||||
This allows a soft-deleted user's email to be reused by a new registration.
|
||||
|
||||
## Adding Soft Delete to a New Model
|
||||
|
||||
1. Add `SoftDeleteMixin` to the model class
|
||||
2. Create an alembic migration adding `deleted_at` and `deleted_by_id` columns
|
||||
3. If the model has unique constraints, convert them to partial unique indexes
|
||||
4. If the model has relationships to users (ForeignKey to users.id), add `foreign_keys=` to those relationships to resolve ambiguity with `deleted_by_id`
|
||||
5. Register the test session factory with `register_soft_delete_filter()` if not already done
|
||||
|
||||
## What Stays as Hard Delete
|
||||
|
||||
Operational and config data that doesn't need investigation trail:
|
||||
|
||||
- Roles, themes, email settings, invoice settings
|
||||
- Cart items, application logs, notifications
|
||||
- Password/email verification tokens
|
||||
- Domains (store and merchant)
|
||||
- Content pages, media files
|
||||
- Import jobs, marketplace products
|
||||
143
docs/proposals/post-soft-delete-followups.md
Normal file
143
docs/proposals/post-soft-delete-followups.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Post Soft-Delete Follow-up Tasks
|
||||
|
||||
**Date:** 2026-03-28
|
||||
**Context:** During the soft-delete implementation session, several gaps were identified in the platform. This proposal outlines 6 follow-up tasks in priority order.
|
||||
|
||||
---
|
||||
|
||||
## 1. Admin Email Verification Gap (Quick Fix)
|
||||
|
||||
**Problem:** Admin users (super_admin, platform_admin) are created with `is_email_verified=False` (model default). Login checks `is_email_verified` and blocks unverified users. But there's no admin-facing email verification flow — no verification email is sent on admin creation, and `/resend-verification` is merchant-scoped.
|
||||
|
||||
**Impact:** Newly created admin accounts can't log in until somehow email-verified.
|
||||
|
||||
**Proposed Fix:** Auto-set `is_email_verified=True` when creating admin users via `admin_platform_service.create_super_admin()` and `create_platform_admin()`. Admins are created by super admins, so trust is implicit.
|
||||
|
||||
**Alternative:** Send a verification email on admin creation using the existing `EmailVerificationToken` model and `/verify-email` page endpoint.
|
||||
|
||||
**Files:**
|
||||
- `app/modules/tenancy/services/admin_platform_service.py` — `create_super_admin()`, `create_platform_admin()`
|
||||
|
||||
**Effort:** Small (< 30 min)
|
||||
|
||||
---
|
||||
|
||||
## 2. Customer Soft-Delete Endpoint (Compliance)
|
||||
|
||||
**Problem:** Customers have no delete endpoint at all — not soft delete, not hard delete. Only customer addresses can be deleted. This is a gap for GDPR/data-subject-deletion compliance.
|
||||
|
||||
**Proposed Fix:** Add soft-delete endpoints:
|
||||
- `DELETE /api/v1/store/customers/{customer_id}` — store owner/staff can soft-delete
|
||||
- `DELETE /api/v1/admin/customers/{customer_id}` — admin can soft-delete
|
||||
|
||||
Customer already has `SoftDeleteMixin`. Consider cascading to orders, addresses, and loyalty cards.
|
||||
|
||||
**Files:**
|
||||
- `app/modules/customers/routes/api/store.py` — new DELETE endpoint
|
||||
- `app/modules/customers/services/customer_service.py` — new `delete_customer()` method
|
||||
|
||||
**Effort:** Medium (1-2 hours)
|
||||
|
||||
---
|
||||
|
||||
## 3. Cascade Restore Utility
|
||||
|
||||
**Problem:** `restore()` only restores a single record. Restoring a merchant doesn't auto-restore its stores/products/customers/orders. Admin has to restore each entity one by one.
|
||||
|
||||
**Proposed Fix:** Add `restore_cascade()` to `app/core/soft_delete.py` mirroring `soft_delete_cascade()`. Walk the same relationship tree. Add optional `cascade=true` query param to existing restore endpoints:
|
||||
- `PUT /api/v1/admin/merchants/{id}/restore?cascade=true`
|
||||
- `PUT /api/v1/admin/stores/{id}/restore?cascade=true`
|
||||
|
||||
**Files:**
|
||||
- `app/core/soft_delete.py` — new `restore_cascade()` function
|
||||
- `app/modules/tenancy/routes/api/admin_stores.py` — update restore endpoint
|
||||
- `app/modules/tenancy/routes/api/admin_merchants.py` — update restore endpoint
|
||||
|
||||
**Effort:** Small-Medium (1 hour)
|
||||
|
||||
---
|
||||
|
||||
## 4. Admin Trash UI
|
||||
|
||||
**Problem:** The soft-delete API supports `?only_deleted=true` on admin list endpoints (stores, merchants, users) but there's no UI to browse or restore deleted records.
|
||||
|
||||
**Proposed Fix:** Add a "Trash" toggle/tab to admin list pages:
|
||||
- `admin/stores.html` — toggle between active stores and trash
|
||||
- `admin/merchants.html` — same
|
||||
- `admin/admin-users.html` — same (super admin only)
|
||||
|
||||
Each deleted row shows `deleted_at`, `deleted_by`, and a "Restore" button calling `PUT /api/v1/admin/{entity}/{id}/restore`.
|
||||
|
||||
**Implementation:** The Alpine.js components need a `showDeleted` toggle state that:
|
||||
- Adds `?only_deleted=true` to the list API call
|
||||
- Shows a different table header (with deleted_at column)
|
||||
- Replaces edit/delete actions with a Restore button
|
||||
|
||||
**Files:**
|
||||
- `app/modules/tenancy/templates/tenancy/admin/stores.html`
|
||||
- `app/modules/tenancy/templates/tenancy/admin/merchants.html`
|
||||
- `app/modules/tenancy/templates/tenancy/admin/admin-users.html`
|
||||
- Corresponding JS files in `app/modules/tenancy/static/admin/js/`
|
||||
|
||||
**Effort:** Medium (2-3 hours)
|
||||
|
||||
---
|
||||
|
||||
## 5. Admin Team Management Page
|
||||
|
||||
**Problem:** There is no admin-level page for managing store teams. The admin can see merchant users at `/admin/merchant-users`, but this is a user-centric view — not team-centric. Admin cannot:
|
||||
- View team members per store
|
||||
- Invite/remove team members on behalf of a store
|
||||
- See team composition across the platform
|
||||
|
||||
Store owners manage their teams at `/store/{code}/team`. Merchants manage across stores at `/merchants/account/team`. But admin has no equivalent.
|
||||
|
||||
**Proposed Fix:** Add `/admin/stores/{store_code}/team` page that reuses the existing store team API endpoints (`/api/v1/store/team/*`) with admin auth context. The admin store detail page should link to it.
|
||||
|
||||
**Components needed:**
|
||||
- Page route in `app/modules/tenancy/routes/pages/admin.py`
|
||||
- Template at `app/modules/tenancy/templates/tenancy/admin/store-team.html`
|
||||
- JS component (can largely reuse `store/js/team.js` patterns)
|
||||
- Menu item or link from store detail page
|
||||
|
||||
**Consideration:** Admin already has `/admin/store-roles` for role CRUD. The team page completes the picture.
|
||||
|
||||
**Effort:** Medium-Large (3-4 hours)
|
||||
|
||||
---
|
||||
|
||||
## 6. Merchant Team Roles Page
|
||||
|
||||
**Problem:** Store frontend has a full roles management page (`/store/{code}/team/roles`) with CRUD for custom roles and granular permissions. Merchant portal has no equivalent — merchants can only assign preset roles (manager, staff, support, viewer, marketing) during invite/edit, not create custom roles.
|
||||
|
||||
**Proposed Fix:** Add `/merchants/account/team/roles` page. Since roles are per-store in the data model, the page should:
|
||||
1. Let merchant pick a store from a dropdown
|
||||
2. Show roles for that store (reusing `GET /account/team/stores/{store_id}/roles`)
|
||||
3. Allow CRUD on custom roles (delegating to store team service)
|
||||
|
||||
**Files:**
|
||||
- New page route in `app/modules/tenancy/routes/pages/merchant.py`
|
||||
- New template at `app/modules/tenancy/templates/tenancy/merchant/team-roles.html`
|
||||
- New JS at `app/modules/tenancy/static/merchant/js/merchant-roles.js`
|
||||
- New API endpoints in `app/modules/tenancy/routes/api/merchant.py`
|
||||
- Menu item in `app/modules/tenancy/definition.py` (merchant menu)
|
||||
- i18n keys in 4 locale files
|
||||
|
||||
**Reference:** Store roles page at `templates/tenancy/store/roles.html` and `static/store/js/roles.js`
|
||||
|
||||
**Effort:** Large (4-5 hours)
|
||||
|
||||
---
|
||||
|
||||
## Priority & Sequencing
|
||||
|
||||
| # | Task | Priority | Effort | Dependency |
|
||||
|---|------|----------|--------|------------|
|
||||
| 1 | Admin email verification | Critical | Small | None |
|
||||
| 2 | Customer soft-delete | High (compliance) | Medium | None |
|
||||
| 3 | Cascade restore | Medium | Small | None |
|
||||
| 4 | Admin trash UI | Medium | Medium | None |
|
||||
| 5 | Admin team management | Medium | Medium-Large | None |
|
||||
| 6 | Merchant roles page | Low | Large | None |
|
||||
|
||||
Tasks 1-3 can be done in a single session. Tasks 4-6 are independent and can be tackled in any order.
|
||||
Reference in New Issue
Block a user