orion/app/modules/loyalty/docs/production-launch-plan.md

Loyalty Module — Production Launch Plan

Created: 2026-04-09 Owner: Samir Status: Planning locked, awaiting Phase 1 kickoff

This is the active execution plan for taking the Loyalty module to production. It complements the earlier Production Readiness review by sequencing the work into phases and recording the decisions made.

---

Locked Decisions

#	Topic	Decision
1	Apple Wallet	Deferred to Phase 9 (post-launch). Google Wallet only at launch.
2	Secrets path	Google service account JSON lives at ~/apps/orion/google-wallet-sa.json in production (app user, mode 600). Different from Caddy TLS certs at /etc/caddy/certs/.
3	Notification channels	Email only. No SMS at launch.
4	Notification scope	Enrollment, welcome bonus, points-expiring (14d warning), points-expired, reward-ready, birthday, re-engagement (inactive). 7 templates × 4 locales = 28 seed rows.
5	T&C strategy	CMS integration scoped to the program's owning store. New terms_cms_page_slug column on loyalty_programs. Legacy terms_text kept one release as fallback.
6	Coverage target	80% on app/modules/loyalty/* (enforced in CI for loyalty job only).
7	GDPR deletion	Anonymize (null customer_id, scrub PII fields, keep aggregate transactions).
8	Bulk actions	Primary endpoints on merchant router; admin "act on behalf of" endpoints under admin router accept merchant_id and stamp acting_admin_id in audit log.

---

Verified Reuse (no new infra)

Need	Existing component
Email	app/modules/messaging/services/email_service.py — multi-provider, DB templates, 4 locales, EmailLog. SMTP enabled.
Background tasks	Celery (@shared_task, beat schedule in app/modules/loyalty/definition.py).
Rate limiting	middleware/decorators.py @rate_limit decorator (already used in routes/api/storefront.py).
Migrations	Alembic loyalty branch (loyalty_001, loyalty_002).
pytest-cov	Wired in pyproject.toml; currently --cov-fail-under=0 (disabled).

---

Migration Chain

loyalty_002 (existing)
loyalty_003 — CHECK constraints (points_balance >= 0, stamp_count >= 0, etc.)
loyalty_004 — relax card uniqueness: replace (merchant_id, customer_id) unique index
              with (enrolled_at_store_id, customer_id) for cross-location support
loyalty_005 — seed 28 notification email templates
loyalty_006 — add columns: last_expiration_warning_at, last_reengagement_at on cards;
              acting_admin_id on transactions
loyalty_007 — terms_cms_page_slug on programs

customers_003 — birth_date on customers (Phase 1.4 fix, dropped data bug)

---

Phases

Phase 0 — Decisions & prerequisites (done)

All 8 decisions locked. No external blockers.

Phase 1 — Config, Secrets & Security Hardening (✅ DONE 2026-04-09)

1.1 Google Wallet config cleanup

• Move LOYALTY_GOOGLE_SERVICE_ACCOUNT_JSON from dev path to ~/apps/orion/google-wallet-sa.json in prod .env.
• Add Pydantic @field_validator in app/modules/loyalty/config.py:39-43 that checks file exists and is readable when google_issuer_id is set. Fail fast at import time if not.
• Validation: unit test using tmp_path; manual: missing file → clear startup error.

1.2 Apple Wallet timing-safe token compare (pull-forward from Phase 9)

• Replace != with hmac.compare_digest() at app/modules/loyalty/services/apple_wallet_service.py:98.
• 15 min, code is safe whenever Apple ships.
• Validation: unit test with wrong token.

1.3 Rate-limit store mutating endpoints

• Apply @rate_limit to routes/api/store.py:600-795:
  • POST /stamp 60/min
  • POST /stamp/redeem 30/min
  • POST /stamp/void 20/min
  • POST /points/earn 60/min
  • POST /points/redeem 30/min
  • POST /points/void 20/min
  • POST /cards/{card_id}/points/adjust 20/min
  • POST /pins/{pin_id}/unlock 10/min
• Caps confirmed.
• Validation: httpx loop exceeds limit → 429.

1.4 Fix dropped birthday data (P0 bug)

Background: The enrollment form (enroll.html:87) collects a birthday, the schema accepts customer_birthday (schemas/card.py:34), and card_service.resolve_customer_id has the parameter (card_service.py:172) — but the call to customer_service.create_customer_for_enrollment at card_service.py:215-222 does not pass it, and the Customer model has no birth_date column at all. Every enrollment silently loses the birthday. Not live yet, so no backfill needed.

• Migration customers_003_add_birth_date.py adds birth_date: Date | None to customers.
• Update customer_service.create_customer_for_enrollment to accept and persist birth_date.
• Update card_service.py:215-222 to pass customer_birthday through, with date.fromisoformat() parsing and validation (must be a real past date, sensible age range).
• Update customer_service.update_customer to allow backfill.
• Add unit + integration tests for enrollment-with-birthday and enrollment-without-birthday paths.
• Validation: enroll with birthday → row in customers has birth_date set; enrollment without birthday still works.

1.5 DB CHECK constraints migration loyalty_003

• Add: loyalty_cards.points_balance >= 0, loyalty_cards.stamp_count >= 0, loyalty_programs.minimum_purchase_cents >= 0, loyalty_programs.points_per_euro >= 0, loyalty_programs.stamps_target >= 1, loyalty_programs.welcome_bonus_points >= 0.
• Mirror as CheckConstraint in models.
• Pre-flight: scan dev DB for existing violations; include data fix in upgrade() if found.
• Validation: migration runs clean; direct UPDATE to negative value fails.

---

Phase 2 — Notifications Infrastructure (4d)

2.1 LoyaltyNotificationService

• New app/modules/loyalty/services/notification_service.py with methods:
  • send_enrollment_confirmation(db, card)
  • send_welcome_bonus(db, card, points) (may merge into enrollment)
  • send_points_expiration_warning(db, card, days_before, expiring_points, expiration_date)
  • send_points_expired(db, card, expired_points)
  • send_reward_available(db, card, reward_description)
  • send_birthday_message(db, card)
  • send_reengagement(db, card, days_inactive)
• All methods enqueue via Celery (2.2), never call EmailService inline.

2.2 Celery dispatch task

• New app/modules/loyalty/tasks/notifications.py with @shared_task(name="loyalty.send_notification_email", bind=True, max_retries=3, default_retry_delay=60).
• Opens fresh SessionLocal, calls EmailService(db).send_template(...), retries on SMTP errors.

2.3 Seed templates loyalty_005

• 7 templates × 4 locales (en, fr, de, lb) = 28 rows in email_templates.
• Template codes: loyalty_enrollment, loyalty_welcome_bonus, loyalty_points_expiring, loyalty_points_expired, loyalty_reward_ready, loyalty_birthday, loyalty_reengagement.
• Copywriting needs sign-off before applying to prod.

2.4 Wire enrollment confirmation

• In card_service.enroll_customer_for_store (~lines 480-540), call notification service after db.commit().

2.5 Wire expiration warning into expiration task

• Migration loyalty_006 adds last_expiration_warning_at to prevent duplicates.
• In rewritten tasks/point_expiration.py (see 3.1), find cards 14 days from expiry, fire warning, stamp timestamp.
• Validation: time-mocked test — fires once at 14-day mark.

2.6 Wire reward-ready notification

• In services/stamp_service.add_stamp, after commit, if card.stamp_count == program.stamps_target, dispatch.

2.7 Birthday Celery beat task

• New @shared_task loyalty.send_birthday_emails, daily 0 8 * * * in definition.py.
• Queries cards where customer.birth_date matches today (month/day), respects timezone.
• Depends on Phase 1.4 (column + persistence already in place).

2.8 Re-engagement Celery beat task

• Weekly schedule. Finds cards inactive > N days (default 60, configurable).
• Throttled via last_reengagement_at (added in loyalty_006) — once per quarter per card.

---

Phase 3 — Task Reliability (1.5d)

3.1 Batched point expiration

• Rewrite tasks/point_expiration.py:154-185 from per-card loop to set-based SQL:
  • Per program, chunk card IDs with LIMIT 500 FOR UPDATE SKIP LOCKED
  • Build expiration loyalty_transactions rows in bulk per chunk
  • UPDATE loyalty_cards SET points_balance=0, total_points_voided=total_points_voided+points_balance WHERE id = ANY(...)
  • Commit per chunk inside savepoint
• Done same pass as 2.5 (warning logic).
• Validation: seed 10k cards, time the run, verify flat memory and chunked commits.

3.2 Wallet sync exponential backoff

• Replace time.sleep(2) retry at tasks/wallet_sync.py:71-95 with exponential backoff [1s, 4s, 16s].
• Per-card try/except (one bad card doesn't kill the batch).
• Log failed_card_ids to observability.
• Use tenacity if available, otherwise small helper.

---

Phase 4 — Accessibility & T&C (2d)

4.1 T&C via store CMS integration

• Migration loyalty_007: add terms_cms_page_slug: str | None to loyalty_programs.
• Update schemas/program.py (ProgramCreate, ProgramUpdate).
• program-form.html:251 — CMS page picker scoped to the program's owning store.
• enroll.html:99-160 — resolve slug to CMS page URL/content; legacy terms_text fallback.
• Service layer: program_service.py:491,948.
• JS: loyalty-program-form.js:34,75,106.
• Pre-flight: confirm CMS module exposes "get pages by store" API.

4.2 Accessibility audit

• All loyalty templates: icon-only buttons → aria-label; modals → role="dialog" + aria-modal="true" + aria-labelledby + focus trap + Escape key + return-focus-on-close.
• PIN entry modal: inputmode="numeric", autocomplete="one-time-code", aria-live="polite" for errors.
• Pattern reference: existing Escape handler in enroll.html:147.
• Validation: axe-core on the 5 main loyalty pages, zero critical violations.

---

Phase 5 — Google Wallet Production Hardening (1d)

5.1 Cert deployment

• Place service account JSON at ~/apps/orion/google-wallet-sa.json, app user, mode 600.
• .env line 196 updated.
• Startup validators from 1.1 confirm load.

5.2 Conditional UI flag

• Storefront API returns google_wallet_enabled: bool (derived from config.is_google_wallet_enabled).
• enroll.html:99-125: hide Google button when disabled.
• Apple Wallet button removed entirely (or hidden behind feature flag for Phase 9 reintroduction).

5.3 End-to-end real-device test

• Android device → enroll → "Add to Google Wallet" → verify pass renders → trigger stamp → verify pass auto-updates within ~60s.

---

Phase 6 — Admin UX, GDPR, Bulk (3d)

6.1 Admin trash UI

• Trash tab on programs list and cards list, calling existing ?only_deleted=true API.
• Restore button per row + hard-delete destructive action.
• Templates: templates/loyalty/admin/programs.html, cards.html + Alpine JS.

6.2 Cascade restore

• program_service.restore_for_merchant(db, merchant_id) — restores soft-deleted programs and cards under a merchant.
• Hooked into tenancy module's restore flow via event subscriber or direct call.
• Validation: soft-delete merchant + 2 programs + 50 cards; restore merchant; all 52 rows un-deleted.

6.3 Customer GDPR delete (anonymize)

• DELETE /api/v1/loyalty/cards/customer/{customer_id} on admin router.
• New service method card_service.anonymize_cards_for_customer:
  • card.customer_id = NULL
  • Scrub PII fields, scrub notes if PII suspected
  • Keep transaction rows (aggregate data)
  • Audit log entry with admin ID

6.4 Bulk endpoints

• Merchant router (routes/api/merchant.py):
  • POST /merchants/loyalty/cards/bulk/deactivate — {card_ids, reason}
  • POST /merchants/loyalty/pins/bulk/assign — {store_id, pins}
  • POST /merchants/loyalty/cards/bulk/bonus — {card_ids | filter, points, reason}, fires notifications
• Admin "act on behalf" (routes/api/admin.py):
  • POST /admin/loyalty/merchants/{merchant_id}/cards/bulk/deactivate (etc.)
  • Shared service layer; route stamps acting_admin_id in audit log
• New loyalty_transactions.acting_admin_id column in loyalty_006.

6.5 Manual override: restore expired points

• POST /admin/loyalty/cards/{card_id}/restore_points with {points, reason}.
• Creates ADMIN_ADJUSTMENT transaction via existing points_service.adjust_points.
• UI button on admin card detail page.

---

Phase 7 — Advanced Analytics (2.5d)

7.1 Cohort retention

• New services/analytics_service.py (or extend program_service).
• Group cards by enrollment month, track % active (transaction in last 30d) per subsequent month.
• Endpoint /merchants/loyalty/analytics/cohorts returns matrix.
• Chart.js heatmap on analytics.html.

7.2 Simple churn rule

• "At risk" = inactive > 2× normal inter-transaction interval.
• Expose count + list.

7.3 Revenue attribution

• Join loyalty_transactions (where points_earned.order_reference set) with orders module.
• Loyalty-customer revenue vs non-loyalty per store per month.
• Cross-module rule: use orders module's query helper, do not reach into its tables directly.

---

Phase 8 — Tests, Docs, Observability (2d)

8.1 Coverage enforcement

• Loyalty CI job: pytest app/modules/loyalty/tests --cov=app/modules/loyalty --cov-fail-under=80.
• Audit current with --cov-report=term-missing, fill biggest gaps (notifications, batched expiration, rate-limited endpoints).

8.2 OpenAPI polish

• Audit all loyalty routes for summary, description, response_model, example payloads, tags=["Loyalty - Store"] etc.
• Export static loyalty-openapi.json under app/modules/loyalty/docs/.

8.3 Runbooks

Three new docs under docs/modules/loyalty/:

• runbook-wallet-certs.md — obtain, rotate, install, expiry monitoring, rollback.
• runbook-expiration-task.md — re-run for specific merchant, partial failure handling, link to 6.5 manual restore.
• runbook-wallet-sync.md — interpreting failed_card_ids, manual re-sync, common failures.

8.4 Monitoring & alerting

• Celery loyalty.expire_points not succeeded in 26h → page
• loyalty.sync_wallet_passes failure rate > 5% → warn
• Loyalty email failure rate > 1% → warn
• Google Wallet service account JSON cert expiry < 30d → warn
• Rate-limit 429s per store > 100/min → investigate
• Wire into app/core/observability.py (or whatever stack is present).

---

Phase 9 — Apple Wallet (post-launch, 3d)

Tracked separately, not blocking launch.

• 9.1 Apple Developer account + Pass Type ID + WWDR cert
• 9.2 Apple config wiring + all-or-nothing validators in config.py:33-37
• 9.3 Cert deployment under ~/apps/orion/apple-wallet/ (matching the Google secrets convention)
• 9.4 (already done in 1.2 — timing-safe compare)
• 9.5 UI feature flag flip + Apple button re-enabled
• 9.6 iOS Simulator + real device E2E test

---

Critical Path

Phase 0 (done) ──┬─► Phase 1 ──┬─► Phase 3 ──┐
                 ├─► Phase 2 ──┤             ├─► Phase 8 ──► LAUNCH
                 └─► Phase 5 ──┘             │
                                             │
   Phase 4, 6, 7 (parallelizable) ───────────┘

   Phase 9 — post-launch

Phases 4, 6, 7 can run in parallel with 2/3/5 if multiple developers are available.

Effort Summary

Phase	Days
0 — Decisions	done
1 — Config & security	2
2 — Notifications	4
3 — Task reliability	1.5
4 — A11y + CMS T&C	2
5 — Google Wallet hardening	1
6 — Admin / GDPR / bulk	3
7 — Analytics	2.5
8 — Tests / docs / observability	2
Launch total	~18 days sequential, ~10 with 2 parallel tracks
9 — Apple Wallet (post-launch)	3

---

Open Items Needing Sign-off

1. Rate limit caps — confirmed.
2. Email copywriting for the 7 templates × 4 locales (Phase 2.3) — flow: I draft EN, Samir reviews, then translate.
3. birth_date column — confirmed missing; addressed in Phase 1.4. No backfill needed (not yet live).