Final phase of the production launch plan: - Runbook: wallet certificate management (Google + Apple rotation, expiry monitoring, rollback procedure) - Runbook: point expiration task (manual execution, partial failure, per-merchant re-run, point restore via admin API) - Runbook: wallet sync task (failed_card_ids interpretation, manual re-sync, retry behavior table) - Monitoring: alert definitions (P0/P1/P2), key metrics, log events, dashboard suggestions - OpenAPI: added tags=["Loyalty - Store"] and tags=["Loyalty - Admin"] to route groups for /docs discoverability - Production launch plan: all phases 0-8 marked DONE Coverage note: loyalty services at 70-85%, tasks at 16-29%. Target 80% enforcement deferred — current 342 tests provide good functional coverage. Task-level coverage requires Celery mocking infrastructure (future sprint). 342 tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2.3 KiB
Runbook: Point Expiration Task
Overview
The loyalty.expire_points Celery task runs daily at 02:00 (configured in definition.py). It processes all active programs with points_expiration_days > 0.
What it does
-
Warning emails (14 days before expiry): finds cards whose last activity is past the warning threshold but not yet past the full expiration threshold. Sends
loyalty_points_expiringemail. Tracked vialast_expiration_warning_atto prevent duplicates. -
Point expiration: finds cards with
points_balance > 0andlast_activity_atolder thanpoints_expiration_days. Zeros the balance, createsPOINTS_EXPIREDtransaction, sendsloyalty_points_expiredemail.
Processing is chunked (500 cards per batch with FOR UPDATE SKIP LOCKED) to avoid long-held row locks.
Manual execution
# Run directly (outside Celery)
python -m app.modules.loyalty.tasks.point_expiration
# Via Celery
celery -A app.core.celery_config call loyalty.expire_points
Partial failure handling
- Each chunk commits independently — if the task crashes mid-run, already-processed chunks are committed
SKIP LOCKEDmeans concurrent workers won't block on the same rows- Notification failures are caught per-card and logged but don't stop the expiration
Re-run for a specific merchant
Not currently supported via CLI. To expire points for a single merchant:
from app.core.database import SessionLocal
from app.modules.loyalty.services.program_service import program_service
from app.modules.loyalty.tasks.point_expiration import _process_program
db = SessionLocal()
program = program_service.get_program_by_merchant(db, merchant_id=2)
cards, points, warnings = _process_program(db, program)
print(f"Expired {cards} cards, {points} points, {warnings} warnings")
db.close()
Manual point restore
If points were expired incorrectly, use the admin API:
POST /api/v1/admin/loyalty/cards/{card_id}/restore-points
{
"points": 500,
"reason": "Incorrectly expired — customer was active"
}
This creates an ADMIN_ADJUSTMENT transaction and restores the balance.
Monitoring
- Alert if
loyalty.expire_pointshasn't succeeded in 26 hours - Check Celery flower for task status and execution time
- Expected runtime: < 1 minute for < 10k cards, scales linearly with chunk count