fix(celery): preload every module's models so mapper resolution succeeds
Some checks failed
Some checks failed
Celery workers were failing every task that touched the DB with
InvalidRequestError: "expression 'ContentPage' failed to locate a name".
Root cause: the worker process only loads task modules during startup,
not routes or services. Models reached only via SQLAlchemy
string-based relationships (e.g. Platform.relationship("ContentPage"))
were never imported, so the mapper couldn't resolve the name when the
first task tried to open a DB session. The FastAPI process avoids this
because api_router transitively imports the world; the worker doesn't.
Add _preload_all_module_models() in celery_config.py that walks the
module registry and importlib.import_module's each "app.modules.<code>.models"
package. Called at module import time, before any task runs.
Surfaced while finally getting loyalty.send_notification_email registered
on the worker — the task ran, hit the DB to load email settings, and
exploded on the unresolved Platform -> ContentPage relationship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -64,6 +64,44 @@ def get_all_task_modules() -> list[str]:
|
||||
return []
|
||||
|
||||
|
||||
def _preload_all_module_models() -> None:
|
||||
"""
|
||||
Import every module's models package up front.
|
||||
|
||||
The Celery worker process loads task modules but not routes/services,
|
||||
so models reached only via SQLAlchemy string-based relationships
|
||||
(e.g. Platform -> "ContentPage") never get imported. The first DB
|
||||
query then explodes with InvalidRequestError when the mapper tries
|
||||
to resolve the unknown name. The FastAPI process avoids this because
|
||||
api_router transitively imports everything; we replicate that effect
|
||||
here for the worker.
|
||||
"""
|
||||
import importlib
|
||||
|
||||
try:
|
||||
from app.modules.registry import MODULES
|
||||
except ImportError as e:
|
||||
logger.warning(f"Could not load module registry for model preload: {e}")
|
||||
return
|
||||
|
||||
loaded = 0
|
||||
for code in MODULES:
|
||||
module_path = f"app.modules.{code}.models"
|
||||
try:
|
||||
importlib.import_module(module_path)
|
||||
loaded += 1
|
||||
except ModuleNotFoundError:
|
||||
# Module doesn't expose a models package — fine, skip.
|
||||
continue
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed preloading {module_path}: {e}")
|
||||
|
||||
logger.info(f"Preloaded models for {loaded} modules (SQLAlchemy mapper resolution)")
|
||||
|
||||
|
||||
_preload_all_module_models()
|
||||
|
||||
|
||||
# Create Celery application
|
||||
celery_app = Celery(
|
||||
"orion",
|
||||
|
||||
Reference in New Issue
Block a user