fix(celery): preload every module's models so mapper resolution succeeds
Some checks failed
Some checks failed
Celery workers were failing every task that touched the DB with
InvalidRequestError: "expression 'ContentPage' failed to locate a name".
Root cause: the worker process only loads task modules during startup,
not routes or services. Models reached only via SQLAlchemy
string-based relationships (e.g. Platform.relationship("ContentPage"))
were never imported, so the mapper couldn't resolve the name when the
first task tried to open a DB session. The FastAPI process avoids this
because api_router transitively imports the world; the worker doesn't.
Add _preload_all_module_models() in celery_config.py that walks the
module registry and importlib.import_module's each "app.modules.<code>.models"
package. Called at module import time, before any task runs.
Surfaced while finally getting loyalty.send_notification_email registered
on the worker — the task ran, hit the DB to load email settings, and
exploded on the unresolved Platform -> ContentPage relationship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -64,6 +64,44 @@ def get_all_task_modules() -> list[str]:
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _preload_all_module_models() -> None:
|
||||||
|
"""
|
||||||
|
Import every module's models package up front.
|
||||||
|
|
||||||
|
The Celery worker process loads task modules but not routes/services,
|
||||||
|
so models reached only via SQLAlchemy string-based relationships
|
||||||
|
(e.g. Platform -> "ContentPage") never get imported. The first DB
|
||||||
|
query then explodes with InvalidRequestError when the mapper tries
|
||||||
|
to resolve the unknown name. The FastAPI process avoids this because
|
||||||
|
api_router transitively imports everything; we replicate that effect
|
||||||
|
here for the worker.
|
||||||
|
"""
|
||||||
|
import importlib
|
||||||
|
|
||||||
|
try:
|
||||||
|
from app.modules.registry import MODULES
|
||||||
|
except ImportError as e:
|
||||||
|
logger.warning(f"Could not load module registry for model preload: {e}")
|
||||||
|
return
|
||||||
|
|
||||||
|
loaded = 0
|
||||||
|
for code in MODULES:
|
||||||
|
module_path = f"app.modules.{code}.models"
|
||||||
|
try:
|
||||||
|
importlib.import_module(module_path)
|
||||||
|
loaded += 1
|
||||||
|
except ModuleNotFoundError:
|
||||||
|
# Module doesn't expose a models package — fine, skip.
|
||||||
|
continue
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Failed preloading {module_path}: {e}")
|
||||||
|
|
||||||
|
logger.info(f"Preloaded models for {loaded} modules (SQLAlchemy mapper resolution)")
|
||||||
|
|
||||||
|
|
||||||
|
_preload_all_module_models()
|
||||||
|
|
||||||
|
|
||||||
# Create Celery application
|
# Create Celery application
|
||||||
celery_app = Celery(
|
celery_app = Celery(
|
||||||
"orion",
|
"orion",
|
||||||
|
|||||||
Reference in New Issue
Block a user