feat: add Celery/Redis task queue with feature flag support
Migrate background tasks from FastAPI BackgroundTasks to Celery with Redis for persistent task queuing, retries, and scheduled jobs. Key changes: - Add Celery configuration with Redis broker/backend - Create task dispatcher with USE_CELERY feature flag for gradual rollout - Add Celery task wrappers for all background operations: - Marketplace imports - Letzshop historical imports - Product exports - Code quality scans - Test runs - Subscription scheduled tasks (via Celery Beat) - Add celery_task_id column to job tables for Flower integration - Add Flower dashboard link to admin background tasks page - Update docker-compose.yml with worker, beat, and flower services - Add Makefile targets: celery-worker, celery-beat, celery-dev, flower When USE_CELERY=false (default), system falls back to FastAPI BackgroundTasks for development without Redis dependency. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -13,6 +13,15 @@
|
||||
|
||||
{% block content %}
|
||||
{% call page_header_flex(title='Background Tasks', subtitle='Monitor running and completed background tasks') %}
|
||||
<!-- Flower Dashboard Link (Celery Monitoring) -->
|
||||
<a href="{{ config.FLOWER_URL | default('http://localhost:5555', true) }}"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
class="flex items-center px-4 py-2 text-sm font-medium leading-5 text-white transition-colors duration-150 bg-purple-600 border border-transparent rounded-lg active:bg-purple-600 hover:bg-purple-700 focus:outline-none focus:shadow-outline-purple mr-2"
|
||||
title="Open Flower dashboard for detailed Celery task monitoring">
|
||||
<span x-html="$icon('chart-bar', 'w-4 h-4 mr-2')"></span>
|
||||
Flower Dashboard
|
||||
</a>
|
||||
{{ refresh_button(variant='secondary') }}
|
||||
{% endcall %}
|
||||
|
||||
@@ -208,6 +217,7 @@
|
||||
<th class="px-4 py-3">Duration</th>
|
||||
<th class="px-4 py-3">Triggered By</th>
|
||||
<th class="px-4 py-3">Status</th>
|
||||
<th class="px-4 py-3">Celery</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody class="bg-white divide-y dark:divide-gray-700 dark:bg-gray-800">
|
||||
@@ -241,6 +251,21 @@
|
||||
x-text="task.status">
|
||||
</span>
|
||||
</td>
|
||||
<td class="px-4 py-3 text-sm">
|
||||
<template x-if="task.celery_task_id">
|
||||
<a :href="'{{ config.FLOWER_URL | default('http://localhost:5555', true) }}/task/' + task.celery_task_id"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
class="text-purple-600 hover:text-purple-800 dark:text-purple-400 dark:hover:text-purple-300 underline"
|
||||
title="View in Flower">
|
||||
<span x-html="$icon('external-link', 'w-4 h-4 inline')"></span>
|
||||
<span x-text="task.celery_task_id.substring(0, 8) + '...'"></span>
|
||||
</a>
|
||||
</template>
|
||||
<template x-if="!task.celery_task_id">
|
||||
<span class="text-gray-400 text-xs">-</span>
|
||||
</template>
|
||||
</td>
|
||||
</tr>
|
||||
</template>
|
||||
</tbody>
|
||||
|
||||
Reference in New Issue
Block a user