docs: migrate module documentation to single source of truth

Move 39 documentation files from top-level docs/ into each module's
docs/ folder, accessible via symlinks from docs/modules/. Create
data-model.md files for 10 modules with full schema documentation.
Replace originals with redirect stubs. Remove empty guide stubs.

Modules migrated: tenancy, billing, loyalty, marketplace, orders,
messaging, cms, catalog, inventory, hosting, prospecting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-08 23:38:37 +01:00
parent 2287f4597d
commit f141cc4e6a
140 changed files with 19921 additions and 17723 deletions

View File

@@ -1,601 +1,3 @@
# Letzshop Order Import - Improvement Plan
## Current Status (2025-12-17)
### Schema Discovery Complete ✅
After running GraphQL introspection queries, we have identified all available fields.
### Available Fields Summary
| Data | GraphQL Path | Notes |
|------|-------------|-------|
| **EAN/GTIN** | `variant.tradeId.number` | The product barcode |
| **Trade ID Type** | `variant.tradeId.parser` | Format: gtin13, gtin14, gtin12, gtin8, isbn13, isbn10 |
| **Brand Name** | `product._brand { ... on Brand { name } }` | Union type requires fragment |
| **MPN** | `variant.mpn` | Manufacturer Part Number |
| **SKU** | `variant.sku` | Merchant's internal SKU |
| **Product Name** | `variant.product.name { en, fr, de }` | Translated names |
| **Price** | `variant.price` | Unit price |
| **Quantity** | Count of `inventoryUnits` | Each unit = 1 item |
| **Customer Language** | `order.locale` | Language for invoice (en, fr, de) |
| **Customer Country** | `order.shipAddress.country` | Country object |
### Key Findings
1. **EAN lives in `tradeId`** - Not a direct field on Variant, but nested in `tradeId.number`
2. **TradeIdParser enum values**: `gtin14`, `gtin13` (EAN-13), `gtin12` (UPC), `gtin8`, `isbn13`, `isbn10`
3. **Brand is a Union** - Must use `... on Brand { name }` fragment, also handles `BrandUnknown`
4. **No quantity field** - Each InventoryUnit represents 1 item; count units to get quantity
## Updated GraphQL Query
```graphql
query {
shipments(state: unconfirmed) {
nodes {
id
number
state
order {
id
number
email
total
completedAt
locale
shipAddress {
firstName
lastName
merchant
streetName
streetNumber
city
zipCode
phone
country {
name
iso
}
}
billAddress {
firstName
lastName
merchant
streetName
streetNumber
city
zipCode
phone
country {
name
iso
}
}
}
inventoryUnits {
id
state
variant {
id
sku
mpn
price
tradeId {
number
parser
}
product {
name { en fr de }
_brand {
... on Brand { name }
}
}
}
}
tracking {
code
provider
}
}
}
}
```
## Implementation Steps
### Step 1: Update GraphQL Queries ✅ DONE
Update in `app/services/letzshop/client_service.py`:
- `QUERY_SHIPMENTS_UNCONFIRMED`
- `QUERY_SHIPMENTS_CONFIRMED`
- `QUERY_SHIPMENT_BY_ID`
- `QUERY_SHIPMENTS_PAGINATED_TEMPLATE` ✅ (new - for historical import)
### Step 2: Update Order Service ✅ DONE
Updated `create_order()` and `update_order_from_shipment()` in `app/services/letzshop/order_service.py`:
- Extract `tradeId.number` as EAN ✅
- Store MPN if available ✅
- Store `locale` for invoice language ✅
- Store shipping/billing country ISO codes ✅
- Enrich inventory_units with EAN, MPN, SKU, product_name ✅
**Database changes:**
- Added `customer_locale` column to `LetzshopOrder`
- Added `shipping_country_iso` column to `LetzshopOrder`
- Added `billing_country_iso` column to `LetzshopOrder`
- Migration: `a9a86cef6cca_add_letzshop_order_locale_and_country_.py`
### Step 3: Match Products by EAN ✅ DONE (Basic)
When importing orders:
- Use `tradeId.number` (EAN) to find matching local product ✅
- `_match_eans_to_products()` function added ✅
- Returns match statistics (products_matched, products_not_found) ✅
**TODO for later:**
- ⬜ Decrease stock for matched product (needs careful implementation)
- ⬜ Show match status in order detail view
### Step 4: Update Frontend ✅ DONE (Historical Import)
- Added "Import History" button to Orders tab ✅
- Added historical import result display ✅
- Added `importHistoricalOrders()` JavaScript function ✅
**TODO for later:**
- ⬜ Show product details in individual order view (EAN, MPN, SKU, match status)
### Step 5: Historical Import Feature ✅ DONE
Import all confirmed orders for:
- Sales analytics (how many products sold)
- Customer records
- Historical data
**Implementation:**
- Pagination support with `get_all_shipments_paginated()`
- Deduplication by `letzshop_order_id`
- EAN matching during import ✅
- Progress callback for large imports ✅
**Endpoints Added:**
- `POST /api/v1/admin/letzshop/stores/{id}/import-history` - Import historical orders
- `GET /api/v1/admin/letzshop/stores/{id}/import-summary` - Get import statistics
**Frontend:**
- "Import History" button in Orders tab
- Result display showing imported/updated/skipped counts
**Tests:**
- Unit tests in `tests/unit/services/test_letzshop_service.py`
- Manual test script `scripts/test_historical_import.py`
## Test Results (2025-12-17)
### Query Test: PASSED ✅
```
Example shipment:
Shipment #: H43748338602
Order #: R702236251
Customer: miriana.leal@letzshop.lu
Locale: fr <<<< LANGUAGE
Total: 32.88 EUR
Ship to: Miriana Leal Ferreira
City: 1468 Luxembourg
Country: LU
Items (1):
- Pocket POP! Keychains: Marvel Avengers Infinity War - Iron Spider
SKU: 00889698273022
MPN: None
EAN: 00889698273022 (gtin14) <<<< BARCODE
Price: 5.88 EUR
```
### Known Issues / Letzshop API Bugs
#### Bug 1: `_brand` field causes server error
- **Error**: `NoMethodError: undefined method 'demodulize' for nil`
- **Trigger**: Querying `_brand { ... on Brand { name } }` on some products
- **Workaround**: Removed `_brand` from queries
- **Status**: To report to Letzshop
#### Bug 2: `tracking` field causes server error (ALL queries)
- **Error**: `NoMethodError: undefined method 'demodulize' for nil`
- **Trigger**: Including `tracking { code provider }` in ANY shipment query
- **Tested and FAILS on**:
- Paginated queries: `shipments(state: confirmed, first: 10) { nodes { tracking { code provider } } }`
- Non-paginated queries: `shipments(state: confirmed) { nodes { tracking { code provider } } }`
- Single shipment queries: Also fails (Letzshop doesn't support `node(id:)` interface)
- **Impact**: Cannot retrieve tracking numbers and carrier info at all
- **Workaround**: None - tracking info is currently unavailable via API
- **Status**: **CRITICAL - Must report to Letzshop**
- **Date discovered**: 2025-12-17
**Note**: Letzshop automatically creates tracking when orders are confirmed. The carrier picks up parcels. But we cannot retrieve this info due to the API bug.
#### Bug 3: Product table missing `gtin` field ✅ FIXED
- **Error**: `type object 'Product' has no attribute 'gtin'`
- **Cause**: `gtin` field only existed on `MarketplaceProduct` (staging table), not on `Product` (operational table)
- **Date discovered**: 2025-12-17
- **Date fixed**: 2025-12-18
- **Fix applied**:
1. Migration `cb88bc9b5f86_add_gtin_columns_to_product_table.py` adds:
- `gtin` (String(50)) - the barcode number
- `gtin_type` (String(20)) - the format type (gtin13, gtin14, etc.)
- Indexes: `idx_product_gtin`, `idx_product_store_gtin`
2. `models/database/product.py` updated with new columns
3. `_match_eans_to_products()` now queries `Product.gtin`
4. `get_products_by_eans()` now returns products by EAN lookup
- **Status**: COMPLETE
**GTIN Types Reference:**
| Type | Digits | Common Name | Region/Use |
|------|--------|-------------|------------|
| gtin13 | 13 | EAN-13 | Europe (most common) |
| gtin12 | 12 | UPC-A | North America |
| gtin14 | 14 | ITF-14 | Logistics/cases |
| gtin8 | 8 | EAN-8 | Small items |
| isbn13 | 13 | ISBN-13 | Books |
| isbn10 | 10 | ISBN-10 | Books (legacy) |
Letzshop API returns:
- `tradeId.number` → store in `gtin`
- `tradeId.parser` → store in `gtin_type`
**Letzshop Shipment States (from official docs):**
| Letzshop State | Our sync_status | Description |
|----------------|-----------------|-------------|
| `unconfirmed` | `pending` | New order, needs store confirmation |
| `confirmed` | `confirmed` | At least one product confirmed |
| `declined` | `rejected` | All products rejected |
Note: There is no "shipped" state in Letzshop. Shipping is tracked via the `tracking` field (code + provider), not as a state change.
---
## Historical Confirmed Orders Import
### Purpose
Import all historical confirmed orders from Letzshop to:
1. **Sales Analytics** - Track total products sold, revenue by product/category
2. **Customer Records** - Build customer database with order history
3. **Inventory Reconciliation** - Understand what was sold to reconcile stock
### Implementation Plan
#### 1. Add "Import Historical Orders" Feature
- New endpoint: `POST /api/v1/admin/letzshop/stores/{id}/import-history`
- Parameters:
- `state`: confirmed/shipped/delivered (default: confirmed)
- `since`: Optional date filter (import orders after this date)
- `dry_run`: Preview without saving
#### 2. Pagination Support
Letzshop likely returns paginated results. Need to handle:
```graphql
query {
shipments(state: confirmed, first: 50, after: $cursor) {
pageInfo {
hasNextPage
endCursor
}
nodes { ... }
}
}
```
#### 3. Deduplication
- Check if order already exists by `letzshop_order_id` before inserting
- Update existing orders if data changed
#### 4. EAN Matching & Stock Adjustment
When importing historical orders:
- Match `tradeId.number` (EAN) to local products
- Calculate total quantity sold per product
- Option to adjust inventory based on historical sales
#### 5. Customer Database
Extract and store customer data:
- Email (unique identifier)
- Name (from shipping address)
- Preferred language (from `order.locale`)
- Order count, total spent
#### 6. UI: Historical Import Page
Admin interface to:
- Trigger historical import
- View import progress
- See summary: X orders imported, Y customers added, Z products matched
### Data Flow
```
Letzshop API (confirmed shipments)
┌───────────────────────┐
│ Import Service │
│ - Fetch all pages │
│ - Deduplicate │
│ - Match EAN to SKU │
└───────────────────────┘
┌───────────────────────┐
│ Database │
│ - letzshop_orders │
│ - customers │
│ - inventory updates │
└───────────────────────┘
┌───────────────────────┐
│ Analytics Dashboard │
│ - Sales by product │
│ - Revenue over time │
│ - Customer insights │
└───────────────────────┘
```
---
## Schema Reference
### Variant Fields
```
baseAmount: String
baseAmountProduct: String
baseUnit: String
countOnHand: Int
id: ID!
images: [Image]!
inPresale: Boolean!
isMaster: Boolean!
mpn: String
price: Float!
priceCrossed: Float
pricePerUnit: Float
product: Product!
properties: [Property]!
releaseAt: Iso8601Time
sku: String
tradeId: TradeId
uniqueId: String
url: String!
```
### TradeId Fields
```
isRestricted: Boolean
number: String! # <-- THE EAN/GTIN
parser: TradeIdParser! # <-- Format identifier
```
### TradeIdParser Enum
```
gtin14 - GTIN-14 (14 digits)
gtin13 - GTIN-13 / EAN-13 (13 digits, most common in Europe)
gtin12 - GTIN-12 / UPC-A (12 digits, common in North America)
gtin8 - GTIN-8 / EAN-8 (8 digits)
isbn13 - ISBN-13 (books)
isbn10 - ISBN-10 (books)
```
### Brand (via BrandUnion)
```
BrandUnion = Brand | BrandUnknown
Brand fields:
id: ID!
name: String!
identifier: String!
descriptor: String
logo: Attachment
url: String!
```
### InventoryUnit Fields
```
id: ID!
price: Float!
state: String!
taxRate: Float!
uniqueId: String
variant: Variant
```
## Reference: Letzshop Frontend Shows
From the Letzshop merchant interface:
- Order number: R532332163
- Shipment number: H74683403433
- Product: "Pop! Rocks: DJ Khaled - DJ Khaled #237"
- Brand: Funko
- Internal merchant number: MH-FU-56757
- Price: 16,95 €
- Quantity: 1
- Shipping: 2,99 €
- Total: 19,94 €
---
## Completed (2025-12-18)
### Order Stats Fix ✅
- **Issue**: Order status cards (Pending, Confirmed, etc.) were showing incorrect counts
- **Cause**: Stats were calculated client-side from only the visible page of orders
- **Fix**:
1. Added `get_order_stats()` method to `LetzshopOrderService`
2. Added `LetzshopOrderStats` schema with pending/confirmed/rejected/shipped counts
3. API now returns `stats` field with counts for ALL orders
4. JavaScript uses server-side stats instead of client-side calculation
- **Status**: COMPLETE
### Tracking Investigation ✅
- **Issue**: Letzshop API bug prevents querying tracking field
- **Added**: `--tracking` option to `letzshop_introspect.py` to investigate workarounds
- **Findings**: Bug is on Letzshop's side, no client-side workaround possible
- **Recommendation**: Store tracking info locally after setting via API
### Item-Level Confirmation ✅
- **Issue**: Orders were being confirmed/declined at order level, but Letzshop requires item-level actions
- **Letzshop Model**:
- Each `inventoryUnit` must be confirmed/declined individually via `confirmInventoryUnits` mutation
- `isAvailable: true` = confirmed, `isAvailable: false` = declined
- Inventory unit states: `unconfirmed``confirmed_available` / `confirmed_unavailable` / `returned`
- Shipment states derived from items: `unconfirmed` / `confirmed` / `declined`
- Partial confirmation allowed (some items confirmed, some declined)
- **Fix**:
1. Order detail modal now shows each item with product details (name, EAN, SKU, MPN, price)
2. Per-item confirm/decline buttons for pending items
3. Admin API endpoints for single-item and bulk operations:
- `POST /stores/{id}/orders/{id}/items/{id}/confirm`
- `POST /stores/{id}/orders/{id}/items/{id}/decline`
4. Order status automatically updates based on item states:
- All items declined → order status = "declined"
- Any item confirmed → order status = "confirmed"
### Terminology Update ✅
- Changed "Rejected" to "Declined" throughout UI to match Letzshop terminology
- Internal `sync_status` value remains "rejected" for backwards compatibility
- Filter dropdown, status badges, and action buttons now use "Declined"
- Added "Declined" stats card to orders dashboard
### Historical Import: Multiple Phases ✅
- Historical import now fetches both `confirmed` AND `unconfirmed` (pending) shipments
- Note: "declined" is NOT a valid Letzshop shipment state - declined items are tracked at inventory unit level
- Combined stats shown in import result
---
## Completed (2025-12-19)
### Historical Import Progress Bar ✅
Real-time progress feedback for historical import using background tasks with database polling.
**Implementation:**
- Background task (`app/tasks/letzshop_tasks.py`) runs historical import asynchronously
- Progress stored in `LetzshopHistoricalImportJob` database model
- Frontend polls status endpoint every 2 seconds
- Two-phase import: confirmed orders first, then unconfirmed (pending) orders
**Backend:**
- `LetzshopHistoricalImportJob` model tracks: status, current_phase, current_page, shipments_fetched, orders_processed, confirmed_stats, declined_stats
- `POST /stores/{id}/import-history` starts background job, returns job_id immediately
- `GET /stores/{id}/import-history/{job_id}/status` returns current progress
**Frontend:**
- Progress panel shows: phase (confirmed/pending), page number, shipments fetched, orders processed
- Disabled "Import History" button during import with spinner
- Final result summary shows combined stats from both phases
**Key Discovery:**
- Letzshop API has NO "declined" shipment state
- Valid states: `awaiting_order_completion`, `unconfirmed`, `completed`, `accepted`, `confirmed`
- Declined items are tracked at inventory unit level with state `confirmed_unavailable`
### Filter for Declined Items ✅
Added ability to filter orders that have at least one declined/unavailable item.
**Backend:**
- `list_orders()` accepts `has_declined_items: bool` parameter
- Uses JSON string contains check: `inventory_units.cast(String).contains("confirmed_unavailable")`
- `get_order_stats()` returns `has_declined_items` count
**Frontend:**
- "Has Declined Items" toggle button in filters section
- Shows count badge when there are orders with declined items
- Toggles between all orders and filtered view
**API:**
- `GET /stores/{id}/orders?has_declined_items=true` - filter orders
### Order Date Display ✅
Orders now display the actual order date from Letzshop instead of the import date.
**Database:**
- Added `order_date` column to `LetzshopOrder` model
- Migration: `2362c2723a93_add_order_date_to_letzshop_orders.py`
**Backend:**
- `create_order()` extracts `completedAt` from Letzshop order data and stores as `order_date`
- `update_order_from_shipment()` populates `order_date` if not already set
- Date parsing handles ISO format with timezone (including `Z` suffix)
**Frontend:**
- Order table displays `order_date` with fallback to `created_at` for legacy orders
- Format: localized date/time string
**Note:** Existing orders imported before this change will continue showing `created_at` until re-imported via historical import.
### Search Filter ✅
Added search functionality to find orders by order number, customer name, or email.
**Backend:**
- `list_orders()` accepts `search: str` parameter
- Uses ILIKE for case-insensitive partial matching across:
- `letzshop_order_number`
- `customer_name`
- `customer_email`
**Frontend:**
- Search input field with magnifying glass icon
- Debounced input (300ms) to avoid excessive API calls
- Clear button to reset search
- Resets to page 1 when search changes
**API:**
- `GET /stores/{id}/orders?search=query` - search orders
---
## Next Steps (TODO)
### Priority 1: Stock Management
When an order is confirmed/imported:
1. Match EAN from order to local product catalog
2. Decrease stock quantity for matched products
3. Handle cases where product not found (alert/log)
**Considerations:**
- Should stock decrease happen on import or only on confirmation?
- Need rollback mechanism if order is rejected
- Handle partial matches (some items found, some not)
### Priority 2: Invoice Generation
Use `customer_locale` to generate invoices in customer's language:
- Invoice template with multi-language support
- PDF generation
### Priority 3: Analytics Dashboard
Build sales analytics based on imported orders:
- Sales by product
- Sales by time period
- Customer statistics
- Revenue breakdown
---
## Files Modified (2025-12-16 to 2025-12-19)
| File | Changes |
|------|---------|
| `app/services/letzshop/client_service.py` | Added paginated query, updated all queries with EAN/locale/country |
| `app/services/letzshop/order_service.py` | Historical import, EAN matching, order stats, has_declined_items filter, search filter, order_date extraction |
| `models/database/letzshop.py` | Added locale/country/order_date columns, `LetzshopHistoricalImportJob` model |
| `models/database/product.py` | Added `gtin` and `gtin_type` columns for EAN matching |
| `models/schema/letzshop.py` | Added `LetzshopOrderStats`, `LetzshopHistoricalImportJobResponse`, `order_date` field |
| `app/api/v1/admin/letzshop.py` | Import-history endpoints, has_declined_items filter, search filter, order_date in response |
| `app/tasks/letzshop_tasks.py` | **NEW** - Background task for historical import with progress tracking |
| `app/templates/admin/partials/letzshop-orders-tab.html` | Import History button, progress panel, declined items filter, search input, order_date display |
| `static/admin/js/marketplace-letzshop.js` | Historical import polling, progress display, declined items filter, search functionality |
| `tests/unit/services/test_letzshop_service.py` | Added tests for new functionality |
| `scripts/test_historical_import.py` | Manual test script for historical import |
| `scripts/debug_historical_import.py` | **NEW** - Debug script for shipment states and declined items |
| `scripts/letzshop_introspect.py` | GraphQL schema introspection tool, tracking workaround tests |
| `alembic/versions/a9a86cef6cca_*.py` | Migration for locale/country columns |
| `alembic/versions/cb88bc9b5f86_*.py` | Migration for gtin columns on Product table |
| `alembic/versions/*_add_historical_import_jobs.py` | **NEW** - Migration for LetzshopHistoricalImportJob table |
| `alembic/versions/2362c2723a93_*.py` | **NEW** - Migration for order_date column |
This document has moved to the marketplace module docs: [Import Improvements](../modules/marketplace/import-improvements.md)