orion/docs/implementation/letzshop-order-import-improvements.md

Letzshop Order Import - Improvement Plan

Current Status (2025-12-17)

Schema Discovery Complete ✅

After running GraphQL introspection queries, we have identified all available fields.

Available Fields Summary

Data	GraphQL Path	Notes
EAN/GTIN	variant.tradeId.number	The product barcode
Trade ID Type	variant.tradeId.parser	Format: gtin13, gtin14, gtin12, gtin8, isbn13, isbn10
Brand Name	product._brand { ... on Brand { name } }	Union type requires fragment
MPN	variant.mpn	Manufacturer Part Number
SKU	variant.sku	Merchant's internal SKU
Product Name	variant.product.name { en, fr, de }	Translated names
Price	variant.price	Unit price
Quantity	Count of inventoryUnits	Each unit = 1 item
Customer Language	order.locale	Language for invoice (en, fr, de)
Customer Country	order.shipAddress.country	Country object

Key Findings

1. EAN lives in tradeId - Not a direct field on Variant, but nested in tradeId.number
2. TradeIdParser enum values: gtin14, gtin13 (EAN-13), gtin12 (UPC), gtin8, isbn13, isbn10
3. Brand is a Union - Must use ... on Brand { name } fragment, also handles BrandUnknown
4. No quantity field - Each InventoryUnit represents 1 item; count units to get quantity

Updated GraphQL Query

query {
  shipments(state: unconfirmed) {
    nodes {
      id
      number
      state
      order {
        id
        number
        email
        total
        completedAt
        locale
        shipAddress {
          firstName
          lastName
          company
          streetName
          streetNumber
          city
          zipCode
          phone
          country {
            name
            iso
          }
        }
        billAddress {
          firstName
          lastName
          company
          streetName
          streetNumber
          city
          zipCode
          phone
          country {
            name
            iso
          }
        }
      }
      inventoryUnits {
        id
        state
        variant {
          id
          sku
          mpn
          price
          tradeId {
            number
            parser
          }
          product {
            name { en fr de }
            _brand {
              ... on Brand { name }
            }
          }
        }
      }
      tracking {
        code
        provider
      }
    }
  }
}

Implementation Steps

Step 1: Update GraphQL Queries ✅ DONE

Update in app/services/letzshop/client_service.py:

• QUERY_SHIPMENTS_UNCONFIRMED ✅
• QUERY_SHIPMENTS_CONFIRMED ✅
• QUERY_SHIPMENT_BY_ID ✅
• QUERY_SHIPMENTS_PAGINATED_TEMPLATE ✅ (new - for historical import)

Step 2: Update Order Service ✅ DONE

Updated create_order() and update_order_from_shipment() in app/services/letzshop/order_service.py:

• Extract tradeId.number as EAN ✅
• Store MPN if available ✅
• Store locale for invoice language ✅
• Store shipping/billing country ISO codes ✅
• Enrich inventory_units with EAN, MPN, SKU, product_name ✅

Database changes:

• Added customer_locale column to LetzshopOrder
• Added shipping_country_iso column to LetzshopOrder
• Added billing_country_iso column to LetzshopOrder
• Migration: a9a86cef6cca_add_letzshop_order_locale_and_country_.py

Step 3: Match Products by EAN ✅ DONE (Basic)

When importing orders:

• Use tradeId.number (EAN) to find matching local product ✅
• _match_eans_to_products() function added ✅
• Returns match statistics (products_matched, products_not_found) ✅

TODO for later:

• ⬜ Decrease stock for matched product (needs careful implementation)
• ⬜ Show match status in order detail view

Step 4: Update Frontend ✅ DONE (Historical Import)

• Added "Import History" button to Orders tab ✅
• Added historical import result display ✅
• Added importHistoricalOrders() JavaScript function ✅

TODO for later:

• ⬜ Show product details in individual order view (EAN, MPN, SKU, match status)

Step 5: Historical Import Feature ✅ DONE

Import all confirmed orders for:

• Sales analytics (how many products sold)
• Customer records
• Historical data

Implementation:

• Pagination support with get_all_shipments_paginated() ✅
• Deduplication by letzshop_order_id ✅
• EAN matching during import ✅
• Progress callback for large imports ✅

Endpoints Added:

• POST /api/v1/admin/letzshop/vendors/{id}/import-history - Import historical orders
• GET /api/v1/admin/letzshop/vendors/{id}/import-summary - Get import statistics

Frontend:

• "Import History" button in Orders tab
• Result display showing imported/updated/skipped counts

Tests:

• Unit tests in tests/unit/services/test_letzshop_service.py ✅
• Manual test script scripts/test_historical_import.py ✅

Test Results (2025-12-17)

Query Test: PASSED ✅

Example shipment:
  Shipment #: H43748338602
  Order #: R702236251
  Customer: miriana.leal@letzshop.lu
  Locale: fr <<<< LANGUAGE
  Total: 32.88 EUR

  Ship to: Miriana Leal Ferreira
  City: 1468 Luxembourg
  Country: LU

  Items (1):
    - Pocket POP! Keychains: Marvel Avengers Infinity War - Iron Spider
      SKU: 00889698273022
      MPN: None
      EAN: 00889698273022 (gtin14) <<<< BARCODE
      Price: 5.88 EUR

Known Issues / Letzshop API Bugs

Bug 1: _brand field causes server error

• Error: NoMethodError: undefined method 'demodulize' for nil
• Trigger: Querying _brand { ... on Brand { name } } on some products
• Workaround: Removed _brand from queries
• Status: To report to Letzshop

Bug 2: tracking field causes server error (ALL queries)

• Error: NoMethodError: undefined method 'demodulize' for nil
• Trigger: Including tracking { code provider } in ANY shipment query
• Tested and FAILS on:
  • Paginated queries: shipments(state: confirmed, first: 10) { nodes { tracking { code provider } } }
  • Non-paginated queries: shipments(state: confirmed) { nodes { tracking { code provider } } }
  • Single shipment queries: Also fails (Letzshop doesn't support node(id:) interface)
• Impact: Cannot retrieve tracking numbers and carrier info at all
• Workaround: None - tracking info is currently unavailable via API
• Status: CRITICAL - Must report to Letzshop
• Date discovered: 2025-12-17

Note: Letzshop automatically creates tracking when orders are confirmed. The carrier picks up parcels. But we cannot retrieve this info due to the API bug.

Bug 3: Product table missing gtin field ✅ FIXED

• Error: type object 'Product' has no attribute 'gtin'
• Cause: gtin field only existed on MarketplaceProduct (staging table), not on Product (operational table)
• Date discovered: 2025-12-17
• Date fixed: 2025-12-18
• Fix applied:
  1. Migration cb88bc9b5f86_add_gtin_columns_to_product_table.py adds:
     • gtin (String(50)) - the barcode number
     • gtin_type (String(20)) - the format type (gtin13, gtin14, etc.)
     • Indexes: idx_product_gtin, idx_product_vendor_gtin
  2. models/database/product.py updated with new columns
  3. _match_eans_to_products() now queries Product.gtin
  4. get_products_by_eans() now returns products by EAN lookup
• Status: COMPLETE

GTIN Types Reference:

Type	Digits	Common Name	Region/Use
gtin13	13	EAN-13	Europe (most common)
gtin12	12	UPC-A	North America
gtin14	14	ITF-14	Logistics/cases
gtin8	8	EAN-8	Small items
isbn13	13	ISBN-13	Books
isbn10	10	ISBN-10	Books (legacy)

Letzshop API returns:

• tradeId.number → store in gtin
• tradeId.parser → store in gtin_type

Letzshop Shipment States (from official docs):

Letzshop State	Our sync_status	Description
unconfirmed	pending	New order, needs vendor confirmation
confirmed	confirmed	At least one product confirmed
declined	rejected	All products rejected

Note: There is no "shipped" state in Letzshop. Shipping is tracked via the tracking field (code + provider), not as a state change.

---

Historical Confirmed Orders Import

Purpose

Import all historical confirmed orders from Letzshop to:

1. Sales Analytics - Track total products sold, revenue by product/category
2. Customer Records - Build customer database with order history
3. Inventory Reconciliation - Understand what was sold to reconcile stock

Implementation Plan

1. Add "Import Historical Orders" Feature

• New endpoint: POST /api/v1/admin/letzshop/vendors/{id}/import-history
• Parameters:
  • state: confirmed/shipped/delivered (default: confirmed)
  • since: Optional date filter (import orders after this date)
  • dry_run: Preview without saving

2. Pagination Support

Letzshop likely returns paginated results. Need to handle:

query {
  shipments(state: confirmed, first: 50, after: $cursor) {
    pageInfo {
      hasNextPage
      endCursor
    }
    nodes { ... }
  }
}

3. Deduplication

• Check if order already exists by letzshop_order_id before inserting
• Update existing orders if data changed

4. EAN Matching & Stock Adjustment

When importing historical orders:

• Match tradeId.number (EAN) to local products
• Calculate total quantity sold per product
• Option to adjust inventory based on historical sales

5. Customer Database

Extract and store customer data:

• Email (unique identifier)
• Name (from shipping address)
• Preferred language (from order.locale)
• Order count, total spent

6. UI: Historical Import Page

Admin interface to:

• Trigger historical import
• View import progress
• See summary: X orders imported, Y customers added, Z products matched

Data Flow

Letzshop API (confirmed shipments)
        │
        ▼
┌───────────────────────┐
│  Import Service       │
│  - Fetch all pages    │
│  - Deduplicate        │
│  - Match EAN to SKU   │
└───────────────────────┘
        │
        ▼
┌───────────────────────┐
│  Database             │
│  - letzshop_orders    │
│  - customers          │
│  - inventory updates  │
└───────────────────────┘
        │
        ▼
┌───────────────────────┐
│  Analytics Dashboard  │
│  - Sales by product   │
│  - Revenue over time  │
│  - Customer insights  │
└───────────────────────┘

---

Schema Reference

Variant Fields

baseAmount: String
baseAmountProduct: String
baseUnit: String
countOnHand: Int
id: ID!
images: [Image]!
inPresale: Boolean!
isMaster: Boolean!
mpn: String
price: Float!
priceCrossed: Float
pricePerUnit: Float
product: Product!
properties: [Property]!
releaseAt: Iso8601Time
sku: String
tradeId: TradeId
uniqueId: String
url: String!

TradeId Fields

isRestricted: Boolean
number: String!        # <-- THE EAN/GTIN
parser: TradeIdParser! # <-- Format identifier

TradeIdParser Enum

gtin14  - GTIN-14 (14 digits)
gtin13  - GTIN-13 / EAN-13 (13 digits, most common in Europe)
gtin12  - GTIN-12 / UPC-A (12 digits, common in North America)
gtin8   - GTIN-8 / EAN-8 (8 digits)
isbn13  - ISBN-13 (books)
isbn10  - ISBN-10 (books)

Brand (via BrandUnion)

BrandUnion = Brand | BrandUnknown

Brand fields:
  id: ID!
  name: String!
  identifier: String!
  descriptor: String
  logo: Attachment
  url: String!

InventoryUnit Fields

id: ID!
price: Float!
state: String!
taxRate: Float!
uniqueId: String
variant: Variant

Reference: Letzshop Frontend Shows

From the Letzshop merchant interface:

• Order number: R532332163
• Shipment number: H74683403433
• Product: "Pop! Rocks: DJ Khaled - DJ Khaled #237"
• Brand: Funko
• Internal merchant number: MH-FU-56757
• Price: 16,95 €
• Quantity: 1
• Shipping: 2,99 €
• Total: 19,94 €

---

Completed (2025-12-18)

Order Stats Fix ✅

• Issue: Order status cards (Pending, Confirmed, etc.) were showing incorrect counts
• Cause: Stats were calculated client-side from only the visible page of orders
• Fix:
  1. Added get_order_stats() method to LetzshopOrderService
  2. Added LetzshopOrderStats schema with pending/confirmed/rejected/shipped counts
  3. API now returns stats field with counts for ALL orders
  4. JavaScript uses server-side stats instead of client-side calculation
• Status: COMPLETE

Tracking Investigation ✅

• Issue: Letzshop API bug prevents querying tracking field
• Added: --tracking option to letzshop_introspect.py to investigate workarounds
• Findings: Bug is on Letzshop's side, no client-side workaround possible
• Recommendation: Store tracking info locally after setting via API

---

Next Steps (TODO)

Priority 1: Historical Import Progress Bar

Add real-time progress feedback for historical import (currently no visibility into import progress).

Requirements:

• Show progress indicator while import is running
• Display current page being fetched (e.g., "Fetching page 3 of 12...")
• Show running count of orders imported/updated
• Prevent user from thinking the process is stuck

Implementation options:

1. Polling approach: Frontend polls a status endpoint every few seconds
2. Server-Sent Events (SSE): Real-time updates pushed to frontend
3. WebSocket: Bi-directional real-time communication

Backend changes needed:

• Store import progress in database or cache (Redis)
• Add endpoint GET /api/v1/admin/letzshop/vendors/{id}/import-progress
• Update import_historical_shipments() to report progress

Frontend changes needed:

• Progress bar component in Orders tab
• Polling/SSE logic to fetch progress updates
• Disable "Import History" button while import is in progress

Priority 2: Stock Management

When an order is confirmed/imported:

1. Match EAN from order to local product catalog
2. Decrease stock quantity for matched products
3. Handle cases where product not found (alert/log)

Considerations:

• Should stock decrease happen on import or only on confirmation?
• Need rollback mechanism if order is rejected
• Handle partial matches (some items found, some not)

Priority 2: Order Detail View Enhancement

Improve the order detail modal to show:

• Product details (name, EAN, MPN, SKU)
• Match status per line item (found/not found in catalog)
• Link to local product if matched

Priority 3: Invoice Generation

Use customer_locale to generate invoices in customer's language:

• Invoice template with multi-language support
• PDF generation

Priority 4: Analytics Dashboard

Build sales analytics based on imported orders:

• Sales by product
• Sales by time period
• Customer statistics
• Revenue breakdown

---

Files Modified (2025-12-16 to 2025-12-18)

File	Changes
app/services/letzshop/client_service.py	Added paginated query, updated all queries with EAN/locale/country
app/services/letzshop/order_service.py	Added historical import, EAN matching, summary endpoint, order stats
models/database/letzshop.py	Added locale and country columns
models/database/product.py	Added gtin and gtin_type columns for EAN matching
models/schema/letzshop.py	Added LetzshopOrderStats schema, stats to order list response
app/api/v1/admin/letzshop.py	Added import-history and import-summary endpoints, stats in orders response
app/templates/admin/partials/letzshop-orders-tab.html	Added Import History button and result display
static/admin/js/marketplace-letzshop.js	Added importHistoricalOrders(), server-side stats
tests/unit/services/test_letzshop_service.py	Added tests for new functionality
scripts/test_historical_import.py	Manual test script for historical import
scripts/letzshop_introspect.py	GraphQL schema introspection tool, tracking workaround tests
alembic/versions/a9a86cef6cca_*.py	Migration for locale/country columns
alembic/versions/cb88bc9b5f86_*.py	Migration for gtin columns on Product table