# Multi-Marketplace Product Architecture ## Overview This document outlines the implementation plan for evolving the product management system to support: 1. **Multiple Marketplaces**: Letzshop, Amazon, eBay, and future sources 2. **Multi-Language Support**: Localized titles, descriptions with language fallback 3. **Store Override Pattern**: Override any field with reset-to-source capability 4. **Digital Products**: Support for digital goods (games, gift cards, downloadable content) 5. **Universal Product Model**: Marketplace-agnostic canonical product representation ## Core Principles (Preserved) | Principle | Description | |-----------|-------------| | **Separation of Concerns** | Raw marketplace data in source tables; store customizations in `products` | | **Multi-Store Support** | Same marketplace product can appear in multiple store catalogs | | **Idempotent Imports** | Re-importing CSV updates existing records, never duplicates | | **Asynchronous Processing** | Large imports run in background tasks | --- ## Architecture Overview ```mermaid graph TB subgraph "Source Layer (Raw Data)" LS[Letzshop CSV] AZ[Amazon API] EB[eBay API] end subgraph "Import Layer" LSI[LetzshopImporter] AZI[AmazonImporter] EBI[EbayImporter] end subgraph "Canonical Layer (Normalized)" MP[marketplace_products] MPT[marketplace_product_translations] end subgraph "Store Layer (Overrides)" P[products] PT[product_translations] end LS --> LSI AZ --> AZI EB --> EBI LSI --> MP AZI --> MP EBI --> MP MP --> MPT MP --> P P --> PT ``` --- ## Database Schema Design ### Phase 1: Enhanced Marketplace Products #### 1.1 Updated `marketplace_products` Table Evolve the existing table to be marketplace-agnostic: ```python # models/database/marketplace_product.py class ProductType(str, Enum): """Product type enumeration.""" PHYSICAL = "physical" DIGITAL = "digital" SERVICE = "service" SUBSCRIPTION = "subscription" class DigitalDeliveryMethod(str, Enum): """Digital product delivery methods.""" DOWNLOAD = "download" EMAIL = "email" IN_APP = "in_app" STREAMING = "streaming" LICENSE_KEY = "license_key" class MarketplaceProduct(Base, TimestampMixin): __tablename__ = "marketplace_products" id = Column(Integer, primary_key=True, index=True) # === UNIVERSAL IDENTIFIERS === marketplace_product_id = Column(String, unique=True, index=True, nullable=False) gtin = Column(String, index=True) # EAN/UPC - primary cross-marketplace matching mpn = Column(String, index=True) # Manufacturer Part Number sku = Column(String, index=True) # Internal SKU if assigned # === SOURCE TRACKING === marketplace = Column(String, index=True, nullable=False) # 'letzshop', 'amazon', 'ebay' source_url = Column(String) # Original product URL store_name = Column(String, index=True) # Seller/store in marketplace # === PRODUCT TYPE (NEW) === product_type = Column( SQLEnum(ProductType, name="product_type_enum"), default=ProductType.PHYSICAL, nullable=False, index=True ) # === DIGITAL PRODUCT FIELDS (NEW) === is_digital = Column(Boolean, default=False, index=True) digital_delivery_method = Column( SQLEnum(DigitalDeliveryMethod, name="digital_delivery_enum"), nullable=True ) download_url = Column(String) # For downloadable products license_type = Column(String) # e.g., "single_use", "subscription", "lifetime" platform = Column(String) # e.g., "steam", "playstation", "xbox", "universal" region_restrictions = Column(JSON) # ["EU", "US"] or null for global # === NON-LOCALIZED FIELDS === brand = Column(String, index=True) google_product_category = Column(String, index=True) category_path = Column(String) # Normalized category hierarchy condition = Column(String) # === PRICING === price = Column(Float) # Normalized numeric price price_raw = Column(String) # Original "19.99 EUR" format sale_price = Column(Float) sale_price_raw = Column(String) currency = Column(String(3), default='EUR') # === MEDIA === image_link = Column(String) additional_image_link = Column(String) # Kept for backward compat additional_images = Column(JSON) # Array of image URLs (new) # === PRODUCT ATTRIBUTES (Flexible) === attributes = Column(JSON) # {color, size, material, etc.} # === PHYSICAL PRODUCT FIELDS === weight = Column(Float) # In kg weight_unit = Column(String, default='kg') dimensions = Column(JSON) # {length, width, height, unit} # === GOOGLE SHOPPING FIELDS (Preserved) === # These remain for Letzshop compatibility link = Column(String) availability = Column(String, index=True) adult = Column(String) multipack = Column(Integer) is_bundle = Column(String) age_group = Column(String) color = Column(String) gender = Column(String) material = Column(String) pattern = Column(String) size = Column(String) size_type = Column(String) size_system = Column(String) item_group_id = Column(String) product_type_raw = Column(String) # Renamed from product_type custom_label_0 = Column(String) custom_label_1 = Column(String) custom_label_2 = Column(String) custom_label_3 = Column(String) custom_label_4 = Column(String) unit_pricing_measure = Column(String) unit_pricing_base_measure = Column(String) identifier_exists = Column(String) shipping = Column(String) # === STATUS === is_active = Column(Boolean, default=True, index=True) # === RELATIONSHIPS === translations = relationship( "MarketplaceProductTranslation", back_populates="marketplace_product", cascade="all, delete-orphan" ) store_products = relationship("Product", back_populates="marketplace_product") # === INDEXES === __table_args__ = ( Index("idx_mp_marketplace_store", "marketplace", "store_name"), Index("idx_mp_marketplace_brand", "marketplace", "brand"), Index("idx_mp_gtin_marketplace", "gtin", "marketplace"), Index("idx_mp_product_type", "product_type", "is_digital"), ) ``` #### 1.2 New `marketplace_product_translations` Table ```python # models/database/marketplace_product_translation.py class MarketplaceProductTranslation(Base, TimestampMixin): """Localized content for marketplace products.""" __tablename__ = "marketplace_product_translations" id = Column(Integer, primary_key=True, index=True) marketplace_product_id = Column( Integer, ForeignKey("marketplace_products.id", ondelete="CASCADE"), nullable=False ) language = Column(String(5), nullable=False) # 'en', 'fr', 'de', 'lb' # === LOCALIZED CONTENT === title = Column(String, nullable=False) description = Column(Text) short_description = Column(String(500)) # === SEO FIELDS === meta_title = Column(String(70)) meta_description = Column(String(160)) url_slug = Column(String(255)) # === SOURCE TRACKING === source_import_id = Column(Integer) # Which import job provided this source_file = Column(String) # e.g., "letzshop_fr.csv" # === RELATIONSHIPS === marketplace_product = relationship( "MarketplaceProduct", back_populates="translations" ) __table_args__ = ( UniqueConstraint( "marketplace_product_id", "language", name="uq_marketplace_product_translation" ), Index("idx_mpt_language", "language"), ) ``` ### Phase 2: Enhanced Store Products with Override Pattern #### 2.1 Updated `products` Table ```python # models/database/product.py class Product(Base, TimestampMixin): """Store-specific product with override capability.""" __tablename__ = "products" id = Column(Integer, primary_key=True, index=True) store_id = Column(Integer, ForeignKey("stores.id"), nullable=False) marketplace_product_id = Column( Integer, ForeignKey("marketplace_products.id"), nullable=False ) # === STORE REFERENCE === store_sku = Column(String, index=True) # Store's internal SKU # === OVERRIDABLE FIELDS (NULL = inherit from marketplace_product) === # Pricing price = Column(Float) sale_price = Column(Float) currency = Column(String(3)) # Product Info brand = Column(String) condition = Column(String) availability = Column(String) # Media primary_image_url = Column(String) additional_images = Column(JSON) # Digital Product Overrides download_url = Column(String) license_type = Column(String) # === STORE-SPECIFIC (No inheritance) === is_featured = Column(Boolean, default=False) is_active = Column(Boolean, default=True) display_order = Column(Integer, default=0) # Inventory Settings min_quantity = Column(Integer, default=1) max_quantity = Column(Integer) # Digital Fulfillment fulfillment_email_template = Column(String) # For digital delivery # === RELATIONSHIPS === store = relationship("Store", back_populates="products") marketplace_product = relationship( "MarketplaceProduct", back_populates="store_products" ) translations = relationship( "ProductTranslation", back_populates="product", cascade="all, delete-orphan" ) inventory_entries = relationship( "Inventory", back_populates="product", cascade="all, delete-orphan" ) __table_args__ = ( UniqueConstraint( "store_id", "marketplace_product_id", name="uq_store_marketplace_product" ), Index("idx_product_store_active", "store_id", "is_active"), Index("idx_product_store_featured", "store_id", "is_featured"), Index("idx_product_store_sku", "store_id", "store_sku"), ) # === EFFECTIVE PROPERTIES (Override Pattern) === OVERRIDABLE_FIELDS = [ "price", "sale_price", "currency", "brand", "condition", "availability", "primary_image_url", "additional_images", "download_url", "license_type" ] @property def effective_price(self) -> float | None: """Get price (store override or marketplace fallback).""" if self.price is not None: return self.price return self.marketplace_product.price if self.marketplace_product else None @property def effective_sale_price(self) -> float | None: if self.sale_price is not None: return self.sale_price return self.marketplace_product.sale_price if self.marketplace_product else None @property def effective_currency(self) -> str: if self.currency is not None: return self.currency return self.marketplace_product.currency if self.marketplace_product else "EUR" @property def effective_brand(self) -> str | None: if self.brand is not None: return self.brand return self.marketplace_product.brand if self.marketplace_product else None @property def effective_condition(self) -> str | None: if self.condition is not None: return self.condition return self.marketplace_product.condition if self.marketplace_product else None @property def effective_availability(self) -> str | None: if self.availability is not None: return self.availability return self.marketplace_product.availability if self.marketplace_product else None @property def effective_primary_image_url(self) -> str | None: if self.primary_image_url is not None: return self.primary_image_url return self.marketplace_product.image_link if self.marketplace_product else None @property def effective_additional_images(self) -> list | None: if self.additional_images is not None: return self.additional_images mp = self.marketplace_product if mp and mp.additional_images: return mp.additional_images return None @property def is_digital(self) -> bool: """Check if this is a digital product.""" return self.marketplace_product.is_digital if self.marketplace_product else False @property def product_type(self) -> str: """Get product type from marketplace product.""" return self.marketplace_product.product_type if self.marketplace_product else "physical" def get_override_info(self) -> dict: """ Get all fields with inheritance flags. Similar to Store.get_contact_info_with_inheritance() """ mp = self.marketplace_product return { # Price "price": self.effective_price, "price_overridden": self.price is not None, "price_source": mp.price if mp else None, # Sale Price "sale_price": self.effective_sale_price, "sale_price_overridden": self.sale_price is not None, "sale_price_source": mp.sale_price if mp else None, # Currency "currency": self.effective_currency, "currency_overridden": self.currency is not None, "currency_source": mp.currency if mp else None, # Brand "brand": self.effective_brand, "brand_overridden": self.brand is not None, "brand_source": mp.brand if mp else None, # Condition "condition": self.effective_condition, "condition_overridden": self.condition is not None, "condition_source": mp.condition if mp else None, # Availability "availability": self.effective_availability, "availability_overridden": self.availability is not None, "availability_source": mp.availability if mp else None, # Images "primary_image_url": self.effective_primary_image_url, "primary_image_url_overridden": self.primary_image_url is not None, "primary_image_url_source": mp.image_link if mp else None, # Product type info "is_digital": self.is_digital, "product_type": self.product_type, } def reset_field_to_source(self, field_name: str) -> bool: """Reset a single field to inherit from marketplace product.""" if field_name in self.OVERRIDABLE_FIELDS: setattr(self, field_name, None) return True return False def reset_all_to_source(self) -> None: """Reset all overridable fields to inherit from marketplace product.""" for field in self.OVERRIDABLE_FIELDS: setattr(self, field, None) ``` #### 2.2 New `product_translations` Table ```python # models/database/product_translation.py class ProductTranslation(Base, TimestampMixin): """Store-specific localized content with override capability.""" __tablename__ = "product_translations" id = Column(Integer, primary_key=True, index=True) product_id = Column( Integer, ForeignKey("products.id", ondelete="CASCADE"), nullable=False ) language = Column(String(5), nullable=False) # === OVERRIDABLE LOCALIZED FIELDS (NULL = inherit) === title = Column(String) description = Column(Text) short_description = Column(String(500)) # SEO Overrides meta_title = Column(String(70)) meta_description = Column(String(160)) url_slug = Column(String(255)) # === RELATIONSHIPS === product = relationship("Product", back_populates="translations") __table_args__ = ( UniqueConstraint("product_id", "language", name="uq_product_translation"), Index("idx_pt_product_language", "product_id", "language"), ) OVERRIDABLE_FIELDS = [ "title", "description", "short_description", "meta_title", "meta_description", "url_slug" ] def get_effective_title(self) -> str | None: """Get title with fallback to marketplace translation.""" if self.title is not None: return self.title return self._get_marketplace_translation_field("title") def get_effective_description(self) -> str | None: if self.description is not None: return self.description return self._get_marketplace_translation_field("description") def _get_marketplace_translation_field(self, field: str) -> str | None: """Helper to get field from marketplace translation.""" mp = self.product.marketplace_product if mp: for t in mp.translations: if t.language == self.language: return getattr(t, field, None) return None def get_override_info(self) -> dict: """Get all fields with inheritance flags.""" return { "title": self.get_effective_title(), "title_overridden": self.title is not None, "description": self.get_effective_description(), "description_overridden": self.description is not None, # ... similar for other fields } def reset_to_source(self) -> None: """Reset all fields to inherit from marketplace translation.""" for field in self.OVERRIDABLE_FIELDS: setattr(self, field, None) ``` --- ## Pydantic Schemas ### 3.1 Product Update Schema with Reset ```python # models/schema/product.py class ProductUpdate(BaseModel): """Update product with override and reset capabilities.""" # === OVERRIDABLE FIELDS === price: float | None = None sale_price: float | None = None currency: str | None = None brand: str | None = None condition: str | None = None availability: str | None = None primary_image_url: str | None = None additional_images: list[str] | None = None download_url: str | None = None license_type: str | None = None # === STORE-SPECIFIC FIELDS === store_sku: str | None = None is_featured: bool | None = None is_active: bool | None = None display_order: int | None = None min_quantity: int | None = None max_quantity: int | None = None fulfillment_email_template: str | None = None # === RESET CONTROLS === reset_all_to_source: bool | None = Field( None, description="Reset ALL overridable fields to marketplace source values" ) reset_fields: list[str] | None = Field( None, description="List of specific fields to reset: ['price', 'brand']" ) model_config = ConfigDict(extra="forbid") @field_validator("reset_fields") @classmethod def validate_reset_fields(cls, v): if v: valid_fields = Product.OVERRIDABLE_FIELDS invalid = [f for f in v if f not in valid_fields] if invalid: raise ValueError(f"Invalid reset fields: {invalid}. Valid: {valid_fields}") return v class ProductTranslationUpdate(BaseModel): """Update product translation with reset capability.""" title: str | None = None description: str | None = None short_description: str | None = None meta_title: str | None = None meta_description: str | None = None url_slug: str | None = None reset_to_source: bool | None = Field( None, description="Reset this translation to marketplace source" ) reset_fields: list[str] | None = Field( None, description="List of specific translation fields to reset" ) class ProductDetailResponse(BaseModel): """Detailed product response with override information.""" id: int store_id: int marketplace_product_id: int store_sku: str | None # === EFFECTIVE VALUES WITH INHERITANCE FLAGS === price: float | None price_overridden: bool price_source: float | None sale_price: float | None sale_price_overridden: bool sale_price_source: float | None currency: str currency_overridden: bool currency_source: str | None brand: str | None brand_overridden: bool brand_source: str | None condition: str | None condition_overridden: bool condition_source: str | None availability: str | None availability_overridden: bool availability_source: str | None primary_image_url: str | None primary_image_url_overridden: bool primary_image_url_source: str | None # === PRODUCT TYPE INFO === is_digital: bool product_type: str # === STORE-SPECIFIC === is_featured: bool is_active: bool display_order: int min_quantity: int max_quantity: int | None # === TRANSLATIONS === translations: list["ProductTranslationResponse"] # === TIMESTAMPS === created_at: datetime updated_at: datetime class ProductTranslationResponse(BaseModel): """Translation response with override information.""" language: str title: str | None title_overridden: bool description: str | None description_overridden: bool short_description: str | None short_description_overridden: bool meta_title: str | None meta_title_overridden: bool meta_description: str | None meta_description_overridden: bool url_slug: str | None url_slug_overridden: bool ``` --- ## Service Layer Implementation ### 4.1 Product Service with Reset Logic ```python # app/services/product_service.py class ProductService: """Service for managing store products with override pattern.""" def update_product( self, db: Session, store_id: int, product_id: int, update_data: ProductUpdate ) -> Product: """ Update product with override and reset support. Reset behavior: - reset_all_to_source=True: Resets ALL overridable fields - reset_fields=['price', 'brand']: Resets specific fields - Setting field to empty string: Resets that field """ product = db.query(Product).filter( Product.id == product_id, Product.store_id == store_id ).first() if not product: raise NotFoundError("Product not found") data = update_data.model_dump(exclude_unset=True) # Handle reset_all_to_source if data.pop("reset_all_to_source", False): product.reset_all_to_source() # Handle reset_fields (selective reset) reset_fields = data.pop("reset_fields", None) if reset_fields: for field in reset_fields: product.reset_field_to_source(field) # Handle empty strings = reset (like store pattern) for field in Product.OVERRIDABLE_FIELDS: if field in data and data[field] == "": data[field] = None # Apply remaining updates for key, value in data.items(): if hasattr(product, key): setattr(product, key, value) db.flush() return product def update_product_translation( self, db: Session, product_id: int, language: str, update_data: ProductTranslationUpdate ) -> ProductTranslation: """Update product translation with reset support.""" translation = db.query(ProductTranslation).filter( ProductTranslation.product_id == product_id, ProductTranslation.language == language ).first() if not translation: # Create new translation if doesn't exist translation = ProductTranslation( product_id=product_id, language=language ) db.add(translation) data = update_data.model_dump(exclude_unset=True) # Handle reset_to_source if data.pop("reset_to_source", False): translation.reset_to_source() # Handle reset_fields reset_fields = data.pop("reset_fields", None) if reset_fields: for field in reset_fields: if field in ProductTranslation.OVERRIDABLE_FIELDS: setattr(translation, field, None) # Handle empty strings for field in ProductTranslation.OVERRIDABLE_FIELDS: if field in data and data[field] == "": data[field] = None # Apply updates for key, value in data.items(): if hasattr(translation, key): setattr(translation, key, value) db.flush() return translation def build_detail_response(self, product: Product) -> ProductDetailResponse: """Build detailed response with override information.""" override_info = product.get_override_info() translations = [] for t in product.translations: trans_info = t.get_override_info() translations.append(ProductTranslationResponse( language=t.language, **trans_info )) return ProductDetailResponse( id=product.id, store_id=product.store_id, marketplace_product_id=product.marketplace_product_id, store_sku=product.store_sku, **override_info, is_featured=product.is_featured, is_active=product.is_active, display_order=product.display_order, min_quantity=product.min_quantity, max_quantity=product.max_quantity, translations=translations, created_at=product.created_at, updated_at=product.updated_at, ) ``` --- ## Import Architecture ### 5.1 Base Importer Pattern ```python # app/utils/marketplace_importers/base.py from abc import ABC, abstractmethod from typing import Generator class BaseMarketplaceImporter(ABC): """Abstract base class for marketplace importers.""" marketplace_name: str = "" @abstractmethod def parse_row(self, row: dict) -> dict: """ Parse marketplace-specific row into intermediate format. Override in subclasses for each marketplace. """ pass @abstractmethod def get_product_identifier(self, row: dict) -> str: """Get unique product identifier from marketplace.""" pass @abstractmethod def get_language(self) -> str: """Get language code for this import.""" pass def normalize_gtin(self, gtin: str | None) -> str | None: """Normalize GTIN to standard format.""" if not gtin: return None gtin = str(gtin).strip().replace(" ", "") if gtin.endswith(".0"): gtin = gtin[:-2] if len(gtin) in (8, 12, 13, 14) and gtin.isdigit(): return gtin.zfill(14) # Pad to GTIN-14 return gtin def parse_price(self, price_str: str | None) -> tuple[float | None, str]: """Parse price string into amount and currency.""" if not price_str: return None, "EUR" # Handle formats like "19.99 EUR", "EUR 19.99", "19,99€" import re price_str = str(price_str).strip() # Extract currency currency = "EUR" currency_patterns = [ (r"EUR", "EUR"), (r"USD", "USD"), (r"€", "EUR"), (r"\$", "USD"), (r"£", "GBP"), ] for pattern, curr in currency_patterns: if re.search(pattern, price_str): currency = curr break # Extract numeric value numbers = re.findall(r"[\d.,]+", price_str) if numbers: num_str = numbers[0].replace(",", ".") try: return float(num_str), currency except ValueError: pass return None, currency def to_canonical(self, row: dict) -> dict: """ Convert parsed row to canonical MarketplaceProduct format. """ parsed = self.parse_row(row) price, currency = self.parse_price(parsed.get("price")) sale_price, _ = self.parse_price(parsed.get("sale_price")) return { "marketplace_product_id": self.get_product_identifier(row), "marketplace": self.marketplace_name, "gtin": self.normalize_gtin(parsed.get("gtin")), "mpn": parsed.get("mpn"), "brand": parsed.get("brand"), "price": price, "price_raw": parsed.get("price"), "sale_price": sale_price, "sale_price_raw": parsed.get("sale_price"), "currency": currency, "image_link": parsed.get("image_url"), "condition": parsed.get("condition"), "availability": parsed.get("availability"), "google_product_category": parsed.get("category"), "product_type": self.determine_product_type(parsed), "is_digital": self.is_digital_product(parsed), "attributes": parsed.get("attributes", {}), # Raw fields preserved **{k: v for k, v in parsed.items() if k.startswith("raw_")} } def to_translation(self, row: dict) -> dict: """Extract translation data from row.""" parsed = self.parse_row(row) return { "language": self.get_language(), "title": parsed.get("title"), "description": parsed.get("description"), "short_description": parsed.get("short_description"), } def determine_product_type(self, parsed: dict) -> str: """Determine if product is physical, digital, etc.""" # Override in subclass for marketplace-specific logic return "physical" def is_digital_product(self, parsed: dict) -> bool: """Check if product is digital.""" # Override in subclass return False ``` ### 5.2 Letzshop Importer ```python # app/utils/marketplace_importers/letzshop.py class LetzshopImporter(BaseMarketplaceImporter): """Importer for Letzshop Google Shopping CSV feeds.""" marketplace_name = "letzshop" # Column mapping from Letzshop CSV to internal format COLUMN_MAP = { "g:id": "id", "g:title": "title", "g:description": "description", "g:link": "link", "g:image_link": "image_url", "g:availability": "availability", "g:price": "price", "g:sale_price": "sale_price", "g:brand": "brand", "g:gtin": "gtin", "g:mpn": "mpn", "g:condition": "condition", "g:google_product_category": "category", "g:product_type": "product_type_raw", "g:color": "color", "g:size": "size", "g:material": "material", "g:gender": "gender", "g:age_group": "age_group", "g:item_group_id": "item_group_id", "g:additional_image_link": "additional_images", "g:custom_label_0": "custom_label_0", "g:custom_label_1": "custom_label_1", "g:custom_label_2": "custom_label_2", "g:custom_label_3": "custom_label_3", "g:custom_label_4": "custom_label_4", } def __init__(self, language: str = "en"): self.language = language def get_language(self) -> str: return self.language def get_product_identifier(self, row: dict) -> str: return row.get("g:id") or row.get("id") def parse_row(self, row: dict) -> dict: """Parse Letzshop CSV row.""" result = {} for csv_col, internal_key in self.COLUMN_MAP.items(): value = row.get(csv_col) or row.get(csv_col.replace("g:", "")) if value: result[internal_key] = value # Build attributes dict result["attributes"] = { k: result.pop(k) for k in ["color", "size", "material", "gender", "age_group"] if k in result } return result def determine_product_type(self, parsed: dict) -> str: """Detect digital products from Letzshop data.""" category = (parsed.get("category") or "").lower() product_type = (parsed.get("product_type_raw") or "").lower() title = (parsed.get("title") or "").lower() digital_keywords = [ "digital", "download", "ebook", "e-book", "software", "license", "subscription", "gift card", "voucher", "game key", "steam", "playstation", "xbox", "nintendo" ] combined = f"{category} {product_type} {title}" if any(kw in combined for kw in digital_keywords): return "digital" return "physical" def is_digital_product(self, parsed: dict) -> bool: return self.determine_product_type(parsed) == "digital" ``` ### 5.3 Amazon Importer (Future) ```python # app/utils/marketplace_importers/amazon.py class AmazonImporter(BaseMarketplaceImporter): """Importer for Amazon product feeds.""" marketplace_name = "amazon" def __init__(self, language: str = "en"): self.language = language def get_language(self) -> str: return self.language def get_product_identifier(self, row: dict) -> str: return f"amazon_{row.get('asin')}" def parse_row(self, row: dict) -> dict: """Parse Amazon product data.""" # Amazon has different field names bullet_points = row.get("bullet_points", []) description = row.get("product_description", "") # Combine bullet points into description if needed if bullet_points and not description: description = "\n".join(f"• {bp}" for bp in bullet_points) return { "id": row.get("asin"), "title": row.get("item_name"), "description": description, "short_description": bullet_points[0] if bullet_points else None, "price": row.get("list_price"), "sale_price": row.get("deal_price"), "brand": row.get("brand_name"), "gtin": row.get("product_id"), # Amazon uses different field "image_url": row.get("main_image_url"), "additional_images": row.get("other_image_urls", []), "category": row.get("browse_node_name"), "condition": "new", # Amazon typically new "availability": "in_stock" if row.get("availability") == "IN_STOCK" else "out_of_stock", "attributes": { "color": row.get("color"), "size": row.get("size"), "material": row.get("material"), }, # Amazon-specific fields preserved "raw_parent_asin": row.get("parent_asin"), "raw_variation_theme": row.get("variation_theme"), } def determine_product_type(self, parsed: dict) -> str: """Amazon has explicit product type indicators.""" # Amazon provides explicit digital flags if parsed.get("raw_is_digital"): return "digital" category = (parsed.get("category") or "").lower() if "digital" in category or "software" in category: return "digital" return "physical" ``` ### 5.4 Importer Factory ```python # app/utils/marketplace_importers/__init__.py from .base import BaseMarketplaceImporter from .letzshop import LetzshopImporter # Registry of available importers IMPORTER_REGISTRY = { "letzshop": LetzshopImporter, # "amazon": AmazonImporter, # Future # "ebay": EbayImporter, # Future } def get_importer( marketplace: str, language: str = "en" ) -> BaseMarketplaceImporter: """ Factory function to get appropriate importer. Args: marketplace: Marketplace name (letzshop, amazon, ebay) language: Language code for import Returns: Configured importer instance Raises: ValueError: If marketplace not supported """ importer_class = IMPORTER_REGISTRY.get(marketplace.lower()) if not importer_class: supported = list(IMPORTER_REGISTRY.keys()) raise ValueError( f"Unsupported marketplace: {marketplace}. " f"Supported: {supported}" ) return importer_class(language=language) ``` --- ## Digital Products Handling ### 6.1 Digital Product Considerations | Aspect | Physical Products | Digital Products | |--------|-------------------|------------------| | **Inventory** | Track stock by location | Unlimited or license-based | | **Fulfillment** | Shipping required | Instant delivery (email/download) | | **Returns** | Physical return process | Different policy (often non-refundable) | | **Variants** | Size, color, material | Platform, region, edition | | **Pricing** | Per unit | Per license/subscription | ### 6.2 Digital Product Fields ```python # In MarketplaceProduct model # Digital product classification product_type = Column(Enum(ProductType)) # physical, digital, service, subscription is_digital = Column(Boolean, default=False) # Digital delivery digital_delivery_method = Column(Enum(DigitalDeliveryMethod)) # Options: download, email, in_app, streaming, license_key # Platform/Region restrictions platform = Column(String) # steam, playstation, xbox, nintendo, universal region_restrictions = Column(JSON) # ["EU", "US"] or null for global # License information license_type = Column(String) # single_use, subscription, lifetime license_duration_days = Column(Integer) # For subscriptions ``` ### 6.3 Digital Product Import Detection ```python def detect_digital_product(row: dict) -> tuple[bool, str | None, str | None]: """ Detect if product is digital and extract delivery info. Returns: (is_digital, delivery_method, platform) """ title = (row.get("title") or "").lower() category = (row.get("category") or "").lower() description = (row.get("description") or "").lower() combined = f"{title} {category} {description}" # Platform detection platform_keywords = { "steam": "steam", "playstation": "playstation", "psn": "playstation", "xbox": "xbox", "nintendo": "nintendo", "switch": "nintendo", "pc": "pc", } detected_platform = None for keyword, platform in platform_keywords.items(): if keyword in combined: detected_platform = platform break # Digital product keywords digital_indicators = [ ("gift card", "email", None), ("voucher", "email", None), ("e-book", "download", None), ("ebook", "download", None), ("download", "download", None), ("digital", "download", None), ("license", "license_key", None), ("game key", "license_key", detected_platform), ("activation code", "license_key", detected_platform), ("subscription", "in_app", None), ] for keyword, delivery_method, platform in digital_indicators: if keyword in combined: return True, delivery_method, platform or detected_platform return False, None, None ``` ### 6.4 Inventory Handling for Digital Products ```python # app/services/inventory_service.py def create_inventory_for_product( db: Session, product: Product, quantity: int = None ) -> Inventory: """ Create inventory record with digital product handling. """ # Digital products get special handling if product.is_digital: return Inventory( product_id=product.id, store_id=product.store_id, location="digital", # Special location for digital quantity=999999, # Effectively unlimited reserved_quantity=0, is_digital=True, # Track license keys if applicable license_pool_id=product.license_pool_id, ) # Physical products return Inventory( product_id=product.id, store_id=product.store_id, location="warehouse", quantity=quantity or 0, reserved_quantity=0, is_digital=False, ) ``` --- ## Migration Plan ### Phase 1: Database Schema Updates **Migration 1: Add product type and digital fields to marketplace_products** ```python # alembic/versions/xxxx_add_product_type_and_digital_fields.py def upgrade(): # Add product type enum product_type_enum = sa.Enum( 'physical', 'digital', 'service', 'subscription', name='product_type_enum' ) product_type_enum.create(op.get_bind()) delivery_method_enum = sa.Enum( 'download', 'email', 'in_app', 'streaming', 'license_key', name='digital_delivery_enum' ) delivery_method_enum.create(op.get_bind()) # Add columns to marketplace_products op.add_column('marketplace_products', sa.Column('product_type', product_type_enum, server_default='physical', nullable=False) ) op.add_column('marketplace_products', sa.Column('is_digital', sa.Boolean(), server_default='false', nullable=False) ) op.add_column('marketplace_products', sa.Column('digital_delivery_method', delivery_method_enum, nullable=True) ) op.add_column('marketplace_products', sa.Column('platform', sa.String(), nullable=True) ) op.add_column('marketplace_products', sa.Column('region_restrictions', sa.JSON(), nullable=True) ) op.add_column('marketplace_products', sa.Column('license_type', sa.String(), nullable=True) ) op.add_column('marketplace_products', sa.Column('attributes', sa.JSON(), nullable=True) ) # Add indexes op.create_index('idx_mp_product_type', 'marketplace_products', ['product_type', 'is_digital']) def downgrade(): op.drop_index('idx_mp_product_type') op.drop_column('marketplace_products', 'attributes') op.drop_column('marketplace_products', 'license_type') op.drop_column('marketplace_products', 'region_restrictions') op.drop_column('marketplace_products', 'platform') op.drop_column('marketplace_products', 'digital_delivery_method') op.drop_column('marketplace_products', 'is_digital') op.drop_column('marketplace_products', 'product_type') sa.Enum(name='digital_delivery_enum').drop(op.get_bind()) sa.Enum(name='product_type_enum').drop(op.get_bind()) ``` **Migration 2: Create translation tables** ```python # alembic/versions/xxxx_create_translation_tables.py def upgrade(): # Create marketplace_product_translations table op.create_table( 'marketplace_product_translations', sa.Column('id', sa.Integer(), primary_key=True), sa.Column('marketplace_product_id', sa.Integer(), sa.ForeignKey('marketplace_products.id', ondelete='CASCADE'), nullable=False), sa.Column('language', sa.String(5), nullable=False), sa.Column('title', sa.String(), nullable=False), sa.Column('description', sa.Text(), nullable=True), sa.Column('short_description', sa.String(500), nullable=True), sa.Column('meta_title', sa.String(70), nullable=True), sa.Column('meta_description', sa.String(160), nullable=True), sa.Column('url_slug', sa.String(255), nullable=True), sa.Column('source_import_id', sa.Integer(), nullable=True), sa.Column('source_file', sa.String(), nullable=True), sa.Column('created_at', sa.DateTime(), nullable=True), sa.Column('updated_at', sa.DateTime(), nullable=True), sa.UniqueConstraint('marketplace_product_id', 'language', name='uq_marketplace_product_translation'), ) op.create_index('idx_mpt_language', 'marketplace_product_translations', ['language']) # Create product_translations table op.create_table( 'product_translations', sa.Column('id', sa.Integer(), primary_key=True), sa.Column('product_id', sa.Integer(), sa.ForeignKey('products.id', ondelete='CASCADE'), nullable=False), sa.Column('language', sa.String(5), nullable=False), sa.Column('title', sa.String(), nullable=True), sa.Column('description', sa.Text(), nullable=True), sa.Column('short_description', sa.String(500), nullable=True), sa.Column('meta_title', sa.String(70), nullable=True), sa.Column('meta_description', sa.String(160), nullable=True), sa.Column('url_slug', sa.String(255), nullable=True), sa.Column('created_at', sa.DateTime(), nullable=True), sa.Column('updated_at', sa.DateTime(), nullable=True), sa.UniqueConstraint('product_id', 'language', name='uq_product_translation'), ) op.create_index('idx_pt_product_language', 'product_translations', ['product_id', 'language']) def downgrade(): op.drop_table('product_translations') op.drop_table('marketplace_product_translations') ``` **Migration 3: Add override fields to products table** ```python # alembic/versions/xxxx_add_product_override_fields.py def upgrade(): # Rename product_id to store_sku for clarity op.alter_column('products', 'product_id', new_column_name='store_sku') # Add new overridable fields op.add_column('products', sa.Column('brand', sa.String(), nullable=True) ) op.add_column('products', sa.Column('primary_image_url', sa.String(), nullable=True) ) op.add_column('products', sa.Column('additional_images', sa.JSON(), nullable=True) ) op.add_column('products', sa.Column('download_url', sa.String(), nullable=True) ) op.add_column('products', sa.Column('license_type', sa.String(), nullable=True) ) op.add_column('products', sa.Column('fulfillment_email_template', sa.String(), nullable=True) ) # Add index for store_sku op.create_index('idx_product_store_sku', 'products', ['store_id', 'store_sku']) def downgrade(): op.drop_index('idx_product_store_sku') op.drop_column('products', 'fulfillment_email_template') op.drop_column('products', 'license_type') op.drop_column('products', 'download_url') op.drop_column('products', 'additional_images') op.drop_column('products', 'primary_image_url') op.drop_column('products', 'brand') op.alter_column('products', 'store_sku', new_column_name='product_id') ``` **Migration 4: Data migration for existing products** ```python # alembic/versions/xxxx_migrate_existing_product_data.py def upgrade(): """Migrate existing title/description to translation tables.""" # Get connection for raw SQL conn = op.get_bind() # Migrate marketplace_products title/description to translations # Default language is 'en' for existing data conn.execute(text(""" INSERT INTO marketplace_product_translations (marketplace_product_id, language, title, description, created_at, updated_at) SELECT id, 'en', -- Default language for existing data title, description, created_at, updated_at FROM marketplace_products WHERE title IS NOT NULL ON CONFLICT (marketplace_product_id, language) DO NOTHING """)) def downgrade(): # Data migration is one-way, but we keep original columns pass ``` ### Phase 2: Code Updates 1. **Update Models**: Modify `models/database/` files 2. **Update Schemas**: Modify `models/schema/` files 3. **Update Services**: Add reset logic to services 4. **Update Importers**: Refactor CSV processor to use importer pattern 5. **Update API Endpoints**: Add translation and reset endpoints ### Phase 3: Testing 1. **Unit Tests**: Test override properties and reset methods 2. **Integration Tests**: Test import with multiple languages 3. **Migration Tests**: Test data migration on copy of production data --- ## API Endpoints ### New Endpoints Required ``` # Product Translations GET /api/v1/store/products/{id}/translations POST /api/v1/store/products/{id}/translations/{lang} PUT /api/v1/store/products/{id}/translations/{lang} DELETE /api/v1/store/products/{id}/translations/{lang} # Reset Operations POST /api/v1/store/products/{id}/reset POST /api/v1/store/products/{id}/translations/{lang}/reset # Marketplace Import with Language POST /api/v1/store/marketplace/import Body: { source_url, marketplace, language } # Admin: Multi-language Import POST /api/v1/admin/marketplace/import Body: { store_id, source_url, marketplace, language } ``` --- ## Implementation Checklist ### Database Layer - [ ] Create `product_type_enum` and `digital_delivery_enum` types - [ ] Add digital product fields to `marketplace_products` - [ ] Create `marketplace_product_translations` table - [ ] Create `product_translations` table - [ ] Add override fields to `products` table - [ ] Run data migration for existing content ### Model Layer - [ ] Update `MarketplaceProduct` model with new fields - [ ] Create `MarketplaceProductTranslation` model - [ ] Update `Product` model with effective properties - [ ] Create `ProductTranslation` model - [ ] Add `reset_*` methods to models ### Schema Layer - [ ] Update `ProductUpdate` with reset fields - [ ] Create `ProductTranslationUpdate` schema - [ ] Update `ProductDetailResponse` with override flags - [ ] Create `ProductTranslationResponse` schema ### Service Layer - [ ] Update `ProductService` with reset logic - [ ] Create `ProductTranslationService` - [ ] Update import service for multi-language ### Import Layer - [ ] Create `BaseMarketplaceImporter` abstract class - [ ] Refactor `LetzshopImporter` from CSV processor - [ ] Create importer factory - [ ] Add digital product detection ### API Layer - [ ] Add translation endpoints - [ ] Add reset endpoints - [ ] Update import endpoint for language parameter ### Testing - [ ] Unit tests for override properties - [ ] Unit tests for reset methods - [ ] Integration tests for multi-language import - [ ] API tests for new endpoints --- ## Summary This architecture provides: 1. **Universal Product Model**: Marketplace-agnostic with flexible attributes 2. **Multi-Language Support**: Translations at both marketplace and store levels 3. **Override Pattern**: Consistent with existing store contact pattern 4. **Reset Capability**: Individual field or bulk reset to source 5. **Digital Products**: Full support for games, gift cards, downloads 6. **Extensibility**: Easy to add Amazon, eBay, or other marketplaces 7. **Backward Compatibility**: Existing Letzshop imports continue to work The implementation preserves all existing principles while adding the flexibility needed for a multi-marketplace, multi-language e-commerce platform.