Files

Samir Boulahtit 4fb67e594d Added marketplace support

2025-09-05 22:14:52 +02:00

19 KiB

Raw Blame History

Letzshop Marketplace API v2.1

A robust, production-ready FastAPI backend for Luxembourg's premier e-commerce marketplace with multi-vendor support, JWT authentication, and advanced CSV import capabilities.

Key Features

Marketplace Architecture

Multi-Vendor Support: Shops can import and manage their product catalogs independently
Centralized Product Catalog: Products exist in main marketplace with shop-specific overrides
Shop Management: Complete vendor onboarding, verification, and management system
Shop-Specific Pricing: Vendors can set their own prices, availability, and conditions
Marketplace Controls: Admin verification and quality control for vendor shops

Security & Authentication

JWT Authentication: Secure token-based authentication with configurable expiration (30 minutes default)
User Management: Registration, login, role-based access control (Admin/User/Shop Owner roles)
Password Security: Bcrypt hashing for secure password storage
Protected Endpoints: All operations require authentication with proper authorization
Default Admin Account: Auto-created admin user for immediate system access

Architecture Improvements

Modular Design: Separated concerns into utility modules, middleware, and models
Database Optimization: Added proper indexing strategy and foreign key relationships
Connection Pooling: PostgreSQL support with connection pooling for production scalability
Background Processing: Asynchronous CSV import with job tracking per shop

Performance Optimizations

Batch Processing: CSV imports processed in configurable batches
Database Indexes: Strategic indexing for common query patterns including shop relationships
Streaming Export: Memory-efficient CSV export for large datasets with shop filtering
Rate Limiting: Sliding window rate limiter to prevent API abuse

Data Processing

Robust GTIN Handling: Centralized GTIN normalization and validation
Multi-currency Support: Advanced price parsing with currency extraction
International Content: Multi-encoding CSV support for global data
Shop Association: Automatic product-shop linking during CSV imports

Project Structure

letzshop_api/
├── main.py                 # FastAPI application entry point with marketplace support
├── models/
│   ├── database_models.py  # SQLAlchemy ORM models (User, Shop, Product, ShopProduct, Stock, ImportJob)
│   └── api_models.py       # Pydantic API models with shop and auth models
├── utils/
│   ├── data_processing.py  # GTIN and price processing utilities
│   ├── csv_processor.py    # CSV import/export handling with shop support
│   └── database.py         # Database configuration
├── middleware/
│   ├── auth.py            # JWT authentication with bcrypt
│   ├── rate_limiter.py    # Rate limiting implementation
│   ├── error_handler.py   # Centralized error handling
│   └── logging_middleware.py # Request/response logging
├── tests/
│   └── test_auth.py       # Authentication tests
├── requirements.txt       # Python dependencies with auth packages
└── README.md             # This file

Quick Start

1. Installation

# Clone the repository
git clone <repository-url>
cd letzshop-api

# Set up virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Environment Configuration

Create a .env file in the project root:

# Database
DATABASE_URL=postgresql://user:password@localhost:5432/letzshop_db
# For SQLite (development): DATABASE_URL=sqlite:///./letzshop.db

# JWT Configuration
JWT_SECRET_KEY=your-super-secret-key-change-in-production-immediately
JWT_EXPIRE_MINUTES=30

# Server Configuration
API_HOST=0.0.0.0
API_PORT=8000
DEBUG=False

Important Security Note: Always change the JWT_SECRET_KEY in production!

3. Database Setup

For SQLite (Development):

# Run the application - it will create tables automatically
python main.py

For PostgreSQL (Production):

# Create PostgreSQL database
createdb letzshop_db

# Run the application - it will create tables and indexes automatically
python main.py

4. Start the Server

# Development server
uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Production server
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4

The API will be available at http://localhost:8000

5. Default Access

The system automatically creates:

Admin User: admin / admin123 / admin@example.com
Demo Shop: DEMOSHOP owned by admin for testing

Security Warning: Change the admin password immediately in production!

Authentication Flow

1. Register a New User

curl -X POST "http://localhost:8000/register" \
     -H "Content-Type: application/json" \
     -d '{
       "email": "vendor@example.com",
       "username": "newvendor",
       "password": "securepassword123"
     }'

curl -X POST "http://localhost:8000/login" \
     -H "Content-Type: application/json" \
     -d '{
       "username": "admin",
       "password": "admin123"
     }'

Response:

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "token_type": "bearer",
  "expires_in": 1800,
  "user": {
    "id": 1,
    "username": "admin",
    "email": "admin@example.com",
    "role": "admin",
    "is_active": true
  }
}

3. Use Token for Protected Endpoints

curl -X GET "http://localhost:8000/shops" \
     -H "Authorization: Bearer YOUR_JWT_TOKEN_HERE"

Marketplace Workflow

1. Create a Shop

curl -X POST "http://localhost:8000/shops" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "shop_code": "TECHSTORE",
       "shop_name": "Tech Store Luxembourg",
       "description": "Electronics and gadgets for Luxembourg",
       "contact_email": "info@techstore.lu",
       "contact_phone": "+352 123 456 789",
       "website": "https://techstore.lu"
     }'

2. Import Products for Your Shop

curl -X POST "http://localhost:8000/import-csv" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "url": "https://techstore.com/products.csv",
       "shop_code": "TECHSTORE",
       "batch_size": 1000
     }'

3. Monitor Import Progress

curl -X GET "http://localhost:8000/import-status/1" \
     -H "Authorization: Bearer YOUR_TOKEN"

4. View Shop Products

curl -X GET "http://localhost:8000/products?shop_code=TECHSTORE" \
     -H "Authorization: Bearer YOUR_TOKEN"

API Endpoints

Public Endpoints

GET / - API information
GET /health - Health check
POST /register - Register new user
POST /login - Login and get JWT token

Protected Endpoints (Require Authentication)

User Management

GET /me - Get current user information

Shop Management

POST /shops - Create new shop
GET /shops - List shops with filtering
GET /shops/{shop_code} - Get shop details
PUT /shops/{shop_code} - Update shop (owners only)
POST /shops/{shop_code}/products - Add product to shop catalog
GET /shops/{shop_code}/products - Get shop products

Products (Marketplace Catalog)

GET /products - List products with filtering (optionally by shop)
POST /products - Create new product in marketplace catalog
GET /products/{product_id} - Get product with stock info and shop listings
PUT /products/{product_id} - Update product
DELETE /products/{product_id} - Delete product and associated shop listings

Stock Management

POST /stock - Set stock quantity (with optional shop association)
GET /stock/{gtin} - Get stock summary by GTIN

CSV Operations

POST /import-csv - Start background CSV import for specific shop
GET /import-status/{job_id} - Check import job status
GET /export-csv - Export products as CSV (optionally filtered by shop)

Statistics

GET /stats - Marketplace statistics

Admin-Only Endpoints

GET /admin/users - List all users
PUT /admin/users/{user_id}/status - Activate/deactivate users
GET /admin/shops - List all shops
PUT /admin/shops/{shop_id}/verify - Verify/unverify shop
PUT /admin/shops/{shop_id}/status - Activate/deactivate shop
GET /admin/import-jobs - View all import jobs

User Roles and Permissions

Regular Users

Can register and login
Can create and manage their own shops
Can import products for their shops
Can manage stock for their products
Can view marketplace products and shops

Shop Owners (Regular Users with Shops)

All regular user permissions
Can manage their shop information
Can import/export products for their shops
Can set shop-specific pricing and availability
Can view their import job history

Admin Users

All user permissions
Can view and manage all users and shops
Can verify/unverify shops
Can view all import jobs from any shop
Can activate/deactivate user accounts and shops

Marketplace Features

Shop Verification System

New shops start as unverified
Admin approval required for public visibility
Verified shops appear in public marketplace listings
Quality control through admin verification

Multi-Vendor Product Catalog

Products exist in central marketplace catalog
Multiple shops can sell the same product
Shop-specific pricing, availability, and conditions
Automatic product matching during CSV imports

Shop-Specific Overrides

{
  "product_id": "LAPTOP123",
  "shop_price": 999.99,
  "shop_currency": "EUR",
  "shop_availability": "in stock",
  "shop_condition": "new",
  "is_featured": true,
  "min_quantity": 1,
  "max_quantity": 5
}

Advanced Product Search

# Search products in specific shop
GET /products?shop_code=TECHSTORE&search=laptop

# Search across all verified shops
GET /products?search=laptop&availability=in%20stock

# Filter by brand and category
GET /products?brand=Apple&category=Electronics

Database Schema

Core Tables

Users Table

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    email VARCHAR UNIQUE NOT NULL,
    username VARCHAR UNIQUE NOT NULL,
    hashed_password VARCHAR NOT NULL,
    role VARCHAR DEFAULT 'user',
    is_active BOOLEAN DEFAULT true,
    last_login TIMESTAMP,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW()
);

Shops Table

CREATE TABLE shops (
    id SERIAL PRIMARY KEY,
    shop_code VARCHAR UNIQUE NOT NULL,
    shop_name VARCHAR NOT NULL,
    description TEXT,
    owner_id INTEGER REFERENCES users(id),
    contact_email VARCHAR,
    contact_phone VARCHAR,
    website VARCHAR,
    business_address TEXT,
    tax_number VARCHAR,
    is_active BOOLEAN DEFAULT true,
    is_verified BOOLEAN DEFAULT false,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW()
);

Products Table

Main marketplace catalog with Google Shopping compatibility
Indexed fields: gtin, brand, google_product_category, availability
Supports all Google Shopping feed attributes

ShopProducts Table

CREATE TABLE shop_products (
    id SERIAL PRIMARY KEY,
    shop_id INTEGER REFERENCES shops(id),
    product_id INTEGER REFERENCES products(id),
    shop_product_id VARCHAR,
    shop_price DECIMAL,
    shop_sale_price DECIMAL,
    shop_currency VARCHAR,
    shop_availability VARCHAR,
    shop_condition VARCHAR,
    is_featured BOOLEAN DEFAULT false,
    is_active BOOLEAN DEFAULT true,
    min_quantity INTEGER DEFAULT 1,
    max_quantity INTEGER,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW(),
    UNIQUE(shop_id, product_id)
);

Stock Table

Location-based inventory tracking with optional shop association
GTIN-based product linking
Support for reserved quantities (for order processing)

Import Jobs Table

Track background import operations per shop
User and shop ownership tracking
Status monitoring and error handling

Advanced Features

Shop-Specific CSV Import

Import products with automatic shop association:

import requests

# Start import for specific shop
response = requests.post(
    'http://localhost:8000/import-csv',
    headers={'Authorization': 'Bearer YOUR_TOKEN'},
    json={
        'url': 'https://myshop.com/products.csv',
        'shop_code': 'MYSHOP',
        'batch_size': 1000
    }
)

job_id = response.json()['job_id']

# Monitor progress
status_response = requests.get(
    f'http://localhost:8000/import-status/{job_id}',
    headers={'Authorization': 'Bearer YOUR_TOKEN'}
)
print(status_response.json())

Multi-Shop Product Management

# Add existing marketplace product to your shop
curl -X POST "http://localhost:8000/shops/MYSHOP/products" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "product_id": "EXISTING_PRODUCT_123",
       "shop_price": 89.99,
       "shop_availability": "in stock",
       "is_featured": true
     }'

# Get products from specific shop
curl -X GET "http://localhost:8000/shops/MYSHOP/products" \
     -H "Authorization: Bearer YOUR_TOKEN"

Stock Management with Shop Context

# Set shop-specific stock
curl -X POST "http://localhost:8000/stock" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "gtin": "1234567890123",
       "location": "MYSHOP_WAREHOUSE",
       "quantity": 50,
       "shop_code": "MYSHOP"
     }'

Production Deployment

Security Checklist for Marketplace

Change default admin password immediately
Set strong JWT_SECRET_KEY (32+ random characters)
Configure JWT_EXPIRE_MINUTES appropriately
Set up HTTPS/TLS termination
Configure CORS for your frontend domains only
Set up database connection limits and pooling
Enable request logging and monitoring
Configure rate limiting per your needs
Set up shop verification workflow
Implement shop quality monitoring
Set up automated backup for shop data
Configure email notifications for shop owners
Regular security audits of user accounts and shops

Environment Variables for Production

# Security
JWT_SECRET_KEY=your-very-long-random-secret-key-at-least-32-characters
JWT_EXPIRE_MINUTES=30

# Database (use PostgreSQL in production)
DATABASE_URL=postgresql://user:password@db-host:5432/letzshop_prod

# Server
DEBUG=False
API_HOST=0.0.0.0
API_PORT=8000

# Marketplace Configuration
MARKETPLACE_NAME=Letzshop
DEFAULT_CURRENCY=EUR
ADMIN_EMAIL=admin@letzshop.lu

# Optional: External services
REDIS_URL=redis://redis-host:6379/0
EMAIL_API_KEY=your-email-service-key

Docker Deployment

# docker-compose.yml
version: '3.8'
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: letzshop
      POSTGRES_USER: letzshop_user
      POSTGRES_PASSWORD: secure_password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  api:
    build: .
    environment:
      DATABASE_URL: postgresql://letzshop_user:secure_password@db:5432/letzshop
      JWT_SECRET_KEY: your-production-secret-key
      JWT_EXPIRE_MINUTES: 30
      MARKETPLACE_NAME: Letzshop
    ports:
      - "8000:8000"
    depends_on:
      - db
    restart: unless-stopped

volumes:
  postgres_data:

Troubleshooting

Marketplace-Specific Issues

Shop Import Failures:

Verify shop exists and is active
Check user permissions for the shop
Ensure CSV format is compatible
Monitor import job status for detailed errors

Shop Product Association:

Products are added to main catalog first
Shop-product relationships created automatically during import
Check shop_products table for associations

Permission Issues:

Shop owners can only manage their own shops
Admin can manage all shops and users
Verify user role and shop ownership

Common API Issues

Shop Not Found Errors:

Check shop_code spelling and case (stored uppercase)
Verify shop is active and verified (for public access)
Check user permissions for shop access

CSV Import with Shop Code:

Shop code is required for all imports
Shop must exist before importing
User must have permission to import for that shop

Migration Guide

From v2.0 to v2.1 (Marketplace Update)

Backup existing data
Update dependencies: pip install -r requirements.txt
Update environment variables (add shop-related configs)
Run application - new tables will be created automatically
Existing products remain in main catalog
Create shops for existing users
Update client applications to use shop-specific endpoints

Data Migration Script Example

# Migrate existing products to demo shop
from models.database_models import Product, Shop, ShopProduct
from sqlalchemy.orm import Session

def migrate_to_shops(db: Session):
    demo_shop = db.query(Shop).filter(Shop.shop_code == "DEMOSHOP").first()
    products = db.query(Product).all()
    
    for product in products:
        shop_product = ShopProduct(
            shop_id=demo_shop.id,
            product_id=product.id,
            shop_price=product.price,
            shop_availability=product.availability,
            is_active=True
        )
        db.add(shop_product)
    
    db.commit()

Contributing

Fork the repository
Create a feature branch: git checkout -b feature-name
Make changes with proper tests
Run security and quality checks
Update documentation if needed
Submit a pull request

Code Quality Standards

All endpoints must have proper authentication and authorization
Shop ownership verification for protected operations
Input validation using Pydantic models
Comprehensive error handling with meaningful messages
Unit tests for marketplace functionality

License

This project is licensed under the MIT License - see the LICENSE file for details.

About Letzshop

Letzshop is Luxembourg's premier e-commerce marketplace, connecting local and international vendors with Luxembourg customers. Our platform supports multi-vendor operations with advanced inventory management and seamless CSV import capabilities.

Support

For marketplace-specific issues and vendor onboarding:

Check the troubleshooting section above
Review existing GitHub issues
Create a new issue with detailed information including:
- Shop code and user information
- CSV format and import details
- Error messages and logs
- Environment configuration (without secrets)

For vendor support: vendor-support@letzshop.lu
For technical issues: tech-support@letzshop.lu

19 KiB Raw Blame History