orion/comprehensive_readme.md

Ecommerce Backend API with Marketplace Support

A comprehensive FastAPI-based product management system with JWT authentication, marketplace-aware CSV import/export, multi-shop support, and advanced stock management capabilities.

Features

• JWT Authentication - Secure user registration, login, and role-based access control
• Marketplace Integration - Support for multiple marketplaces (Letzshop, Amazon, eBay, Etsy, Shopify, etc.)
• Multi-Shop Management - Shop creation, ownership validation, and product catalog management
• Advanced Product Management - GTIN validation, price processing, and comprehensive filtering
• Stock Management - Multi-location inventory tracking with add/remove/set operations
• CSV Import/Export - Background processing of marketplace CSV files with progress tracking
• Rate Limiting - Built-in request rate limiting for API protection
• Admin Panel - Administrative functions for user and shop management
• Statistics & Analytics - Comprehensive reporting on products, marketplaces, and inventory

Technology Stack

• FastAPI - Modern, fast web framework for building APIs
• SQLAlchemy - SQL toolkit and Object-Relational Mapping (ORM)
• PostgreSQL - Primary database (SQLite supported for development)
• JWT - JSON Web Tokens for secure authentication
• Pydantic - Data validation using Python type annotations
• Pandas - Data processing for CSV operations
• bcrypt - Password hashing
• Pytest - Comprehensive testing framework

Directory Structure

letzshop_api/
├── main.py                     # FastAPI application entry point
├── app/
│   ├── core/
│   │   ├── config.py           # Configuration settings
│   │   ├── database.py         # Database setup
│   │   └── lifespan.py         # App lifecycle management
│   ├── api/
│   │   ├── deps.py             # Common dependencies
│   │   ├── main.py             # API router setup
│   │   └── v1/                 # API version 1 routes
│   │       ├── auth.py         # Authentication endpoints
│   │       ├── products.py     # Product management
│   │       ├── stock.py        # Stock operations
│   │       ├── shops.py        # Shop management
│   │       ├── marketplace.py  # Marketplace imports
│   │       ├── admin.py        # Admin functions
│   │       └── stats.py        # Statistics
│   ├── services/               # Business logic layer
│   └── tasks/                  # Background task processing
├── models/
│   ├── database_models.py      # SQLAlchemy ORM models
│   └── api_models.py          # Pydantic API models
├── utils/
│   ├── data_processing.py     # GTIN and price processing
│   ├── csv_processor.py       # CSV import/export
│   └── database.py           # Database utilities
├── middleware/
│   ├── auth.py               # JWT authentication
│   ├── rate_limiter.py       # Rate limiting
│   ├── error_handler.py      # Error handling
│   └── logging_middleware.py # Request logging
├── tests/                    # Comprehensive test suite
└── requirements.txt          # Dependencies

Quick Start

1. Installation

# Clone the repository
git clone <repository-url>
cd letzshop_api

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Environment Configuration

Create a .env file in the project root:

# Database Configuration
DATABASE_URL=postgresql://user:password@localhost/ecommerce
# For development, you can use SQLite:
# DATABASE_URL=sqlite:///./ecommerce.db

# JWT Configuration
SECRET_KEY=your-super-secret-key-change-in-production
ACCESS_TOKEN_EXPIRE_MINUTES=30

# API Configuration
ALLOWED_HOSTS=["*"]
RATE_LIMIT_REQUESTS=100
RATE_LIMIT_WINDOW=3600

# Application Settings
PROJECT_NAME="Ecommerce Backend API"
VERSION="2.2.0"
DEBUG=True

3. Database Setup

# The application will automatically create tables on startup
# For production, consider using Alembic for migrations

# Install PostgreSQL (if using PostgreSQL)
# Create database
createdb ecommerce

# Run the application (tables will be created automatically)
python main.py

4. Running the Application

# Development server
python main.py

# Or using uvicorn directly
uvicorn main:app --reload --host 0.0.0.0 --port 8000

The API will be available at:

• API Documentation: http://localhost:8000/docs (Swagger UI)
• Alternative Docs: http://localhost:8000/redoc
• Health Check: http://localhost:8000/health

API Usage

Authentication

Register a new user

curl -X POST "http://localhost:8000/api/v1/auth/register" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "username": "testuser",
    "password": "securepassword123"
  }'

Login

curl -X POST "http://localhost:8000/api/v1/auth/login" \
  -H "Content-Type: application/json" \
  -d '{
    "username": "testuser",
    "password": "securepassword123"
  }'

Use JWT Token

# Get token from login response and use in subsequent requests
curl -X GET "http://localhost:8000/api/v1/products" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Product Management

Create a product

curl -X POST "http://localhost:8000/api/v1/products" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "product_id": "PROD001",
    "title": "Amazing Product",
    "description": "An amazing product description",
    "price": "29.99",
    "currency": "EUR",
    "brand": "BrandName",
    "gtin": "1234567890123",
    "availability": "in stock",
    "marketplace": "Letzshop",
    "shop_name": "MyShop"
  }'

Get products with filtering

# Get all products
curl -X GET "http://localhost:8000/api/v1/products" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Filter by marketplace
curl -X GET "http://localhost:8000/api/v1/products?marketplace=Amazon&limit=50" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Search products
curl -X GET "http://localhost:8000/api/v1/products?search=Amazing&brand=BrandName" \
  -H "Authorization: Bearer YOUR_TOKEN"

Stock Management

Set stock quantity

curl -X POST "http://localhost:8000/api/v1/stock" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 100
  }'

Add stock

curl -X POST "http://localhost:8000/api/v1/stock/add" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 25
  }'

Check stock levels

curl -X GET "http://localhost:8000/api/v1/stock/1234567890123" \
  -H "Authorization: Bearer YOUR_TOKEN"

Marketplace Import

Import products from CSV

curl -X POST "http://localhost:8000/api/v1/marketplace/import-from-marketplace" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products.csv",
    "marketplace": "Amazon",
    "shop_code": "MYSHOP",
    "batch_size": 1000
  }'

Check import status

curl -X GET "http://localhost:8000/api/v1/marketplace/marketplace-import-status/1" \
  -H "Authorization: Bearer YOUR_TOKEN"

Export Data

Export products to CSV

# Export all products
curl -X GET "http://localhost:8000/api/v1/export-csv" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o products_export.csv

# Export with filters
curl -X GET "http://localhost:8000/api/v1/export-csv?marketplace=Amazon&shop_name=MyShop" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o amazon_products.csv

CSV Import Format

The system supports CSV imports with the following headers:

Required Fields

• product_id - Unique product identifier
• title - Product title

Optional Fields

• description - Product description
• link - Product URL
• image_link - Product image URL
• availability - Stock availability (in stock, out of stock, preorder)
• price - Product price
• currency - Price currency (EUR, USD, etc.)
• brand - Product brand
• gtin - Global Trade Item Number (EAN/UPC)
• google_product_category - Product category
• marketplace - Source marketplace
• shop_name - Shop/seller name

Example CSV

product_id,title,description,price,currency,brand,gtin,marketplace,shop_name
PROD001,"Amazing Widget","The best widget ever",29.99,EUR,WidgetCorp,1234567890123,Letzshop,MyShop
PROD002,"Super Gadget","A fantastic gadget",19.99,EUR,GadgetInc,9876543210987,Amazon,TechStore

Testing

Run Tests

# Install test dependencies
pip install -r tests/requirements_test.txt

# Run all tests
pytest tests/ -v

# Run with coverage report
pytest tests/ --cov=app --cov=models --cov=utils --cov-report=html

# Run specific test categories
pytest tests/ -m unit -v                    # Unit tests only
pytest tests/ -m integration -v             # Integration tests only
pytest tests/ -m "not slow" -v              # Fast tests only

# Run specific test files
pytest tests/test_auth.py -v                # Authentication tests
pytest tests/test_products.py -v            # Product tests
pytest tests/test_stock.py -v               # Stock management tests

Test Coverage

The test suite includes:

• Unit Tests - Individual component testing
• Integration Tests - Full workflow testing
• Security Tests - Authentication, authorization, input validation
• Performance Tests - Load testing with large datasets
• Error Handling Tests - Edge cases and error conditions

API Reference

Authentication Endpoints

• POST /api/v1/auth/register - Register new user
• POST /api/v1/auth/login - Login user
• GET /api/v1/auth/me - Get current user info

Product Endpoints

• GET /api/v1/products - List products with filtering
• POST /api/v1/products - Create new product
• GET /api/v1/products/{product_id} - Get specific product
• PUT /api/v1/products/{product_id} - Update product
• DELETE /api/v1/products/{product_id} - Delete product

Stock Endpoints

• POST /api/v1/stock - Set stock quantity
• POST /api/v1/stock/add - Add to stock
• POST /api/v1/stock/remove - Remove from stock
• GET /api/v1/stock/{gtin} - Get stock by GTIN
• GET /api/v1/stock/{gtin}/total - Get total stock
• GET /api/v1/stock - List all stock entries

Shop Endpoints

• POST /api/v1/shops - Create new shop
• GET /api/v1/shops - List shops
• GET /api/v1/shops/{shop_code} - Get specific shop

Marketplace Endpoints

• POST /api/v1/marketplace/import-from-marketplace - Start CSV import
• GET /api/v1/marketplace/marketplace-import-status/{job_id} - Check import status
• GET /api/v1/marketplace/marketplace-import-jobs - List import jobs

Statistics Endpoints

• GET /api/v1/stats - Get general statistics
• GET /api/v1/stats/marketplace-stats - Get marketplace statistics

Admin Endpoints (Admin only)

• GET /api/v1/admin/users - List all users
• PUT /api/v1/admin/users/{user_id}/status - Toggle user status
• GET /api/v1/admin/shops - List all shops
• PUT /api/v1/admin/shops/{shop_id}/verify - Verify/unverify shop

Database Schema

Core Tables

• users - User accounts and authentication
• products - Product catalog with marketplace info
• stock - Inventory tracking by location and GTIN
• shops - Shop/seller information
• shop_products - Shop-specific product settings
• marketplace_import_jobs - Background import job tracking

Key Relationships

• Users own shops (one-to-many)
• Products belong to marketplaces and shops
• Stock entries are linked to products via GTIN
• Import jobs track user-initiated imports

Security Features

• JWT Authentication - Secure token-based authentication
• Password Hashing - bcrypt for secure password storage
• Role-Based Access - User and admin role separation
• Rate Limiting - Protection against API abuse
• Input Validation - Comprehensive data validation
• SQL Injection Protection - Parameterized queries
• CORS Configuration - Cross-origin request handling

Performance Optimizations

• Database Indexing - Strategic indexes on key columns
• Pagination - Efficient data retrieval with skip/limit
• Streaming Responses - Memory-efficient CSV exports
• Background Processing - Async import job handling
• Connection Pooling - Efficient database connections
• Query Optimization - Optimized database queries

Deployment

Docker Deployment

FROM python:3.11

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Production Considerations

• Use PostgreSQL for production database
• Set strong SECRET_KEY in environment
• Configure proper CORS settings
• Enable HTTPS
• Set up monitoring and logging
• Use a reverse proxy (nginx)
• Configure database connection pooling
• Set up backup strategies

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass
6. Submit a pull request

Development Setup

# Install development dependencies
pip install -r requirements_dev.txt

# Run pre-commit hooks
pre-commit install

# Run linting
flake8 app/ models/ utils/
black app/ models/ utils/

# Run type checking
mypy app/

Support & Documentation

• API Documentation: http://localhost:8000/docs
• Health Check: http://localhost:8000/health
• Version Info: http://localhost:8000/

For issues and feature requests, please create an issue in the repository.

License

[Specify your license here]

Changelog

v2.2.0

• Added marketplace-aware product import
• Implemented multi-shop support
• Enhanced stock management with location tracking
• Added comprehensive test suite
• Improved authentication and authorization

v2.1.0

• Added JWT authentication
• Implemented role-based access control
• Added CSV import/export functionality

v2.0.0

• Complete rewrite with FastAPI
• Added PostgreSQL support
• Implemented comprehensive API documentation