orion/README.md

Ecommerce Backend API with Marketplace Support

A comprehensive FastAPI-based product management system with JWT authentication, marketplace-aware CSV import/export, multi-shop support, and advanced inventory management capabilities.

Overview

This FastAPI application provides a complete ecommerce backend solution designed for marketplace operations. The system supports multiple stores (shops), centralized product catalogs, advanced inventory management, and automated CSV processing with background job handling.

Key Features

• JWT Authentication - Secure user registration, login, and role-based access control
• Marketplace Integration - Support for multiple marketplaces (Letzshop, Amazon, eBay, Etsy, Shopify, etc.)
• Multi-Shop Management - Shop creation, ownership validation, and product catalog management
• Advanced MarketplaceProduct Management - GTIN validation, price processing, and comprehensive filtering
• Inventory Management - Multi-location inventory tracking with add/remove/set operations
• CSV Import/Export - Background processing of marketplace CSV files with progress tracking
• Rate Limiting - Built-in request rate limiting for API protection
• Admin Panel - Administrative functions for user and shop management
• Statistics & Analytics - Comprehensive reporting on products, marketplaces, and inventory

Technology Stack

• FastAPI - Modern, fast web framework for building APIs
• SQLAlchemy - SQL toolkit and Object-Relational Mapping (ORM)
• PostgreSQL - Primary database (SQLite supported for development)
• JWT - JSON Web Tokens for secure authentication
• Pydantic - Data validation using Python type annotations
• Pandas - Data processing for CSV operations
• bcrypt - Password hashing
• Pytest - Comprehensive testing framework

Architecture

Project Structure

orion/
├── main.py                     # FastAPI application entry point
├── app/
│   ├── core/
│   │   ├── config.py           # Configuration settings
│   │   ├── database.py         # Database setup
│   │   └── lifespan.py         # App lifecycle management
│   ├── api/
│   │   ├── deps.py             # Common dependencies
│   │   ├── main.py             # API router setup
│   │   └── v1/                 # API version 1 routes
│   │       ├── auth.py         # Authentication endpoints
│   │       ├── products.py     # MarketplaceProduct management
│   │       ├── inventory.py        # Inventory operations
│   │       ├── shops.py        # Shop management
│   │       ├── marketplace.py  # Marketplace imports
│   │       ├── admin.py        # Admin functions
│   │       └── stats.py        # Statistics
│   ├── services/               # Business logic layer
│   └── tasks/                  # Background task processing
├── models/
│   ├── database/
│   │   ├── __init__.py           # Import all models for easy access
│   │   ├── base.py              # Base model class and common mixins
│   │   ├── user.py              # User, UserProfile models
│   │   ├── auth.py              # Authentication-related models
│   │   ├── product.py           # MarketplaceProduct, ProductVariant models
│   │   ├── inventory.py             # Inventory, InventoryMovement models
│   │   ├── shop.py              # Shop, ShopLocation models
│   │   ├── marketplace.py       # Marketplace integration models
│   │   └── admin.py             # Admin-specific models
│   └── api/
│       ├── __init__.py          # Common imports
│       ├── base.py              # Base Pydantic models
│       ├── auth.py              # Login, Token, User response models
│       ├── product.py           # MarketplaceProduct request/response models
│       ├── inventory.py             # Inventory operation models
│       ├── shop.py              # Shop management models
│       ├── marketplace.py       # Marketplace import models
│       ├── admin.py             # Admin operation models
│       └── stats.py             # Statistics response models
├── utils/
│   ├── data_processing.py     # GTIN and price processing
│   ├── csv_processor.py       # CSV import/export
│   └── database.py           # Database utilities
├── middleware/
│   ├── auth.py               # JWT authentication
│   ├── rate_limiter.py       # Rate limiting
│   ├── error_handler.py      # Error handling
│   └── logging_middleware.py # Request logging
├── tests/                    # Comprehensive test suite
└── requirements.txt          # Dependencies

Service Layer Architecture

The application follows a Service Layer Architecture pattern that separates HTTP concerns from business logic:

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   HTTP Client   │    │   FastAPI App   │    │    Database     │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         │ HTTP Request           │                       │
         ├─────────────────────► │                       │
         │                       │                       │
         │              ┌─────────────────┐               │
         │              │    Router       │               │
         │              │  (HTTP Layer)   │               │
         │              └─────────────────┘               │
         │                       │                       │
         │                       │ Delegates to          │
         │                       ▼                       │
         │              ┌─────────────────┐               │
         │              │    Service      │               │
         │              │ (Business Layer)│               │
         │              └─────────────────┘               │
         │                       │                       │
         │                       │ Database Operations   │
         │                       ├─────────────────────► │
         │                       │                       │
         │ HTTP Response          │                       │
         ◄─────────────────────── │                       │

Development Tools

This project includes a comprehensive Makefile that provides all necessary development commands. Here are the key workflows:

Quick Start Commands

# First-time setup (installs dependencies and sets up database)
make setup

# Start development environment (API + docs)
make dev-full

# Start API server only
make dev

# Start documentation server only
make docs

Installation Options

# Install all dependencies (production + development + testing + docs)
make install-all

# Install only production dependencies
make install

# Install only development dependencies
make install-dev

# Install only testing dependencies
make install-test

# Install only documentation dependencies
make install-docs

Development Workflow

# Format code (black + isort)
make format

# Run linting (flake8 + mypy)
make lint

# Format and lint together
make check

# Complete CI pipeline (format + lint + test with coverage)
make ci

# Quality assurance check (format + lint + test + docs check)
make qa

# Clone the repository
git clone <orion-repo>
cd orion-repo

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Complete setup (recommended)
make setup

# Or install dependencies manually
make install-all

The make setup command will install all dependencies and set up the database automatically.

Environment Configuration

Create a .env file in the project root:

# Database Configuration
DATABASE_URL=postgresql://user:password@localhost/ecommerce
# For development, you can use SQLite:
# DATABASE_URL=sqlite:///./ecommerce.db

# JWT Configuration
SECRET_KEY=your-super-secret-key-change-in-production
ACCESS_TOKEN_EXPIRE_MINUTES=30

# API Configuration
ALLOWED_HOSTS=["*"]
RATE_LIMIT_REQUESTS=100
RATE_LIMIT_WINDOW=3600

# Application Settings
PROJECT_NAME="Ecommerce Backend API"
VERSION="2.2.0"
DEBUG=True

Database Setup

The application will automatically create tables on startup. For production, consider using Alembic for migrations.

# For PostgreSQL - create database first
createdb ecommerce

# Run database migrations
make migrate-up

# Reset database if needed
make migrate-reset

Running the Application

# Development server
make dev

# Development server with documentation
make dev-full

# Production server (manual)
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4

The make dev-full command starts both the API server and documentation server simultaneously.

The API will be available at:

• API Documentation: http://localhost:8000/docs (Swagger UI)
• Alternative Docs: http://localhost:8000/redoc
• Health Check: http://localhost:8000/health
• MkDocs Documentation: http://localhost:8001 (when using make dev-full)

Authentication

User Registration

curl -X POST "http://localhost:8000/api/v1/auth/register" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "username": "testuser",
    "password": "securepassword123"
  }'

User Login

curl -X POST "http://localhost:8000/api/v1/auth/login" \
  -H "Content-Type: application/json" \
  -d '{
    "username": "testuser",
    "password": "securepassword123"
  }'

Using JWT Token

# Get token from login response and use in subsequent requests
curl -X GET "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Core Features

MarketplaceProduct Management

Create a MarketplaceProduct

curl -X POST "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "marketplace_product_id": "PROD001",
    "title": "Amazing MarketplaceProduct",
    "description": "An amazing product description",
    "price": "29.99",
    "currency": "EUR",
    "brand": "BrandName",
    "gtin": "1234567890123",
    "availability": "in inventory",
    "marketplace": "Letzshop",
    "shop_name": "MyShop"
  }'

Search and Filter Products

# Get all products
curl -X GET "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Filter by marketplace
curl -X GET "http://localhost:8000/api/v1/marketplace/product?marketplace=Amazon&limit=50" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Search products
curl -X GET "http://localhost:8000/api/v1/marketplace/product?search=Amazing&brand=BrandName" \
  -H "Authorization: Bearer YOUR_TOKEN"

Inventory Management

Set Inventory Quantity

curl -X POST "http://localhost:8000/api/v1/inventory" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 100
  }'

Add Inventory

curl -X POST "http://localhost:8000/api/v1/inventory/add" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 25
  }'

Check Inventory Levels

curl -X GET "http://localhost:8000/api/v1/inventory/1234567890123" \
  -H "Authorization: Bearer YOUR_TOKEN"

Marketplace Import

Import Products from CSV

curl -X POST "http://localhost:8000/api/v1/marketplace/import-product" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products.csv",
    "marketplace": "Amazon",
    "shop_code": "MYSHOP",
    "batch_size": 1000
  }'

Check Import Status

curl -X GET "http://localhost:8000/api/v1/marketplace/import-status/1" \
  -H "Authorization: Bearer YOUR_TOKEN"

Export Data

Export Products to CSV

# Export all products
curl -X GET "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o products_export.csv

# Export with filters
curl -X GET "http://localhost:8000/api/v1/marketplace/product?marketplace=Amazon&shop_name=MyShop" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o amazon_products.csv

CSV Import Format

The system supports CSV imports with the following headers:

Required Fields

• product_id - Unique product identifier
• title - MarketplaceProduct title

Optional Fields

• description - MarketplaceProduct description
• link - MarketplaceProduct URL
• image_link - MarketplaceProduct image URL
• availability - Inventory availability (in inventory, out of inventory, preorder)
• price - MarketplaceProduct price
• currency - Price currency (EUR, USD, etc.)
• brand - MarketplaceProduct brand
• gtin - Global Trade Item Number (EAN/UPC)
• google_product_category - MarketplaceProduct category
• marketplace - Source marketplace
• shop_name - Shop/seller name

Example CSV

product_id,title,description,price,currency,brand,gtin,marketplace,shop_name
PROD001,"Amazing Widget","The best widget ever",29.99,EUR,WidgetCorp,1234567890123,Letzshop,MyShop
PROD002,"Super Gadget","A fantastic gadget",19.99,EUR,GadgetInc,9876543210987,Amazon,TechStore

API Reference

Authentication Endpoints

• POST /api/v1/auth/register - Register new user
• POST /api/v1/auth/login - Login user
• GET /api/v1/auth/me - Get current user info

Marketplace Endpoints

• GET /api/v1/marketplace/product - List marketplace products with filtering
• POST /api/v1/marketplace/product - Create new marketplace product
• GET /api/v1/marketplace/product/{product_id} - Get specific marketplace product
• PUT /api/v1/marketplace/product/{product_id} - Update marketplace product
• DELETE /api/v1/marketplace/product/{product_id} - Delete marketplace product
• POST /api/v1/marketplace/import-product - Start CSV import
• GET /api/v1/marketplace/import-status/{job_id} - Check import status
• GET /api/v1/marketplace/import-jobs - List import jobs

Inventory Endpoints

• POST /api/v1/inventory - Set inventory quantity
• POST /api/v1/inventory/add - Add to inventory
• POST /api/v1/inventory/remove - Remove from inventory
• GET /api/v1/inventory/{gtin} - Get inventory by GTIN
• GET /api/v1/inventory/{gtin}/total - Get total inventory
• GET /api/v1/inventory - List all inventory entries

Shop Endpoints

• POST /api/v1/shop - Create new shop
• GET /api/v1/shop - List shops
• GET /api/v1/shop/{shop_code} - Get specific shop

Statistics Endpoints

• GET /api/v1/stats - Get general statistics
• GET /api/v1/stats/marketplace-stats - Get marketplace statistics

Admin Endpoints (Admin only)

• GET /api/v1/admin/users - List all users
• PUT /api/v1/admin/users/{user_id}/status - Toggle user status
• GET /api/v1/admin/shops - List all shops
• PUT /api/v1/admin/shops/{shop_id}/verify - Verify/unverify shop

Documentation

This project uses MkDocs for documentation. The Makefile provides comprehensive documentation management commands.

Documentation Development

# Start documentation development server
make docs

# Start both API and documentation servers
make dev-full

# Build documentation site
make docs-build

# Check documentation for issues
make docs-check

# Update docs with latest API information
make docs-update

Documentation Deployment

# Deploy to GitHub Pages
make docs-deploy

# Force deploy to GitHub Pages
make docs-deploy-force

# Build and deploy in one command
make docs-publish

# Preview built documentation locally
make docs-preview

Documentation Maintenance

# Setup documentation environment
make setup-docs

# Clean documentation build files
make docs-clean

# Get help with MkDocs commands
make docs-help

Database Schema

Core Tables

• users - User accounts and authentication
• products - MarketplaceProduct catalog with marketplace info
• inventory - Inventory tracking by location and GTIN
• shops - Shop/seller information
• shop_products - Shop-specific product settings
• marketplace_import_jobs - Background import job tracking

Key Relationships

• Users own shops (one-to-many)
• Products belong to marketplaces and shops
• Inventory entries are linked to products via GTIN
• Import jobs track user-initiated imports

Security Features

• JWT Authentication - Secure token-based authentication
• Password Hashing - bcrypt for secure password storage
• Role-Based Access - User and admin role separation
• Rate Limiting - Protection against API abuse
• Input Validation - Comprehensive data validation
• SQL Injection Protection - Parameterized queries
• CORS Configuration - Cross-origin request handling

Testing

This project includes a comprehensive test suite with multiple testing strategies.

Running Tests

# Run all tests
make test

# Run tests with coverage report
make test-coverage

# Run only unit tests
make test-unit

# Run only integration tests
make test-integration

# Run fast tests (exclude slow ones)
make test-fast

# Run slow tests only
make test-slow

Test Specific Components

# Authentication tests
make test-auth

# MarketplaceProduct management tests
make test-products

# Inventory management tests
make test-inventory

# Marketplace functionality tests
make test-marketplace

# Admin functionality tests
make test-admin

Test Setup

# Install test dependencies only
make setup-test

# Complete test workflow (format + lint + coverage)
make full-test

Test Coverage

The test suite includes:

• Unit Tests - Individual component testing
• Integration Tests - Full workflow testing
• Security Tests - Authentication, authorization, input validation
• Performance Tests - Load testing with large datasets
• Error Handling Tests - Edge cases and error conditions

Deployment

Development Deployment

# Docker deployment
make docker-up

# Staging deployment
make deploy-staging

# Production deployment
make deploy-prod

Cleanup and Maintenance

# Clean test artifacts and cache files
make clean

# Clean all build artifacts (including documentation)
make clean-all

# Release readiness check
make release-check

Performance Optimizations

• Database Indexing - Strategic indexes on key columns
• Pagination - Efficient data retrieval with skip/limit
• Streaming Responses - Memory-efficient CSV exports
• Background Processing - Async import job handling
• Connection Pooling - Efficient database connections
• Query Optimization - Optimized database queries

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run quality assurance checks
5. Ensure all tests pass
6. Submit a pull request

Development Setup for Contributors

# Complete development setup
make setup

# Install only development dependencies
make install-dev

# Run quality assurance pipeline
make qa

# Run complete CI pipeline
make ci

Code Quality Standards

# Format code automatically
make format

# Check code quality
make check

# Run full test suite
make full-test

# Prepare for release
make release-check

Available Make Commands

For a complete list of available commands, run:

make help

This will display all available commands organized by category:

• Setup: Installation and environment setup
• Development: Development servers and workflows
• Documentation: Documentation building and deployment
• Testing: Various test execution options
• Code Quality: Formatting, linting, and quality checks
• Database: Migration and database management
• Docker: Container management
• Cleanup: Artifact removal and maintenance
• Workflows: Combined command sequences

Changelog

v2.2.0

• Added marketplace-aware product import
• Implemented multi-shop support
• Enhanced inventory management with location tracking
• Added comprehensive test suite
• Improved authentication and authorization

v2.1.0

• Added JWT authentication
• Implemented role-based access control
• Added CSV import/export functionality

v2.0.0

• Complete rewrite with FastAPI
• Added PostgreSQL support
• Implemented comprehensive API documentation

Support & Documentation

• API Documentation: http://localhost:8000/docs
• Health Check: http://localhost:8000/health
• Version Info: http://localhost:8000/

For issues and feature requests, please create an issue in the repository.