orion/README.md

# Ecommerce Backend API with Marketplace Support

A comprehensive FastAPI-based product management system with JWT authentication, marketplace-aware CSV import/export, multi-shop support, and advanced inventory management capabilities.

## Overview

This FastAPI application provides a complete ecommerce backend solution designed for marketplace operations. The system supports multiple stores (shops), centralized product catalogs, advanced inventory management, and automated CSV processing with background job handling.

### Key Features

- **JWT Authentication** - Secure user registration, login, and role-based access control
- **Marketplace Integration** - Support for multiple marketplaces (Letzshop, Amazon, eBay, Etsy, Shopify, etc.)
- **Multi-Shop Management** - Shop creation, ownership validation, and product catalog management
- **Advanced MarketplaceProduct Management** - GTIN validation, price processing, and comprehensive filtering
- **Inventory Management** - Multi-location inventory tracking with add/remove/set operations
- **CSV Import/Export** - Background processing of marketplace CSV files with progress tracking
- **Rate Limiting** - Built-in request rate limiting for API protection
- **Admin Panel** - Administrative functions for user and shop management
- **Statistics & Analytics** - Comprehensive reporting on products, marketplaces, and inventory

### Technology Stack

- **FastAPI** - Modern, fast web framework for building APIs
- **SQLAlchemy** - SQL toolkit and Object-Relational Mapping (ORM)
- **PostgreSQL** - Primary database (SQLite supported for development)
- **JWT** - JSON Web Tokens for secure authentication
- **Pydantic** - Data validation using Python type annotations
- **Pandas** - Data processing for CSV operations
- **bcrypt** - Password hashing
- **Pytest** - Comprehensive testing framework

## Architecture

### Project Structure

```
orion/
├── main.py                     # FastAPI application entry point
├── app/
│   ├── core/
│   │   ├── config.py           # Configuration settings
│   │   ├── database.py         # Database setup
│   │   └── lifespan.py         # App lifecycle management
│   ├── api/
│   │   ├── deps.py             # Common dependencies
│   │   ├── main.py             # API router setup
│   │   └── v1/                 # API version 1 routes
│   │       ├── auth.py         # Authentication endpoints
│   │       ├── products.py     # MarketplaceProduct management
│   │       ├── inventory.py        # Inventory operations
│   │       ├── shops.py        # Shop management
│   │       ├── marketplace.py  # Marketplace imports
│   │       ├── admin.py        # Admin functions
│   │       └── stats.py        # Statistics
│   ├── services/               # Business logic layer
│   └── tasks/                  # Background task processing
├── models/
│   ├── database/
│   │   ├── __init__.py           # Import all models for easy access
│   │   ├── base.py              # Base model class and common mixins
│   │   ├── user.py              # User, UserProfile models
│   │   ├── auth.py              # Authentication-related models
│   │   ├── product.py           # MarketplaceProduct, ProductVariant models
│   │   ├── inventory.py             # Inventory, InventoryMovement models
│   │   ├── shop.py              # Shop, ShopLocation models
│   │   ├── marketplace.py       # Marketplace integration models
│   │   └── admin.py             # Admin-specific models
│   └── api/
│       ├── __init__.py          # Common imports
│       ├── base.py              # Base Pydantic models
│       ├── auth.py              # Login, Token, User response models
│       ├── product.py           # MarketplaceProduct request/response models
│       ├── inventory.py             # Inventory operation models
│       ├── shop.py              # Shop management models
│       ├── marketplace.py       # Marketplace import models
│       ├── admin.py             # Admin operation models
│       └── stats.py             # Statistics response models
├── utils/
│   ├── data_processing.py     # GTIN and price processing
│   ├── csv_processor.py       # CSV import/export
│   └── database.py           # Database utilities
├── middleware/
│   ├── auth.py               # JWT authentication
│   ├── rate_limiter.py       # Rate limiting
│   ├── error_handler.py      # Error handling
│   └── logging_middleware.py # Request logging
├── tests/                    # Comprehensive test suite
└── requirements.txt          # Dependencies
```

### Service Layer Architecture

The application follows a Service Layer Architecture pattern that separates HTTP concerns from business logic:

```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   HTTP Client   │    │   FastAPI App   │    │    Database     │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         │ HTTP Request           │                       │
         ├─────────────────────► │                       │
         │                       │                       │
         │              ┌─────────────────┐               │
         │              │    Router       │               │
         │              │  (HTTP Layer)   │               │
         │              └─────────────────┘               │
         │                       │                       │
         │                       │ Delegates to          │
         │                       ▼                       │
         │              ┌─────────────────┐               │
         │              │    Service      │               │
         │              │ (Business Layer)│               │
         │              └─────────────────┘               │
         │                       │                       │
         │                       │ Database Operations   │
         │                       ├─────────────────────► │
         │                       │                       │
         │ HTTP Response          │                       │
         ◄─────────────────────── │                       │
```

## Development Tools

This project includes a comprehensive Makefile that provides all necessary development commands. Here are the key workflows:

### Quick Start Commands

```bash
# First-time setup (installs dependencies and sets up database)
make setup

# Start development environment (API + docs)
make dev-full

# Start API server only
make dev

# Start documentation server only
make docs
```

### Installation Options

```bash
# Install all dependencies (production + development + testing + docs)
make install-all

# Install only production dependencies
make install

# Install only development dependencies
make install-dev

# Install only testing dependencies
make install-test

# Install only documentation dependencies
make install-docs
```

### Development Workflow

```bash
# Format code (black + isort)
make format

# Run linting (flake8 + mypy)
make lint

# Format and lint together
make check

# Complete CI pipeline (format + lint + test with coverage)
make ci

# Quality assurance check (format + lint + test + docs check)
make qa
```

```bash
# Clone the repository
git clone <orion-repo>
cd orion-repo

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Complete setup (recommended)
make setup

# Or install dependencies manually
make install-all
```

The `make setup` command will install all dependencies and set up the database automatically.

### Environment Configuration

Create a `.env` file in the project root:

```env
# Database Configuration
DATABASE_URL=postgresql://user:password@localhost/ecommerce
# For development, you can use SQLite:
# DATABASE_URL=sqlite:///./ecommerce.db

# JWT Configuration
SECRET_KEY=your-super-secret-key-change-in-production
ACCESS_TOKEN_EXPIRE_MINUTES=30

# API Configuration
ALLOWED_HOSTS=["*"]
RATE_LIMIT_REQUESTS=100
RATE_LIMIT_WINDOW=3600

# Application Settings
PROJECT_NAME="Ecommerce Backend API"
VERSION="2.2.0"
DEBUG=True
```

### Database Setup

The application will automatically create tables on startup. For production, consider using Alembic for migrations.

```bash
# For PostgreSQL - create database first
createdb ecommerce

# Run database migrations
make migrate-up

# Reset database if needed
make migrate-reset
```

### Running the Application

```bash
# Development server
make dev

# Development server with documentation
make dev-full

# Production server (manual)
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
```

The `make dev-full` command starts both the API server and documentation server simultaneously.

The API will be available at:
- **API Documentation**: http://localhost:8000/docs (Swagger UI)
- **Alternative Docs**: http://localhost:8000/redoc
- **Health Check**: http://localhost:8000/health
- **MkDocs Documentation**: http://localhost:8001 (when using `make dev-full`)

## Authentication

### User Registration

```bash
curl -X POST "http://localhost:8000/api/v1/auth/register" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "username": "testuser",
    "password": "securepassword123"
  }'
```

### User Login

```bash
curl -X POST "http://localhost:8000/api/v1/auth/login" \
  -H "Content-Type: application/json" \
  -d '{
    "username": "testuser",
    "password": "securepassword123"
  }'
```

### Using JWT Token

```bash
# Get token from login response and use in subsequent requests
curl -X GET "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
```

## Core Features

### MarketplaceProduct Management

#### Create a MarketplaceProduct

```bash
curl -X POST "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "marketplace_product_id": "PROD001",
    "title": "Amazing MarketplaceProduct",
    "description": "An amazing product description",
    "price": "29.99",
    "currency": "EUR",
    "brand": "BrandName",
    "gtin": "1234567890123",
    "availability": "in inventory",
    "marketplace": "Letzshop",
    "shop_name": "MyShop"
  }'
```

#### Search and Filter Products

```bash
# Get all products
curl -X GET "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Filter by marketplace
curl -X GET "http://localhost:8000/api/v1/marketplace/product?marketplace=Amazon&limit=50" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Search products
curl -X GET "http://localhost:8000/api/v1/marketplace/product?search=Amazing&brand=BrandName" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Inventory Management

#### Set Inventory Quantity

```bash
curl -X POST "http://localhost:8000/api/v1/inventory" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 100
  }'
```

#### Add Inventory

```bash
curl -X POST "http://localhost:8000/api/v1/inventory/add" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 25
  }'
```

#### Check Inventory Levels

```bash
curl -X GET "http://localhost:8000/api/v1/inventory/1234567890123" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Marketplace Import

#### Import Products from CSV

```bash
curl -X POST "http://localhost:8000/api/v1/marketplace/import-product" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products.csv",
    "marketplace": "Amazon",
    "shop_code": "MYSHOP",
    "batch_size": 1000
  }'
```

#### Check Import Status

```bash
curl -X GET "http://localhost:8000/api/v1/marketplace/import-status/1" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Export Data

#### Export Products to CSV

```bash
# Export all products
curl -X GET "http://localhost:8000/api/v1/marketplace/product" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o products_export.csv

# Export with filters
curl -X GET "http://localhost:8000/api/v1/marketplace/product?marketplace=Amazon&shop_name=MyShop" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o amazon_products.csv
```

## CSV Import Format

The system supports CSV imports with the following headers:

### Required Fields
- `product_id` - Unique product identifier
- `title` - MarketplaceProduct title

### Optional Fields
- `description` - MarketplaceProduct description
- `link` - MarketplaceProduct URL
- `image_link` - MarketplaceProduct image URL
- `availability` - Inventory availability (in inventory, out of inventory, preorder)
- `price` - MarketplaceProduct price
- `currency` - Price currency (EUR, USD, etc.)
- `brand` - MarketplaceProduct brand
- `gtin` - Global Trade Item Number (EAN/UPC)
- `google_product_category` - MarketplaceProduct category
- `marketplace` - Source marketplace
- `shop_name` - Shop/seller name

### Example CSV
```csv
product_id,title,description,price,currency,brand,gtin,marketplace,shop_name
PROD001,"Amazing Widget","The best widget ever",29.99,EUR,WidgetCorp,1234567890123,Letzshop,MyShop
PROD002,"Super Gadget","A fantastic gadget",19.99,EUR,GadgetInc,9876543210987,Amazon,TechStore
```

## API Reference

### Authentication Endpoints
- `POST /api/v1/auth/register` - Register new user
- `POST /api/v1/auth/login` - Login user
- `GET /api/v1/auth/me` - Get current user info

### Marketplace Endpoints
- `GET /api/v1/marketplace/product` - List marketplace products with filtering
- `POST /api/v1/marketplace/product` - Create new marketplace product
- `GET /api/v1/marketplace/product/{product_id}` - Get specific marketplace product
- `PUT /api/v1/marketplace/product/{product_id}` - Update marketplace product
- `DELETE /api/v1/marketplace/product/{product_id}` - Delete marketplace product
- `POST /api/v1/marketplace/import-product` - Start CSV import
- `GET /api/v1/marketplace/import-status/{job_id}` - Check import status
- `GET /api/v1/marketplace/import-jobs` - List import jobs
-
### Inventory Endpoints
- `POST /api/v1/inventory` - Set inventory quantity
- `POST /api/v1/inventory/add` - Add to inventory
- `POST /api/v1/inventory/remove` - Remove from inventory
- `GET /api/v1/inventory/{gtin}` - Get inventory by GTIN
- `GET /api/v1/inventory/{gtin}/total` - Get total inventory
- `GET /api/v1/inventory` - List all inventory entries

### Shop Endpoints
- `POST /api/v1/shop` - Create new shop
- `GET /api/v1/shop` - List shops
- `GET /api/v1/shop/{shop_code}` - Get specific shop

### Statistics Endpoints
- `GET /api/v1/stats` - Get general statistics
- `GET /api/v1/stats/marketplace-stats` - Get marketplace statistics

### Admin Endpoints (Admin only)
- `GET /api/v1/admin/users` - List all users
- `PUT /api/v1/admin/users/{user_id}/status` - Toggle user status
- `GET /api/v1/admin/shops` - List all shops
- `PUT /api/v1/admin/shops/{shop_id}/verify` - Verify/unverify shop

## Documentation

This project uses MkDocs for documentation. The Makefile provides comprehensive documentation management commands.

### Documentation Development

```bash
# Start documentation development server
make docs

# Start both API and documentation servers
make dev-full

# Build documentation site
make docs-build

# Check documentation for issues
make docs-check

# Update docs with latest API information
make docs-update
```

### Documentation Deployment

```bash
# Deploy to GitHub Pages
make docs-deploy

# Force deploy to GitHub Pages
make docs-deploy-force

# Build and deploy in one command
make docs-publish

# Preview built documentation locally
make docs-preview
```

### Documentation Maintenance

```bash
# Setup documentation environment
make setup-docs

# Clean documentation build files
make docs-clean

# Get help with MkDocs commands
make docs-help
```

## Database Schema

### Core Tables
- **users** - User accounts and authentication
- **products** - MarketplaceProduct catalog with marketplace info
- **inventory** - Inventory tracking by location and GTIN
- **shops** - Shop/seller information
- **shop_products** - Shop-specific product settings
- **marketplace_import_jobs** - Background import job tracking

### Key Relationships
- Users own shops (one-to-many)
- Products belong to marketplaces and shops
- Inventory entries are linked to products via GTIN
- Import jobs track user-initiated imports

## Security Features

- **JWT Authentication** - Secure token-based authentication
- **Password Hashing** - bcrypt for secure password storage
- **Role-Based Access** - User and admin role separation
- **Rate Limiting** - Protection against API abuse
- **Input Validation** - Comprehensive data validation
- **SQL Injection Protection** - Parameterized queries
- **CORS Configuration** - Cross-origin request handling

## Testing

This project includes a comprehensive test suite with multiple testing strategies.

### Running Tests

```bash
# Run all tests
make test

# Run tests with coverage report
make test-coverage

# Run only unit tests
make test-unit

# Run only integration tests
make test-integration

# Run fast tests (exclude slow ones)
make test-fast

# Run slow tests only
make test-slow
```

### Test Specific Components

```bash
# Authentication tests
make test-auth

# MarketplaceProduct management tests
make test-products

# Inventory management tests
make test-inventory

# Marketplace functionality tests
make test-marketplace

# Admin functionality tests
make test-admin
```

### Test Setup

```bash
# Install test dependencies only
make setup-test

# Complete test workflow (format + lint + coverage)
make full-test
```

### Test Coverage
The test suite includes:
- **Unit Tests** - Individual component testing
- **Integration Tests** - Full workflow testing
- **Security Tests** - Authentication, authorization, input validation
- **Performance Tests** - Load testing with large datasets
- **Error Handling Tests** - Edge cases and error conditions

## Deployment

### Development Deployment

```bash
# Docker deployment
make docker-up

# Staging deployment
make deploy-staging

# Production deployment
make deploy-prod
```

### Cleanup and Maintenance

```bash
# Clean test artifacts and cache files
make clean

# Clean all build artifacts (including documentation)
make clean-all

# Release readiness check
make release-check
```

## Performance Optimizations

- **Database Indexing** - Strategic indexes on key columns
- **Pagination** - Efficient data retrieval with skip/limit
- **Streaming Responses** - Memory-efficient CSV exports
- **Background Processing** - Async import job handling
- **Connection Pooling** - Efficient database connections
- **Query Optimization** - Optimized database queries

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run quality assurance checks
5. Ensure all tests pass
6. Submit a pull request

### Development Setup for Contributors

```bash
# Complete development setup
make setup

# Install only development dependencies
make install-dev

# Run quality assurance pipeline
make qa

# Run complete CI pipeline
make ci
```

### Code Quality Standards

```bash
# Format code automatically
make format

# Check code quality
make check

# Run full test suite
make full-test

# Prepare for release
make release-check
```

### Available Make Commands

For a complete list of available commands, run:

```bash
make help
```

This will display all available commands organized by category:
- **Setup**: Installation and environment setup
- **Development**: Development servers and workflows
- **Documentation**: Documentation building and deployment
- **Testing**: Various test execution options
- **Code Quality**: Formatting, linting, and quality checks
- **Database**: Migration and database management
- **Docker**: Container management
- **Cleanup**: Artifact removal and maintenance
- **Workflows**: Combined command sequences

## Changelog

### v2.2.0
- Added marketplace-aware product import
- Implemented multi-shop support
- Enhanced inventory management with location tracking
- Added comprehensive test suite
- Improved authentication and authorization

### v2.1.0
- Added JWT authentication
- Implemented role-based access control
- Added CSV import/export functionality

### v2.0.0
- Complete rewrite with FastAPI
- Added PostgreSQL support
- Implemented comprehensive API documentation

## Support & Documentation

- **API Documentation**: http://localhost:8000/docs
- **Health Check**: http://localhost:8000/health
- **Version Info**: http://localhost:8000/

For issues and feature requests, please create an issue in the repository.