orion/comprehensive_readme.md

# Ecommerce Backend API with Marketplace Support

A comprehensive FastAPI-based product management system with JWT authentication, marketplace-aware CSV import/export, multi-shop support, and advanced stock management capabilities.

## Features

- **JWT Authentication** - Secure user registration, login, and role-based access control
- **Marketplace Integration** - Support for multiple marketplaces (Letzshop, Amazon, eBay, Etsy, Shopify, etc.)
- **Multi-Shop Management** - Shop creation, ownership validation, and product catalog management
- **Advanced Product Management** - GTIN validation, price processing, and comprehensive filtering
- **Stock Management** - Multi-location inventory tracking with add/remove/set operations
- **CSV Import/Export** - Background processing of marketplace CSV files with progress tracking
- **Rate Limiting** - Built-in request rate limiting for API protection
- **Admin Panel** - Administrative functions for user and shop management
- **Statistics & Analytics** - Comprehensive reporting on products, marketplaces, and inventory

## Technology Stack

- **FastAPI** - Modern, fast web framework for building APIs
- **SQLAlchemy** - SQL toolkit and Object-Relational Mapping (ORM)
- **PostgreSQL** - Primary database (SQLite supported for development)
- **JWT** - JSON Web Tokens for secure authentication
- **Pydantic** - Data validation using Python type annotations
- **Pandas** - Data processing for CSV operations
- **bcrypt** - Password hashing
- **Pytest** - Comprehensive testing framework

## Directory Structure

```
letzshop_api/
├── main.py                     # FastAPI application entry point
├── app/
│   ├── core/
│   │   ├── config.py           # Configuration settings
│   │   ├── database.py         # Database setup
│   │   └── lifespan.py         # App lifecycle management
│   ├── api/
│   │   ├── deps.py             # Common dependencies
│   │   ├── main.py             # API router setup
│   │   └── v1/                 # API version 1 routes
│   │       ├── auth.py         # Authentication endpoints
│   │       ├── products.py     # Product management
│   │       ├── stock.py        # Stock operations
│   │       ├── shops.py        # Shop management
│   │       ├── marketplace.py  # Marketplace imports
│   │       ├── admin.py        # Admin functions
│   │       └── stats.py        # Statistics
│   ├── services/               # Business logic layer
│   └── tasks/                  # Background task processing
├── models/
│   ├── database_models.py      # SQLAlchemy ORM models
│   └── api_models.py          # Pydantic API models
├── utils/
│   ├── data_processing.py     # GTIN and price processing
│   ├── csv_processor.py       # CSV import/export
│   └── database.py           # Database utilities
├── middleware/
│   ├── auth.py               # JWT authentication
│   ├── rate_limiter.py       # Rate limiting
│   ├── error_handler.py      # Error handling
│   └── logging_middleware.py # Request logging
├── tests/                    # Comprehensive test suite
└── requirements.txt          # Dependencies
```

## Quick Start

### 1. Installation

```bash
# Clone the repository
git clone <repository-url>
cd letzshop_api

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

### 2. Environment Configuration

Create a `.env` file in the project root:

```env
# Database Configuration
DATABASE_URL=postgresql://user:password@localhost/ecommerce
# For development, you can use SQLite:
# DATABASE_URL=sqlite:///./ecommerce.db

# JWT Configuration
SECRET_KEY=your-super-secret-key-change-in-production
ACCESS_TOKEN_EXPIRE_MINUTES=30

# API Configuration
ALLOWED_HOSTS=["*"]
RATE_LIMIT_REQUESTS=100
RATE_LIMIT_WINDOW=3600

# Application Settings
PROJECT_NAME="Ecommerce Backend API"
VERSION="2.2.0"
DEBUG=True
```

### 3. Database Setup

```bash
# The application will automatically create tables on startup
# For production, consider using Alembic for migrations

# Install PostgreSQL (if using PostgreSQL)
# Create database
createdb ecommerce

# Run the application (tables will be created automatically)
python main.py
```

### 4. Running the Application

```bash
# Development server
python main.py

# Or using uvicorn directly
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

The API will be available at:
- **API Documentation**: http://localhost:8000/docs (Swagger UI)
- **Alternative Docs**: http://localhost:8000/redoc
- **Health Check**: http://localhost:8000/health

## API Usage

### Authentication

#### Register a new user
```bash
curl -X POST "http://localhost:8000/api/v1/auth/register" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "username": "testuser",
    "password": "securepassword123"
  }'
```

#### Login
```bash
curl -X POST "http://localhost:8000/api/v1/auth/login" \
  -H "Content-Type: application/json" \
  -d '{
    "username": "testuser",
    "password": "securepassword123"
  }'
```

#### Use JWT Token
```bash
# Get token from login response and use in subsequent requests
curl -X GET "http://localhost:8000/api/v1/products" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
```

### Product Management

#### Create a product
```bash
curl -X POST "http://localhost:8000/api/v1/products" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "product_id": "PROD001",
    "title": "Amazing Product",
    "description": "An amazing product description",
    "price": "29.99",
    "currency": "EUR",
    "brand": "BrandName",
    "gtin": "1234567890123",
    "availability": "in stock",
    "marketplace": "Letzshop",
    "shop_name": "MyShop"
  }'
```

#### Get products with filtering
```bash
# Get all products
curl -X GET "http://localhost:8000/api/v1/products" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Filter by marketplace
curl -X GET "http://localhost:8000/api/v1/products?marketplace=Amazon&limit=50" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Search products
curl -X GET "http://localhost:8000/api/v1/products?search=Amazing&brand=BrandName" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Stock Management

#### Set stock quantity
```bash
curl -X POST "http://localhost:8000/api/v1/stock" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 100
  }'
```

#### Add stock
```bash
curl -X POST "http://localhost:8000/api/v1/stock/add" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "gtin": "1234567890123",
    "location": "WAREHOUSE_A",
    "quantity": 25
  }'
```

#### Check stock levels
```bash
curl -X GET "http://localhost:8000/api/v1/stock/1234567890123" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Marketplace Import

#### Import products from CSV
```bash
curl -X POST "http://localhost:8000/api/v1/marketplace/import-from-marketplace" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products.csv",
    "marketplace": "Amazon",
    "shop_code": "MYSHOP",
    "batch_size": 1000
  }'
```

#### Check import status
```bash
curl -X GET "http://localhost:8000/api/v1/marketplace/marketplace-import-status/1" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Export Data

#### Export products to CSV
```bash
# Export all products
curl -X GET "http://localhost:8000/api/v1/export-csv" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o products_export.csv

# Export with filters
curl -X GET "http://localhost:8000/api/v1/export-csv?marketplace=Amazon&shop_name=MyShop" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -o amazon_products.csv
```

## CSV Import Format

The system supports CSV imports with the following headers:

### Required Fields
- `product_id` - Unique product identifier
- `title` - Product title

### Optional Fields
- `description` - Product description
- `link` - Product URL
- `image_link` - Product image URL
- `availability` - Stock availability (in stock, out of stock, preorder)
- `price` - Product price
- `currency` - Price currency (EUR, USD, etc.)
- `brand` - Product brand
- `gtin` - Global Trade Item Number (EAN/UPC)
- `google_product_category` - Product category
- `marketplace` - Source marketplace
- `shop_name` - Shop/seller name

### Example CSV
```csv
product_id,title,description,price,currency,brand,gtin,marketplace,shop_name
PROD001,"Amazing Widget","The best widget ever",29.99,EUR,WidgetCorp,1234567890123,Letzshop,MyShop
PROD002,"Super Gadget","A fantastic gadget",19.99,EUR,GadgetInc,9876543210987,Amazon,TechStore
```

## Testing

### Run Tests
```bash
# Install test dependencies
pip install -r tests/requirements_test.txt

# Run all tests
pytest tests/ -v

# Run with coverage report
pytest tests/ --cov=app --cov=models --cov=utils --cov-report=html

# Run specific test categories
pytest tests/ -m unit -v                    # Unit tests only
pytest tests/ -m integration -v             # Integration tests only
pytest tests/ -m "not slow" -v              # Fast tests only

# Run specific test files
pytest tests/test_auth.py -v                # Authentication tests
pytest tests/test_products.py -v            # Product tests
pytest tests/test_stock.py -v               # Stock management tests
```

### Test Coverage
The test suite includes:
- **Unit Tests** - Individual component testing
- **Integration Tests** - Full workflow testing
- **Security Tests** - Authentication, authorization, input validation
- **Performance Tests** - Load testing with large datasets
- **Error Handling Tests** - Edge cases and error conditions

## API Reference

### Authentication Endpoints
- `POST /api/v1/auth/register` - Register new user
- `POST /api/v1/auth/login` - Login user
- `GET /api/v1/auth/me` - Get current user info

### Product Endpoints
- `GET /api/v1/products` - List products with filtering
- `POST /api/v1/products` - Create new product
- `GET /api/v1/products/{product_id}` - Get specific product
- `PUT /api/v1/products/{product_id}` - Update product
- `DELETE /api/v1/products/{product_id}` - Delete product

### Stock Endpoints
- `POST /api/v1/stock` - Set stock quantity
- `POST /api/v1/stock/add` - Add to stock
- `POST /api/v1/stock/remove` - Remove from stock
- `GET /api/v1/stock/{gtin}` - Get stock by GTIN
- `GET /api/v1/stock/{gtin}/total` - Get total stock
- `GET /api/v1/stock` - List all stock entries

### Shop Endpoints
- `POST /api/v1/shops` - Create new shop
- `GET /api/v1/shops` - List shops
- `GET /api/v1/shops/{shop_code}` - Get specific shop

### Marketplace Endpoints
- `POST /api/v1/marketplace/import-from-marketplace` - Start CSV import
- `GET /api/v1/marketplace/marketplace-import-status/{job_id}` - Check import status
- `GET /api/v1/marketplace/marketplace-import-jobs` - List import jobs

### Statistics Endpoints
- `GET /api/v1/stats` - Get general statistics
- `GET /api/v1/stats/marketplace-stats` - Get marketplace statistics

### Admin Endpoints (Admin only)
- `GET /api/v1/admin/users` - List all users
- `PUT /api/v1/admin/users/{user_id}/status` - Toggle user status
- `GET /api/v1/admin/shops` - List all shops
- `PUT /api/v1/admin/shops/{shop_id}/verify` - Verify/unverify shop

## Database Schema

### Core Tables
- **users** - User accounts and authentication
- **products** - Product catalog with marketplace info
- **stock** - Inventory tracking by location and GTIN
- **shops** - Shop/seller information
- **shop_products** - Shop-specific product settings
- **marketplace_import_jobs** - Background import job tracking

### Key Relationships
- Users own shops (one-to-many)
- Products belong to marketplaces and shops
- Stock entries are linked to products via GTIN
- Import jobs track user-initiated imports

## Security Features

- **JWT Authentication** - Secure token-based authentication
- **Password Hashing** - bcrypt for secure password storage
- **Role-Based Access** - User and admin role separation
- **Rate Limiting** - Protection against API abuse
- **Input Validation** - Comprehensive data validation
- **SQL Injection Protection** - Parameterized queries
- **CORS Configuration** - Cross-origin request handling

## Performance Optimizations

- **Database Indexing** - Strategic indexes on key columns
- **Pagination** - Efficient data retrieval with skip/limit
- **Streaming Responses** - Memory-efficient CSV exports
- **Background Processing** - Async import job handling
- **Connection Pooling** - Efficient database connections
- **Query Optimization** - Optimized database queries

## Deployment

### Docker Deployment
```dockerfile
FROM python:3.11

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
```

### Production Considerations
- Use PostgreSQL for production database
- Set strong SECRET_KEY in environment
- Configure proper CORS settings
- Enable HTTPS
- Set up monitoring and logging
- Use a reverse proxy (nginx)
- Configure database connection pooling
- Set up backup strategies

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass
6. Submit a pull request

### Development Setup
```bash
# Install development dependencies
pip install -r requirements_dev.txt

# Run pre-commit hooks
pre-commit install

# Run linting
flake8 app/ models/ utils/
black app/ models/ utils/

# Run type checking
mypy app/
```

## Support & Documentation

- **API Documentation**: http://localhost:8000/docs
- **Health Check**: http://localhost:8000/health
- **Version Info**: http://localhost:8000/

For issues and feature requests, please create an issue in the repository.

## License

[Specify your license here]

## Changelog

### v2.2.0
- Added marketplace-aware product import
- Implemented multi-shop support
- Enhanced stock management with location tracking
- Added comprehensive test suite
- Improved authentication and authorization

### v2.1.0
- Added JWT authentication
- Implemented role-based access control
- Added CSV import/export functionality

### v2.0.0
- Complete rewrite with FastAPI
- Added PostgreSQL support
- Implemented comprehensive API documentation