orion/updated_readme.md

Ecommerce Backend API v2.1

A robust, production-ready FastAPI backend for ecommerce product catalog and inventory management with JWT authentication and advanced CSV import capabilities.

Key Features

Security & Authentication

• JWT Authentication: Secure token-based authentication with configurable expiration (30 minutes default)
• User Management: Registration, login, role-based access control (Admin/User roles)
• Password Security: Bcrypt hashing for secure password storage
• Protected Endpoints: All product management operations require authentication
• Default Admin Account: Auto-created admin user for immediate system access

Architecture Improvements

• Modular Design: Separated concerns into utility modules, middleware, and models
• Database Optimization: Added proper indexing strategy and foreign key relationships
• Connection Pooling: PostgreSQL support with connection pooling for production scalability
• Background Processing: Asynchronous CSV import with job tracking

Performance Optimizations

• Batch Processing: CSV imports processed in configurable batches
• Database Indexes: Strategic indexing for common query patterns
• Streaming Export: Memory-efficient CSV export for large datasets
• Rate Limiting: Sliding window rate limiter to prevent API abuse

Data Processing

• Robust GTIN Handling: Centralized GTIN normalization and validation
• Multi-currency Support: Advanced price parsing with currency extraction
• International Content: Multi-encoding CSV support for global data

Project Structure

ecommerce_api/
├── main.py                 # FastAPI application entry point with auth
├── models/
│   ├── database_models.py  # SQLAlchemy ORM models (User, Product, Stock, ImportJob)
│   └── api_models.py       # Pydantic API models with auth models
├── utils/
│   ├── data_processing.py  # GTIN and price processing utilities
│   ├── csv_processor.py    # CSV import/export handling
│   └── database.py         # Database configuration
├── middleware/
│   ├── auth.py            # JWT authentication with bcrypt
│   ├── rate_limiter.py    # Rate limiting implementation
│   ├── error_handler.py   # Centralized error handling
│   └── logging_middleware.py # Request/response logging
├── tests/
│   └── test_auth.py       # Authentication tests
├── requirements.txt       # Python dependencies with auth packages
└── README.md             # This file

Quick Start

1. Installation

# Clone the repository
git clone <repository-url>
cd ecommerce-api

# Set up virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Environment Configuration

Create a .env file in the project root:

# Database
DATABASE_URL=postgresql://user:password@localhost:5432/ecommerce_db
# For SQLite (development): DATABASE_URL=sqlite:///./ecommerce.db

# JWT Configuration
JWT_SECRET_KEY=your-super-secret-key-change-in-production-immediately
JWT_EXPIRE_MINUTES=30

# Server Configuration
API_HOST=0.0.0.0
API_PORT=8000
DEBUG=False

Important Security Note: Always change the JWT_SECRET_KEY in production!

3. Database Setup

For SQLite (Development):

# Run the application - it will create tables automatically
python main.py

For PostgreSQL (Production):

# Create PostgreSQL database
createdb ecommerce_db

# Run the application - it will create tables and indexes automatically
python main.py

4. Start the Server

# Development server
uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Production server
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4

The API will be available at http://localhost:8000

5. Default Admin Access

The system automatically creates a default admin user:

• Username: admin
• Password: admin123
• Email: admin@example.com

Security Warning: Change the admin password immediately in production!

Authentication Flow

1. Register a New User

curl -X POST "http://localhost:8000/register" \
     -H "Content-Type: application/json" \
     -d '{
       "email": "user@example.com",
       "username": "newuser",
       "password": "securepassword123"
     }'

2. Login and Get JWT Token

curl -X POST "http://localhost:8000/login" \
     -H "Content-Type: application/json" \
     -d '{
       "username": "admin",
       "password": "admin123"
     }'

Response:

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "token_type": "bearer",
  "expires_in": 1800,
  "user": {
    "id": 1,
    "username": "admin",
    "email": "admin@example.com",
    "role": "admin",
    "is_active": true
  }
}

3. Use Token for Protected Endpoints

curl -X GET "http://localhost:8000/products" \
     -H "Authorization: Bearer YOUR_JWT_TOKEN_HERE"

API Endpoints

Public Endpoints

• GET / - API information
• GET /health - Health check
• POST /register - Register new user
• POST /login - Login and get JWT token

Protected Endpoints (Require Authentication)

User Management

• GET /me - Get current user information

Products (All users)

• GET /products - List products with filtering and search
• POST /products - Create new product
• GET /products/{product_id} - Get product with stock info
• PUT /products/{product_id} - Update product
• DELETE /products/{product_id} - Delete product and associated stock

Stock Management (All users)

• POST /stock - Set exact stock quantity
• GET /stock/{gtin} - Get stock summary by GTIN

CSV Operations (All users)

• POST /import-csv - Start background CSV import
• GET /import-status/{job_id} - Check import job status (own jobs only)
• GET /export-csv - Export products as CSV (streaming)

Statistics (All users)

• GET /stats - System statistics

Admin-Only Endpoints

• GET /admin/users - List all users
• PUT /admin/users/{user_id}/status - Activate/deactivate users

User Roles and Permissions

Regular Users

• Can register and login
• Access all product and stock management features
• Can import/export CSV files
• Can only view their own import jobs
• Cannot manage other users

Admin Users

• All regular user permissions
• Can view and manage all users
• Can view all import jobs from any user
• Can activate/deactivate user accounts

Security Features

Password Security

• Passwords hashed using bcrypt with salt
• Minimum password length: 6 characters
• No password storage in plaintext anywhere

JWT Token Security

• Tokens include expiration timestamp
• Tokens include user role and permissions
• Configurable expiration time (default: 30 minutes)
• Secure token validation on each request

Rate Limiting

• CSV imports: 10 requests per hour
• General API: 100 requests per hour per client
• Customizable per endpoint

Input Validation

• All inputs validated using Pydantic models
• Email format validation for registration
• Username alphanumeric validation
• GTIN format validation and normalization

Advanced Features

Background CSV Import

Import large CSV files asynchronously:

import requests

# Start import
response = requests.post(
    'http://localhost:8000/import-csv',
    headers={'Authorization': 'Bearer YOUR_TOKEN'},
    json={
        'url': 'https://example.com/products.csv',
        'batch_size': 1000
    }
)

job_id = response.json()['job_id']

# Check status
status_response = requests.get(
    f'http://localhost:8000/import-status/{job_id}',
    headers={'Authorization': 'Bearer YOUR_TOKEN'}
)
print(status_response.json())

Product Search and Filtering

# Search in title and description
GET /products?search=laptop

# Filter by brand and category
GET /products?brand=Apple&category=Electronics

# Combine filters with pagination
GET /products?brand=Samsung&availability=in%20stock&search=phone&skip=0&limit=50

Stock Management

# Set stock for a specific location
curl -X POST "http://localhost:8000/stock" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "gtin": "1234567890123",
       "location": "WAREHOUSE_A",
       "quantity": 100
     }'

# Get stock summary
curl -X GET "http://localhost:8000/stock/1234567890123" \
     -H "Authorization: Bearer YOUR_TOKEN"

Database Schema

Users Table

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    email VARCHAR UNIQUE NOT NULL,
    username VARCHAR UNIQUE NOT NULL,
    hashed_password VARCHAR NOT NULL,
    role VARCHAR DEFAULT 'user',
    is_active BOOLEAN DEFAULT true,
    last_login TIMESTAMP,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW()
);

Products Table

• Full product catalog with Google Shopping compatibility
• Indexed fields: gtin, brand, google_product_category, availability
• Supports all Google Shopping feed attributes

Stock Table

• Location-based inventory tracking
• GTIN-based product linking
• Unique constraint on GTIN+location combinations

Import Jobs Table

• Track background import operations
• User ownership and access control
• Status monitoring and error handling

Development

Running Tests

# Install test dependencies
pip install pytest pytest-asyncio httpx

# Run all tests
pytest

# Run with coverage
pytest --cov=. tests/

Development Server with Auto-reload

uvicorn main:app --reload --host 0.0.0.0 --port 8000

Production Deployment

Security Checklist

• Change default admin password immediately
• Set strong JWT_SECRET_KEY (32+ random characters)
• Configure JWT_EXPIRE_MINUTES appropriately
• Set up HTTPS/TLS termination
• Configure CORS for your frontend domains only
• Set up database connection limits and pooling
• Enable request logging and monitoring
• Configure rate limiting per your needs
• Set up user account monitoring and alerting
• Regular security audits of user accounts

Environment Variables for Production

# Security
JWT_SECRET_KEY=your-very-long-random-secret-key-at-least-32-characters
JWT_EXPIRE_MINUTES=30

# Database (use PostgreSQL in production)
DATABASE_URL=postgresql://user:password@db-host:5432/ecommerce_prod

# Server
DEBUG=False
API_HOST=0.0.0.0
API_PORT=8000

# Optional: External services
REDIS_URL=redis://redis-host:6379/0

Docker Deployment

# docker-compose.yml
version: '3.8'
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: ecommerce
      POSTGRES_USER: ecommerce_user
      POSTGRES_PASSWORD: secure_password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  api:
    build: .
    environment:
      DATABASE_URL: postgresql://ecommerce_user:secure_password@db:5432/ecommerce
      JWT_SECRET_KEY: your-production-secret-key
      JWT_EXPIRE_MINUTES: 30
    ports:
      - "8000:8000"
    depends_on:
      - db
    restart: unless-stopped

volumes:
  postgres_data:

Nginx Configuration

server {
    listen 80;
    server_name your-api-domain.com;

    location / {
        proxy_pass http://localhost:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Troubleshooting

Authentication Issues

Invalid Token Errors:

• Check token expiration (default 30 minutes)
• Verify JWT_SECRET_KEY is consistent
• Ensure Bearer token format: Authorization: Bearer <token>

Login Failures:

• Verify username/email and password
• Check if user account is active (is_active: true)
• Review user registration process

Permission Denied:

• Confirm user role permissions
• Check endpoint access requirements
• Verify token contains correct role information

Database Issues

Connection Errors:

• Verify DATABASE_URL format and credentials
• Check database server accessibility
• Monitor connection pool limits

Migration Issues:

• Tables are created automatically on startup
• For schema changes, implement proper migrations
• Backup data before major updates

Common API Issues

CSV Import Failures:

• Check file URL accessibility
• Verify CSV format and encoding
• Monitor import job status for detailed errors
• Ensure proper authentication token

Rate Limiting:

• Default limits: 100 requests/hour, 10 CSV imports/hour
• Check rate limit headers in responses
• Implement proper retry logic with backoff

Logging and Monitoring

Application logs include:

• Authentication events (login, failed attempts)
• API request/response times
• Import job progress and errors
• Rate limiting events
• Database query performance

# View logs in development
python main.py  # Logs to console

# Docker logs
docker-compose logs -f api

Migration from v2.0

If upgrading from v2.0 to v2.1:

1. Install new dependencies:

   pip install -r requirements.txt
   

2. Update environment variables:

   echo "JWT_SECRET_KEY=your-secret-key" >> .env
   echo "JWT_EXPIRE_MINUTES=30" >> .env
   

3. Database migration: The application will automatically create the new users table and update the import_jobs table on startup.

4. Update client code:

   • Add authentication to all product management API calls
   • Implement login flow in your frontend
   • Handle JWT token refresh

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Make changes with proper tests
4. Run security and quality checks
5. Update documentation if needed
6. Submit a pull request

Code Quality Standards

• All endpoints must have proper authentication
• Password handling must use bcrypt
• JWT tokens must have expiration
• Input validation using Pydantic models
• Comprehensive error handling
• Unit tests for authentication logic

License

This project is licensed under the MIT License - see the LICENSE file for details.

Security

For security issues, please email the maintainers directly instead of creating a public issue.

Support

For issues and questions:

1. Check the troubleshooting section above
2. Review existing GitHub issues
3. Create a new issue with detailed information including:
   • Authentication steps you've tried
   • Error messages and logs
   • API endpoint and request details
   • Environment configuration (without secrets)