663 lines
19 KiB
Markdown
663 lines
19 KiB
Markdown
# Letzshop Marketplace API v2.1
|
|
|
|
A robust, production-ready FastAPI backend for Luxembourg's premier e-commerce marketplace with multi-vendor support, JWT authentication, and advanced CSV import capabilities.
|
|
|
|
## Key Features
|
|
|
|
### Marketplace Architecture
|
|
- **Multi-Vendor Support**: Shops can import and manage their product catalogs independently
|
|
- **Centralized Product Catalog**: Products exist in main marketplace with shop-specific overrides
|
|
- **Shop Management**: Complete vendor onboarding, verification, and management system
|
|
- **Shop-Specific Pricing**: Vendors can set their own prices, availability, and conditions
|
|
- **Marketplace Controls**: Admin verification and quality control for vendor shops
|
|
|
|
### Security & Authentication
|
|
- **JWT Authentication**: Secure token-based authentication with configurable expiration (30 minutes default)
|
|
- **User Management**: Registration, login, role-based access control (Admin/User/Shop Owner roles)
|
|
- **Password Security**: Bcrypt hashing for secure password storage
|
|
- **Protected Endpoints**: All operations require authentication with proper authorization
|
|
- **Default Admin Account**: Auto-created admin user for immediate system access
|
|
|
|
### Architecture Improvements
|
|
- **Modular Design**: Separated concerns into utility modules, middleware, and models
|
|
- **Database Optimization**: Added proper indexing strategy and foreign key relationships
|
|
- **Connection Pooling**: PostgreSQL support with connection pooling for production scalability
|
|
- **Background Processing**: Asynchronous CSV import with job tracking per shop
|
|
|
|
### Performance Optimizations
|
|
- **Batch Processing**: CSV imports processed in configurable batches
|
|
- **Database Indexes**: Strategic indexing for common query patterns including shop relationships
|
|
- **Streaming Export**: Memory-efficient CSV export for large datasets with shop filtering
|
|
- **Rate Limiting**: Sliding window rate limiter to prevent API abuse
|
|
|
|
### Data Processing
|
|
- **Robust GTIN Handling**: Centralized GTIN normalization and validation
|
|
- **Multi-currency Support**: Advanced price parsing with currency extraction
|
|
- **International Content**: Multi-encoding CSV support for global data
|
|
- **Shop Association**: Automatic product-shop linking during CSV imports
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
letzshop_api/
|
|
├── main.py # FastAPI application entry point with marketplace support
|
|
├── models/
|
|
│ ├── database_models.py # SQLAlchemy ORM models (User, Shop, Product, ShopProduct, Stock, ImportJob)
|
|
│ └── api_models.py # Pydantic API models with shop and auth models
|
|
├── utils/
|
|
│ ├── data_processing.py # GTIN and price processing utilities
|
|
│ ├── csv_processor.py # CSV import/export handling with shop support
|
|
│ └── database.py # Database configuration
|
|
├── middleware/
|
|
│ ├── auth.py # JWT authentication with bcrypt
|
|
│ ├── rate_limiter.py # Rate limiting implementation
|
|
│ ├── error_handler.py # Centralized error handling
|
|
│ └── logging_middleware.py # Request/response logging
|
|
├── tests/
|
|
│ └── test_auth.py # Authentication tests
|
|
├── requirements.txt # Python dependencies with auth packages
|
|
└── README.md # This file
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### 1. Installation
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone <repository-url>
|
|
cd letzshop-api
|
|
|
|
# Set up virtual environment
|
|
python -m venv venv
|
|
source venv/bin/activate # On Windows: venv\Scripts\activate
|
|
|
|
# Install dependencies
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
### 2. Environment Configuration
|
|
|
|
Create a `.env` file in the project root:
|
|
|
|
```env
|
|
# Database
|
|
DATABASE_URL=postgresql://user:password@localhost:5432/letzshop_db
|
|
# For SQLite (development): DATABASE_URL=sqlite:///./letzshop.db
|
|
|
|
# JWT Configuration
|
|
JWT_SECRET_KEY=your-super-secret-key-change-in-production-immediately
|
|
JWT_EXPIRE_MINUTES=30
|
|
|
|
# Server Configuration
|
|
API_HOST=0.0.0.0
|
|
API_PORT=8000
|
|
DEBUG=False
|
|
```
|
|
|
|
**Important Security Note**: Always change the `JWT_SECRET_KEY` in production!
|
|
|
|
### 3. Database Setup
|
|
|
|
**For SQLite (Development):**
|
|
```bash
|
|
# Run the application - it will create tables automatically
|
|
python main.py
|
|
```
|
|
|
|
**For PostgreSQL (Production):**
|
|
```bash
|
|
# Create PostgreSQL database
|
|
createdb letzshop_db
|
|
|
|
# Run the application - it will create tables and indexes automatically
|
|
python main.py
|
|
```
|
|
|
|
### 4. Start the Server
|
|
|
|
```bash
|
|
# Development server
|
|
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
|
|
|
# Production server
|
|
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
|
|
```
|
|
|
|
The API will be available at `http://localhost:8000`
|
|
|
|
### 5. Default Access
|
|
|
|
The system automatically creates:
|
|
- **Admin User**: `admin` / `admin123` / `admin@example.com`
|
|
- **Demo Shop**: `DEMOSHOP` owned by admin for testing
|
|
|
|
**Security Warning**: Change the admin password immediately in production!
|
|
|
|
## Authentication Flow
|
|
|
|
### 1. Register a New User
|
|
|
|
```bash
|
|
curl -X POST "http://localhost:8000/register" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"email": "vendor@example.com",
|
|
"username": "newvendor",
|
|
"password": "securepassword123"
|
|
}'
|
|
```
|
|
|
|
### 2. Login and Get JWT Token
|
|
|
|
```bash
|
|
curl -X POST "http://localhost:8000/login" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"username": "admin",
|
|
"password": "admin123"
|
|
}'
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
|
|
"token_type": "bearer",
|
|
"expires_in": 1800,
|
|
"user": {
|
|
"id": 1,
|
|
"username": "admin",
|
|
"email": "admin@example.com",
|
|
"role": "admin",
|
|
"is_active": true
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3. Use Token for Protected Endpoints
|
|
|
|
```bash
|
|
curl -X GET "http://localhost:8000/shops" \
|
|
-H "Authorization: Bearer YOUR_JWT_TOKEN_HERE"
|
|
```
|
|
|
|
## Marketplace Workflow
|
|
|
|
### 1. Create a Shop
|
|
|
|
```bash
|
|
curl -X POST "http://localhost:8000/shops" \
|
|
-H "Authorization: Bearer YOUR_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"shop_code": "TECHSTORE",
|
|
"shop_name": "Tech Store Luxembourg",
|
|
"description": "Electronics and gadgets for Luxembourg",
|
|
"contact_email": "info@techstore.lu",
|
|
"contact_phone": "+352 123 456 789",
|
|
"website": "https://techstore.lu"
|
|
}'
|
|
```
|
|
|
|
### 2. Import Products for Your Shop
|
|
|
|
```bash
|
|
curl -X POST "http://localhost:8000/import-csv" \
|
|
-H "Authorization: Bearer YOUR_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"url": "https://techstore.com/products.csv",
|
|
"shop_code": "TECHSTORE",
|
|
"batch_size": 1000
|
|
}'
|
|
```
|
|
|
|
### 3. Monitor Import Progress
|
|
|
|
```bash
|
|
curl -X GET "http://localhost:8000/import-status/1" \
|
|
-H "Authorization: Bearer YOUR_TOKEN"
|
|
```
|
|
|
|
### 4. View Shop Products
|
|
|
|
```bash
|
|
curl -X GET "http://localhost:8000/products?shop_code=TECHSTORE" \
|
|
-H "Authorization: Bearer YOUR_TOKEN"
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
### Public Endpoints
|
|
- `GET /` - API information
|
|
- `GET /health` - Health check
|
|
- `POST /register` - Register new user
|
|
- `POST /login` - Login and get JWT token
|
|
|
|
### Protected Endpoints (Require Authentication)
|
|
|
|
#### User Management
|
|
- `GET /me` - Get current user information
|
|
|
|
#### Shop Management
|
|
- `POST /shops` - Create new shop
|
|
- `GET /shops` - List shops with filtering
|
|
- `GET /shops/{shop_code}` - Get shop details
|
|
- `PUT /shops/{shop_code}` - Update shop (owners only)
|
|
- `POST /shops/{shop_code}/products` - Add product to shop catalog
|
|
- `GET /shops/{shop_code}/products` - Get shop products
|
|
|
|
#### Products (Marketplace Catalog)
|
|
- `GET /products` - List products with filtering (optionally by shop)
|
|
- `POST /products` - Create new product in marketplace catalog
|
|
- `GET /products/{product_id}` - Get product with stock info and shop listings
|
|
- `PUT /products/{product_id}` - Update product
|
|
- `DELETE /products/{product_id}` - Delete product and associated shop listings
|
|
|
|
#### Stock Management
|
|
- `POST /stock` - Set stock quantity (with optional shop association)
|
|
- `GET /stock/{gtin}` - Get stock summary by GTIN
|
|
|
|
#### CSV Operations
|
|
- `POST /import-csv` - Start background CSV import for specific shop
|
|
- `GET /import-status/{job_id}` - Check import job status
|
|
- `GET /export-csv` - Export products as CSV (optionally filtered by shop)
|
|
|
|
#### Statistics
|
|
- `GET /stats` - Marketplace statistics
|
|
|
|
#### Admin-Only Endpoints
|
|
- `GET /admin/users` - List all users
|
|
- `PUT /admin/users/{user_id}/status` - Activate/deactivate users
|
|
- `GET /admin/shops` - List all shops
|
|
- `PUT /admin/shops/{shop_id}/verify` - Verify/unverify shop
|
|
- `PUT /admin/shops/{shop_id}/status` - Activate/deactivate shop
|
|
- `GET /admin/import-jobs` - View all import jobs
|
|
|
|
## User Roles and Permissions
|
|
|
|
### Regular Users
|
|
- Can register and login
|
|
- Can create and manage their own shops
|
|
- Can import products for their shops
|
|
- Can manage stock for their products
|
|
- Can view marketplace products and shops
|
|
|
|
### Shop Owners (Regular Users with Shops)
|
|
- All regular user permissions
|
|
- Can manage their shop information
|
|
- Can import/export products for their shops
|
|
- Can set shop-specific pricing and availability
|
|
- Can view their import job history
|
|
|
|
### Admin Users
|
|
- All user permissions
|
|
- Can view and manage all users and shops
|
|
- Can verify/unverify shops
|
|
- Can view all import jobs from any shop
|
|
- Can activate/deactivate user accounts and shops
|
|
|
|
## Marketplace Features
|
|
|
|
### Shop Verification System
|
|
- New shops start as unverified
|
|
- Admin approval required for public visibility
|
|
- Verified shops appear in public marketplace listings
|
|
- Quality control through admin verification
|
|
|
|
### Multi-Vendor Product Catalog
|
|
- Products exist in central marketplace catalog
|
|
- Multiple shops can sell the same product
|
|
- Shop-specific pricing, availability, and conditions
|
|
- Automatic product matching during CSV imports
|
|
|
|
### Shop-Specific Overrides
|
|
```json
|
|
{
|
|
"product_id": "LAPTOP123",
|
|
"shop_price": 999.99,
|
|
"shop_currency": "EUR",
|
|
"shop_availability": "in stock",
|
|
"shop_condition": "new",
|
|
"is_featured": true,
|
|
"min_quantity": 1,
|
|
"max_quantity": 5
|
|
}
|
|
```
|
|
|
|
### Advanced Product Search
|
|
```bash
|
|
# Search products in specific shop
|
|
GET /products?shop_code=TECHSTORE&search=laptop
|
|
|
|
# Search across all verified shops
|
|
GET /products?search=laptop&availability=in%20stock
|
|
|
|
# Filter by brand and category
|
|
GET /products?brand=Apple&category=Electronics
|
|
```
|
|
|
|
## Database Schema
|
|
|
|
### Core Tables
|
|
|
|
#### Users Table
|
|
```sql
|
|
CREATE TABLE users (
|
|
id SERIAL PRIMARY KEY,
|
|
email VARCHAR UNIQUE NOT NULL,
|
|
username VARCHAR UNIQUE NOT NULL,
|
|
hashed_password VARCHAR NOT NULL,
|
|
role VARCHAR DEFAULT 'user',
|
|
is_active BOOLEAN DEFAULT true,
|
|
last_login TIMESTAMP,
|
|
created_at TIMESTAMP DEFAULT NOW(),
|
|
updated_at TIMESTAMP DEFAULT NOW()
|
|
);
|
|
```
|
|
|
|
#### Shops Table
|
|
```sql
|
|
CREATE TABLE shops (
|
|
id SERIAL PRIMARY KEY,
|
|
shop_code VARCHAR UNIQUE NOT NULL,
|
|
shop_name VARCHAR NOT NULL,
|
|
description TEXT,
|
|
owner_id INTEGER REFERENCES users(id),
|
|
contact_email VARCHAR,
|
|
contact_phone VARCHAR,
|
|
website VARCHAR,
|
|
business_address TEXT,
|
|
tax_number VARCHAR,
|
|
is_active BOOLEAN DEFAULT true,
|
|
is_verified BOOLEAN DEFAULT false,
|
|
created_at TIMESTAMP DEFAULT NOW(),
|
|
updated_at TIMESTAMP DEFAULT NOW()
|
|
);
|
|
```
|
|
|
|
#### Products Table
|
|
- Main marketplace catalog with Google Shopping compatibility
|
|
- Indexed fields: `gtin`, `brand`, `google_product_category`, `availability`
|
|
- Supports all Google Shopping feed attributes
|
|
|
|
#### ShopProducts Table
|
|
```sql
|
|
CREATE TABLE shop_products (
|
|
id SERIAL PRIMARY KEY,
|
|
shop_id INTEGER REFERENCES shops(id),
|
|
product_id INTEGER REFERENCES products(id),
|
|
shop_product_id VARCHAR,
|
|
shop_price DECIMAL,
|
|
shop_sale_price DECIMAL,
|
|
shop_currency VARCHAR,
|
|
shop_availability VARCHAR,
|
|
shop_condition VARCHAR,
|
|
is_featured BOOLEAN DEFAULT false,
|
|
is_active BOOLEAN DEFAULT true,
|
|
min_quantity INTEGER DEFAULT 1,
|
|
max_quantity INTEGER,
|
|
created_at TIMESTAMP DEFAULT NOW(),
|
|
updated_at TIMESTAMP DEFAULT NOW(),
|
|
UNIQUE(shop_id, product_id)
|
|
);
|
|
```
|
|
|
|
#### Stock Table
|
|
- Location-based inventory tracking with optional shop association
|
|
- GTIN-based product linking
|
|
- Support for reserved quantities (for order processing)
|
|
|
|
#### Import Jobs Table
|
|
- Track background import operations per shop
|
|
- User and shop ownership tracking
|
|
- Status monitoring and error handling
|
|
|
|
## Advanced Features
|
|
|
|
### Shop-Specific CSV Import
|
|
|
|
Import products with automatic shop association:
|
|
|
|
```python
|
|
import requests
|
|
|
|
# Start import for specific shop
|
|
response = requests.post(
|
|
'http://localhost:8000/import-csv',
|
|
headers={'Authorization': 'Bearer YOUR_TOKEN'},
|
|
json={
|
|
'url': 'https://myshop.com/products.csv',
|
|
'shop_code': 'MYSHOP',
|
|
'batch_size': 1000
|
|
}
|
|
)
|
|
|
|
job_id = response.json()['job_id']
|
|
|
|
# Monitor progress
|
|
status_response = requests.get(
|
|
f'http://localhost:8000/import-status/{job_id}',
|
|
headers={'Authorization': 'Bearer YOUR_TOKEN'}
|
|
)
|
|
print(status_response.json())
|
|
```
|
|
|
|
### Multi-Shop Product Management
|
|
|
|
```bash
|
|
# Add existing marketplace product to your shop
|
|
curl -X POST "http://localhost:8000/shops/MYSHOP/products" \
|
|
-H "Authorization: Bearer YOUR_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"product_id": "EXISTING_PRODUCT_123",
|
|
"shop_price": 89.99,
|
|
"shop_availability": "in stock",
|
|
"is_featured": true
|
|
}'
|
|
|
|
# Get products from specific shop
|
|
curl -X GET "http://localhost:8000/shops/MYSHOP/products" \
|
|
-H "Authorization: Bearer YOUR_TOKEN"
|
|
```
|
|
|
|
### Stock Management with Shop Context
|
|
|
|
```bash
|
|
# Set shop-specific stock
|
|
curl -X POST "http://localhost:8000/stock" \
|
|
-H "Authorization: Bearer YOUR_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"gtin": "1234567890123",
|
|
"location": "MYSHOP_WAREHOUSE",
|
|
"quantity": 50,
|
|
"shop_code": "MYSHOP"
|
|
}'
|
|
```
|
|
|
|
## Production Deployment
|
|
|
|
### Security Checklist for Marketplace
|
|
|
|
- [ ] Change default admin password immediately
|
|
- [ ] Set strong JWT_SECRET_KEY (32+ random characters)
|
|
- [ ] Configure JWT_EXPIRE_MINUTES appropriately
|
|
- [ ] Set up HTTPS/TLS termination
|
|
- [ ] Configure CORS for your frontend domains only
|
|
- [ ] Set up database connection limits and pooling
|
|
- [ ] Enable request logging and monitoring
|
|
- [ ] Configure rate limiting per your needs
|
|
- [ ] Set up shop verification workflow
|
|
- [ ] Implement shop quality monitoring
|
|
- [ ] Set up automated backup for shop data
|
|
- [ ] Configure email notifications for shop owners
|
|
- [ ] Regular security audits of user accounts and shops
|
|
|
|
### Environment Variables for Production
|
|
|
|
```env
|
|
# Security
|
|
JWT_SECRET_KEY=your-very-long-random-secret-key-at-least-32-characters
|
|
JWT_EXPIRE_MINUTES=30
|
|
|
|
# Database (use PostgreSQL in production)
|
|
DATABASE_URL=postgresql://user:password@db-host:5432/letzshop_prod
|
|
|
|
# Server
|
|
DEBUG=False
|
|
API_HOST=0.0.0.0
|
|
API_PORT=8000
|
|
|
|
# Marketplace Configuration
|
|
MARKETPLACE_NAME=Letzshop
|
|
DEFAULT_CURRENCY=EUR
|
|
ADMIN_EMAIL=admin@letzshop.lu
|
|
|
|
# Optional: External services
|
|
REDIS_URL=redis://redis-host:6379/0
|
|
EMAIL_API_KEY=your-email-service-key
|
|
```
|
|
|
|
### Docker Deployment
|
|
|
|
```yaml
|
|
# docker-compose.yml
|
|
version: '3.8'
|
|
services:
|
|
db:
|
|
image: postgres:15
|
|
environment:
|
|
POSTGRES_DB: letzshop
|
|
POSTGRES_USER: letzshop_user
|
|
POSTGRES_PASSWORD: secure_password
|
|
volumes:
|
|
- postgres_data:/var/lib/postgresql/data
|
|
ports:
|
|
- "5432:5432"
|
|
|
|
api:
|
|
build: .
|
|
environment:
|
|
DATABASE_URL: postgresql://letzshop_user:secure_password@db:5432/letzshop
|
|
JWT_SECRET_KEY: your-production-secret-key
|
|
JWT_EXPIRE_MINUTES: 30
|
|
MARKETPLACE_NAME: Letzshop
|
|
ports:
|
|
- "8000:8000"
|
|
depends_on:
|
|
- db
|
|
restart: unless-stopped
|
|
|
|
volumes:
|
|
postgres_data:
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Marketplace-Specific Issues
|
|
|
|
**Shop Import Failures:**
|
|
- Verify shop exists and is active
|
|
- Check user permissions for the shop
|
|
- Ensure CSV format is compatible
|
|
- Monitor import job status for detailed errors
|
|
|
|
**Shop Product Association:**
|
|
- Products are added to main catalog first
|
|
- Shop-product relationships created automatically during import
|
|
- Check shop_products table for associations
|
|
|
|
**Permission Issues:**
|
|
- Shop owners can only manage their own shops
|
|
- Admin can manage all shops and users
|
|
- Verify user role and shop ownership
|
|
|
|
### Common API Issues
|
|
|
|
**Shop Not Found Errors:**
|
|
- Check shop_code spelling and case (stored uppercase)
|
|
- Verify shop is active and verified (for public access)
|
|
- Check user permissions for shop access
|
|
|
|
**CSV Import with Shop Code:**
|
|
- Shop code is required for all imports
|
|
- Shop must exist before importing
|
|
- User must have permission to import for that shop
|
|
|
|
## Migration Guide
|
|
|
|
### From v2.0 to v2.1 (Marketplace Update)
|
|
|
|
1. **Backup existing data**
|
|
2. **Update dependencies:** `pip install -r requirements.txt`
|
|
3. **Update environment variables** (add shop-related configs)
|
|
4. **Run application** - new tables will be created automatically
|
|
5. **Existing products remain in main catalog**
|
|
6. **Create shops for existing users**
|
|
7. **Update client applications** to use shop-specific endpoints
|
|
|
|
### Data Migration Script Example
|
|
|
|
```python
|
|
# Migrate existing products to demo shop
|
|
from models.database_models import Product, Shop, ShopProduct
|
|
from sqlalchemy.orm import Session
|
|
|
|
def migrate_to_shops(db: Session):
|
|
demo_shop = db.query(Shop).filter(Shop.shop_code == "DEMOSHOP").first()
|
|
products = db.query(Product).all()
|
|
|
|
for product in products:
|
|
shop_product = ShopProduct(
|
|
shop_id=demo_shop.id,
|
|
product_id=product.id,
|
|
shop_price=product.price,
|
|
shop_availability=product.availability,
|
|
is_active=True
|
|
)
|
|
db.add(shop_product)
|
|
|
|
db.commit()
|
|
```
|
|
|
|
## Contributing
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch: `git checkout -b feature-name`
|
|
3. Make changes with proper tests
|
|
4. Run security and quality checks
|
|
5. Update documentation if needed
|
|
6. Submit a pull request
|
|
|
|
### Code Quality Standards
|
|
|
|
- All endpoints must have proper authentication and authorization
|
|
- Shop ownership verification for protected operations
|
|
- Input validation using Pydantic models
|
|
- Comprehensive error handling with meaningful messages
|
|
- Unit tests for marketplace functionality
|
|
|
|
## License
|
|
|
|
This project is licensed under the MIT License - see the LICENSE file for details.
|
|
|
|
## About Letzshop
|
|
|
|
Letzshop is Luxembourg's premier e-commerce marketplace, connecting local and international vendors with Luxembourg customers. Our platform supports multi-vendor operations with advanced inventory management and seamless CSV import capabilities.
|
|
|
|
## Support
|
|
|
|
For marketplace-specific issues and vendor onboarding:
|
|
1. Check the troubleshooting section above
|
|
2. Review existing GitHub issues
|
|
3. Create a new issue with detailed information including:
|
|
- Shop code and user information
|
|
- CSV format and import details
|
|
- Error messages and logs
|
|
- Environment configuration (without secrets)
|
|
|
|
For vendor support: vendor-support@letzshop.lu
|
|
For technical issues: tech-support@letzshop.lu |