sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added marketplace support

Browse Source
This commit is contained in:
Samir Boulahtit
2025-09-05 22:14:52 +02:00
parent 9dd177bddc
commit 4fb67e594d
6 changed files with 1307 additions and 117 deletions
Download Patch File Download Diff File Expand all files Collapse all files

663
updated_readme_marketplace.md Normal file
View File

@@ -0,0 +1,663 @@
# Letzshop Marketplace API v2.1
A robust, production-ready FastAPI backend for Luxembourg's premier e-commerce marketplace with multi-vendor support, JWT authentication, and advanced CSV import capabilities.
## Key Features
### Marketplace Architecture
- **Multi-Vendor Support**: Shops can import and manage their product catalogs independently
- **Centralized Product Catalog**: Products exist in main marketplace with shop-specific overrides
- **Shop Management**: Complete vendor onboarding, verification, and management system
- **Shop-Specific Pricing**: Vendors can set their own prices, availability, and conditions
- **Marketplace Controls**: Admin verification and quality control for vendor shops
### Security & Authentication
- **JWT Authentication**: Secure token-based authentication with configurable expiration (30 minutes default)
- **User Management**: Registration, login, role-based access control (Admin/User/Shop Owner roles)
- **Password Security**: Bcrypt hashing for secure password storage
- **Protected Endpoints**: All operations require authentication with proper authorization
- **Default Admin Account**: Auto-created admin user for immediate system access
### Architecture Improvements
- **Modular Design**: Separated concerns into utility modules, middleware, and models
- **Database Optimization**: Added proper indexing strategy and foreign key relationships
- **Connection Pooling**: PostgreSQL support with connection pooling for production scalability
- **Background Processing**: Asynchronous CSV import with job tracking per shop
### Performance Optimizations
- **Batch Processing**: CSV imports processed in configurable batches
- **Database Indexes**: Strategic indexing for common query patterns including shop relationships
- **Streaming Export**: Memory-efficient CSV export for large datasets with shop filtering
- **Rate Limiting**: Sliding window rate limiter to prevent API abuse
### Data Processing
- **Robust GTIN Handling**: Centralized GTIN normalization and validation
- **Multi-currency Support**: Advanced price parsing with currency extraction
- **International Content**: Multi-encoding CSV support for global data
- **Shop Association**: Automatic product-shop linking during CSV imports
## Project Structure
```
letzshop_api/
├── main.py # FastAPI application entry point with marketplace support
├── models/
│ ├── database_models.py # SQLAlchemy ORM models (User, Shop, Product, ShopProduct, Stock, ImportJob)
│ └── api_models.py # Pydantic API models with shop and auth models
├── utils/
│ ├── data_processing.py # GTIN and price processing utilities
│ ├── csv_processor.py # CSV import/export handling with shop support
│ └── database.py # Database configuration
├── middleware/
│ ├── auth.py # JWT authentication with bcrypt
│ ├── rate_limiter.py # Rate limiting implementation
│ ├── error_handler.py # Centralized error handling
│ └── logging_middleware.py # Request/response logging
├── tests/
│ └── test_auth.py # Authentication tests
├── requirements.txt # Python dependencies with auth packages
└── README.md # This file
```
## Quick Start
### 1. Installation
```bash
# Clone the repository
git clone <repository-url>
cd letzshop-api
# Set up virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
```
### 2. Environment Configuration
Create a `.env` file in the project root:
```env
# Database
DATABASE_URL=postgresql://user:password@localhost:5432/letzshop_db
# For SQLite (development): DATABASE_URL=sqlite:///./letzshop.db
# JWT Configuration
JWT_SECRET_KEY=your-super-secret-key-change-in-production-immediately
JWT_EXPIRE_MINUTES=30
# Server Configuration
API_HOST=0.0.0.0
API_PORT=8000
DEBUG=False
```
**Important Security Note**: Always change the `JWT_SECRET_KEY` in production!
### 3. Database Setup
**For SQLite (Development):**
```bash
# Run the application - it will create tables automatically
python main.py
```
**For PostgreSQL (Production):**
```bash
# Create PostgreSQL database
createdb letzshop_db
# Run the application - it will create tables and indexes automatically
python main.py
```
### 4. Start the Server
```bash
# Development server
uvicorn main:app --reload --host 0.0.0.0 --port 8000
# Production server
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
```
The API will be available at `http://localhost:8000`
### 5. Default Access
The system automatically creates:
- **Admin User**: `admin` / `admin123` / `admin@example.com`
- **Demo Shop**: `DEMOSHOP` owned by admin for testing
**Security Warning**: Change the admin password immediately in production!
## Authentication Flow
### 1. Register a New User
```bash
curl -X POST "http://localhost:8000/register" \
-H "Content-Type: application/json" \
-d '{
"email": "vendor@example.com",
"username": "newvendor",
"password": "securepassword123"
}'
```
### 2. Login and Get JWT Token
```bash
curl -X POST "http://localhost:8000/login" \
-H "Content-Type: application/json" \
-d '{
"username": "admin",
"password": "admin123"
}'
```
Response:
```json
{
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"token_type": "bearer",
"expires_in": 1800,
"user": {
"id": 1,
"username": "admin",
"email": "admin@example.com",
"role": "admin",
"is_active": true
}
}
```
### 3. Use Token for Protected Endpoints
```bash
curl -X GET "http://localhost:8000/shops" \
-H "Authorization: Bearer YOUR_JWT_TOKEN_HERE"
```
## Marketplace Workflow
### 1. Create a Shop
```bash
curl -X POST "http://localhost:8000/shops" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"shop_code": "TECHSTORE",
"shop_name": "Tech Store Luxembourg",
"description": "Electronics and gadgets for Luxembourg",
"contact_email": "info@techstore.lu",
"contact_phone": "+352 123 456 789",
"website": "https://techstore.lu"
}'
```
### 2. Import Products for Your Shop
```bash
curl -X POST "http://localhost:8000/import-csv" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"url": "https://techstore.com/products.csv",
"shop_code": "TECHSTORE",
"batch_size": 1000
}'
```
### 3. Monitor Import Progress
```bash
curl -X GET "http://localhost:8000/import-status/1" \
-H "Authorization: Bearer YOUR_TOKEN"
```
### 4. View Shop Products
```bash
curl -X GET "http://localhost:8000/products?shop_code=TECHSTORE" \
-H "Authorization: Bearer YOUR_TOKEN"
```
## API Endpoints
### Public Endpoints
- `GET /` - API information
- `GET /health` - Health check
- `POST /register` - Register new user
- `POST /login` - Login and get JWT token
### Protected Endpoints (Require Authentication)
#### User Management
- `GET /me` - Get current user information
#### Shop Management
- `POST /shops` - Create new shop
- `GET /shops` - List shops with filtering
- `GET /shops/{shop_code}` - Get shop details
- `PUT /shops/{shop_code}` - Update shop (owners only)
- `POST /shops/{shop_code}/products` - Add product to shop catalog
- `GET /shops/{shop_code}/products` - Get shop products
#### Products (Marketplace Catalog)
- `GET /products` - List products with filtering (optionally by shop)
- `POST /products` - Create new product in marketplace catalog
- `GET /products/{product_id}` - Get product with stock info and shop listings
- `PUT /products/{product_id}` - Update product
- `DELETE /products/{product_id}` - Delete product and associated shop listings
#### Stock Management
- `POST /stock` - Set stock quantity (with optional shop association)
- `GET /stock/{gtin}` - Get stock summary by GTIN
#### CSV Operations
- `POST /import-csv` - Start background CSV import for specific shop
- `GET /import-status/{job_id}` - Check import job status
- `GET /export-csv` - Export products as CSV (optionally filtered by shop)
#### Statistics
- `GET /stats` - Marketplace statistics
#### Admin-Only Endpoints
- `GET /admin/users` - List all users
- `PUT /admin/users/{user_id}/status` - Activate/deactivate users
- `GET /admin/shops` - List all shops
- `PUT /admin/shops/{shop_id}/verify` - Verify/unverify shop
- `PUT /admin/shops/{shop_id}/status` - Activate/deactivate shop
- `GET /admin/import-jobs` - View all import jobs
## User Roles and Permissions
### Regular Users
- Can register and login
- Can create and manage their own shops
- Can import products for their shops
- Can manage stock for their products
- Can view marketplace products and shops
### Shop Owners (Regular Users with Shops)
- All regular user permissions
- Can manage their shop information
- Can import/export products for their shops
- Can set shop-specific pricing and availability
- Can view their import job history
### Admin Users
- All user permissions
- Can view and manage all users and shops
- Can verify/unverify shops
- Can view all import jobs from any shop
- Can activate/deactivate user accounts and shops
## Marketplace Features
### Shop Verification System
- New shops start as unverified
- Admin approval required for public visibility
- Verified shops appear in public marketplace listings
- Quality control through admin verification
### Multi-Vendor Product Catalog
- Products exist in central marketplace catalog
- Multiple shops can sell the same product
- Shop-specific pricing, availability, and conditions
- Automatic product matching during CSV imports
### Shop-Specific Overrides
```json
{
"product_id": "LAPTOP123",
"shop_price": 999.99,
"shop_currency": "EUR",
"shop_availability": "in stock",
"shop_condition": "new",
"is_featured": true,
"min_quantity": 1,
"max_quantity": 5
}
```
### Advanced Product Search
```bash
# Search products in specific shop
GET /products?shop_code=TECHSTORE&search=laptop
# Search across all verified shops
GET /products?search=laptop&availability=in%20stock
# Filter by brand and category
GET /products?brand=Apple&category=Electronics
```
## Database Schema
### Core Tables
#### Users Table
```sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR UNIQUE NOT NULL,
username VARCHAR UNIQUE NOT NULL,
hashed_password VARCHAR NOT NULL,
role VARCHAR DEFAULT 'user',
is_active BOOLEAN DEFAULT true,
last_login TIMESTAMP,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
#### Shops Table
```sql
CREATE TABLE shops (
id SERIAL PRIMARY KEY,
shop_code VARCHAR UNIQUE NOT NULL,
shop_name VARCHAR NOT NULL,
description TEXT,
owner_id INTEGER REFERENCES users(id),
contact_email VARCHAR,
contact_phone VARCHAR,
website VARCHAR,
business_address TEXT,
tax_number VARCHAR,
is_active BOOLEAN DEFAULT true,
is_verified BOOLEAN DEFAULT false,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
#### Products Table
- Main marketplace catalog with Google Shopping compatibility
- Indexed fields: `gtin`, `brand`, `google_product_category`, `availability`
- Supports all Google Shopping feed attributes
#### ShopProducts Table
```sql
CREATE TABLE shop_products (
id SERIAL PRIMARY KEY,
shop_id INTEGER REFERENCES shops(id),
product_id INTEGER REFERENCES products(id),
shop_product_id VARCHAR,
shop_price DECIMAL,
shop_sale_price DECIMAL,
shop_currency VARCHAR,
shop_availability VARCHAR,
shop_condition VARCHAR,
is_featured BOOLEAN DEFAULT false,
is_active BOOLEAN DEFAULT true,
min_quantity INTEGER DEFAULT 1,
max_quantity INTEGER,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW(),
UNIQUE(shop_id, product_id)
);
```
#### Stock Table
- Location-based inventory tracking with optional shop association
- GTIN-based product linking
- Support for reserved quantities (for order processing)
#### Import Jobs Table
- Track background import operations per shop
- User and shop ownership tracking
- Status monitoring and error handling
## Advanced Features
### Shop-Specific CSV Import
Import products with automatic shop association:
```python
import requests
# Start import for specific shop
response = requests.post(
'http://localhost:8000/import-csv',
headers={'Authorization': 'Bearer YOUR_TOKEN'},
json={
'url': 'https://myshop.com/products.csv',
'shop_code': 'MYSHOP',
'batch_size': 1000
}
)
job_id = response.json()['job_id']
# Monitor progress
status_response = requests.get(
f'http://localhost:8000/import-status/{job_id}',
headers={'Authorization': 'Bearer YOUR_TOKEN'}
)
print(status_response.json())
```
### Multi-Shop Product Management
```bash
# Add existing marketplace product to your shop
curl -X POST "http://localhost:8000/shops/MYSHOP/products" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"product_id": "EXISTING_PRODUCT_123",
"shop_price": 89.99,
"shop_availability": "in stock",
"is_featured": true
}'
# Get products from specific shop
curl -X GET "http://localhost:8000/shops/MYSHOP/products" \
-H "Authorization: Bearer YOUR_TOKEN"
```
### Stock Management with Shop Context
```bash
# Set shop-specific stock
curl -X POST "http://localhost:8000/stock" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"gtin": "1234567890123",
"location": "MYSHOP_WAREHOUSE",
"quantity": 50,
"shop_code": "MYSHOP"
}'
```
## Production Deployment
### Security Checklist for Marketplace
- [ ] Change default admin password immediately
- [ ] Set strong JWT_SECRET_KEY (32+ random characters)
- [ ] Configure JWT_EXPIRE_MINUTES appropriately
- [ ] Set up HTTPS/TLS termination
- [ ] Configure CORS for your frontend domains only
- [ ] Set up database connection limits and pooling
- [ ] Enable request logging and monitoring
- [ ] Configure rate limiting per your needs
- [ ] Set up shop verification workflow
- [ ] Implement shop quality monitoring
- [ ] Set up automated backup for shop data
- [ ] Configure email notifications for shop owners
- [ ] Regular security audits of user accounts and shops
### Environment Variables for Production
```env
# Security
JWT_SECRET_KEY=your-very-long-random-secret-key-at-least-32-characters
JWT_EXPIRE_MINUTES=30
# Database (use PostgreSQL in production)
DATABASE_URL=postgresql://user:password@db-host:5432/letzshop_prod
# Server
DEBUG=False
API_HOST=0.0.0.0
API_PORT=8000
# Marketplace Configuration
MARKETPLACE_NAME=Letzshop
DEFAULT_CURRENCY=EUR
ADMIN_EMAIL=admin@letzshop.lu
# Optional: External services
REDIS_URL=redis://redis-host:6379/0
EMAIL_API_KEY=your-email-service-key
```
### Docker Deployment
```yaml
# docker-compose.yml
version: '3.8'
services:
db:
image: postgres:15
environment:
POSTGRES_DB: letzshop
POSTGRES_USER: letzshop_user
POSTGRES_PASSWORD: secure_password
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
api:
build: .
environment:
DATABASE_URL: postgresql://letzshop_user:secure_password@db:5432/letzshop
JWT_SECRET_KEY: your-production-secret-key
JWT_EXPIRE_MINUTES: 30
MARKETPLACE_NAME: Letzshop
ports:
- "8000:8000"
depends_on:
- db
restart: unless-stopped
volumes:
postgres_data:
```
## Troubleshooting
### Marketplace-Specific Issues
**Shop Import Failures:**
- Verify shop exists and is active
- Check user permissions for the shop
- Ensure CSV format is compatible
- Monitor import job status for detailed errors
**Shop Product Association:**
- Products are added to main catalog first
- Shop-product relationships created automatically during import
- Check shop_products table for associations
**Permission Issues:**
- Shop owners can only manage their own shops
- Admin can manage all shops and users
- Verify user role and shop ownership
### Common API Issues
**Shop Not Found Errors:**
- Check shop_code spelling and case (stored uppercase)
- Verify shop is active and verified (for public access)
- Check user permissions for shop access
**CSV Import with Shop Code:**
- Shop code is required for all imports
- Shop must exist before importing
- User must have permission to import for that shop
## Migration Guide
### From v2.0 to v2.1 (Marketplace Update)
1. **Backup existing data**
2. **Update dependencies:** `pip install -r requirements.txt`
3. **Update environment variables** (add shop-related configs)
4. **Run application** - new tables will be created automatically
5. **Existing products remain in main catalog**
6. **Create shops for existing users**
7. **Update client applications** to use shop-specific endpoints
### Data Migration Script Example
```python
# Migrate existing products to demo shop
from models.database_models import Product, Shop, ShopProduct
from sqlalchemy.orm import Session
def migrate_to_shops(db: Session):
demo_shop = db.query(Shop).filter(Shop.shop_code == "DEMOSHOP").first()
products = db.query(Product).all()
for product in products:
shop_product = ShopProduct(
shop_id=demo_shop.id,
product_id=product.id,
shop_price=product.price,
shop_availability=product.availability,
is_active=True
)
db.add(shop_product)
db.commit()
```
## Contributing
1. Fork the repository
2. Create a feature branch: `git checkout -b feature-name`
3. Make changes with proper tests
4. Run security and quality checks
5. Update documentation if needed
6. Submit a pull request
### Code Quality Standards
- All endpoints must have proper authentication and authorization
- Shop ownership verification for protected operations
- Input validation using Pydantic models
- Comprehensive error handling with meaningful messages
- Unit tests for marketplace functionality
## License
This project is licensed under the MIT License - see the LICENSE file for details.
## About Letzshop
Letzshop is Luxembourg's premier e-commerce marketplace, connecting local and international vendors with Luxembourg customers. Our platform supports multi-vendor operations with advanced inventory management and seamless CSV import capabilities.
## Support
For marketplace-specific issues and vendor onboarding:
1. Check the troubleshooting section above
2. Review existing GitHub issues
3. Create a new issue with detailed information including:
- Shop code and user information
- CSV format and import details
- Error messages and logs
- Environment configuration (without secrets)
For vendor support: vendor-support@letzshop.lu
For technical issues: tech-support@letzshop.lu