sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Samir Boulahtit
2025-09-05 17:27:39 +02:00
commit 9dd177bddc
36 changed files with 3755 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

569
updated_readme.md Normal file
View File

@@ -0,0 +1,569 @@
# Ecommerce Backend API v2.1
A robust, production-ready FastAPI backend for ecommerce product catalog and inventory management with JWT authentication and advanced CSV import capabilities.
## Key Features
### Security & Authentication
- **JWT Authentication**: Secure token-based authentication with configurable expiration (30 minutes default)
- **User Management**: Registration, login, role-based access control (Admin/User roles)
- **Password Security**: Bcrypt hashing for secure password storage
- **Protected Endpoints**: All product management operations require authentication
- **Default Admin Account**: Auto-created admin user for immediate system access
### Architecture Improvements
- **Modular Design**: Separated concerns into utility modules, middleware, and models
- **Database Optimization**: Added proper indexing strategy and foreign key relationships
- **Connection Pooling**: PostgreSQL support with connection pooling for production scalability
- **Background Processing**: Asynchronous CSV import with job tracking
### Performance Optimizations
- **Batch Processing**: CSV imports processed in configurable batches
- **Database Indexes**: Strategic indexing for common query patterns
- **Streaming Export**: Memory-efficient CSV export for large datasets
- **Rate Limiting**: Sliding window rate limiter to prevent API abuse
### Data Processing
- **Robust GTIN Handling**: Centralized GTIN normalization and validation
- **Multi-currency Support**: Advanced price parsing with currency extraction
- **International Content**: Multi-encoding CSV support for global data
## Project Structure
```
ecommerce_api/
├── main.py # FastAPI application entry point with auth
├── models/
│ ├── database_models.py # SQLAlchemy ORM models (User, Product, Stock, ImportJob)
│ └── api_models.py # Pydantic API models with auth models
├── utils/
│ ├── data_processing.py # GTIN and price processing utilities
│ ├── csv_processor.py # CSV import/export handling
│ └── database.py # Database configuration
├── middleware/
│ ├── auth.py # JWT authentication with bcrypt
│ ├── rate_limiter.py # Rate limiting implementation
│ ├── error_handler.py # Centralized error handling
│ └── logging_middleware.py # Request/response logging
├── tests/
│ └── test_auth.py # Authentication tests
├── requirements.txt # Python dependencies with auth packages
└── README.md # This file
```
## Quick Start
### 1. Installation
```bash
# Clone the repository
git clone <repository-url>
cd ecommerce-api
# Set up virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
```
### 2. Environment Configuration
Create a `.env` file in the project root:
```env
# Database
DATABASE_URL=postgresql://user:password@localhost:5432/ecommerce_db
# For SQLite (development): DATABASE_URL=sqlite:///./ecommerce.db
# JWT Configuration
JWT_SECRET_KEY=your-super-secret-key-change-in-production-immediately
JWT_EXPIRE_MINUTES=30
# Server Configuration
API_HOST=0.0.0.0
API_PORT=8000
DEBUG=False
```
**Important Security Note**: Always change the `JWT_SECRET_KEY` in production!
### 3. Database Setup
**For SQLite (Development):**
```bash
# Run the application - it will create tables automatically
python main.py
```
**For PostgreSQL (Production):**
```bash
# Create PostgreSQL database
createdb ecommerce_db
# Run the application - it will create tables and indexes automatically
python main.py
```
### 4. Start the Server
```bash
# Development server
uvicorn main:app --reload --host 0.0.0.0 --port 8000
# Production server
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
```
The API will be available at `http://localhost:8000`
### 5. Default Admin Access
The system automatically creates a default admin user:
- **Username**: `admin`
- **Password**: `admin123`
- **Email**: `admin@example.com`
**Security Warning**: Change the admin password immediately in production!
## Authentication Flow
### 1. Register a New User
```bash
curl -X POST "http://localhost:8000/register" \
-H "Content-Type: application/json" \
-d '{
"email": "user@example.com",
"username": "newuser",
"password": "securepassword123"
}'
```
### 2. Login and Get JWT Token
```bash
curl -X POST "http://localhost:8000/login" \
-H "Content-Type: application/json" \
-d '{
"username": "admin",
"password": "admin123"
}'
```
Response:
```json
{
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"token_type": "bearer",
"expires_in": 1800,
"user": {
"id": 1,
"username": "admin",
"email": "admin@example.com",
"role": "admin",
"is_active": true
}
}
```
### 3. Use Token for Protected Endpoints
```bash
curl -X GET "http://localhost:8000/products" \
-H "Authorization: Bearer YOUR_JWT_TOKEN_HERE"
```
## API Endpoints
### Public Endpoints
- `GET /` - API information
- `GET /health` - Health check
- `POST /register` - Register new user
- `POST /login` - Login and get JWT token
### Protected Endpoints (Require Authentication)
#### User Management
- `GET /me` - Get current user information
#### Products (All users)
- `GET /products` - List products with filtering and search
- `POST /products` - Create new product
- `GET /products/{product_id}` - Get product with stock info
- `PUT /products/{product_id}` - Update product
- `DELETE /products/{product_id}` - Delete product and associated stock
#### Stock Management (All users)
- `POST /stock` - Set exact stock quantity
- `GET /stock/{gtin}` - Get stock summary by GTIN
#### CSV Operations (All users)
- `POST /import-csv` - Start background CSV import
- `GET /import-status/{job_id}` - Check import job status (own jobs only)
- `GET /export-csv` - Export products as CSV (streaming)
#### Statistics (All users)
- `GET /stats` - System statistics
#### Admin-Only Endpoints
- `GET /admin/users` - List all users
- `PUT /admin/users/{user_id}/status` - Activate/deactivate users
## User Roles and Permissions
### Regular Users
- Can register and login
- Access all product and stock management features
- Can import/export CSV files
- Can only view their own import jobs
- Cannot manage other users
### Admin Users
- All regular user permissions
- Can view and manage all users
- Can view all import jobs from any user
- Can activate/deactivate user accounts
## Security Features
### Password Security
- Passwords hashed using bcrypt with salt
- Minimum password length: 6 characters
- No password storage in plaintext anywhere
### JWT Token Security
- Tokens include expiration timestamp
- Tokens include user role and permissions
- Configurable expiration time (default: 30 minutes)
- Secure token validation on each request
### Rate Limiting
- CSV imports: 10 requests per hour
- General API: 100 requests per hour per client
- Customizable per endpoint
### Input Validation
- All inputs validated using Pydantic models
- Email format validation for registration
- Username alphanumeric validation
- GTIN format validation and normalization
## Advanced Features
### Background CSV Import
Import large CSV files asynchronously:
```python
import requests
# Start import
response = requests.post(
'http://localhost:8000/import-csv',
headers={'Authorization': 'Bearer YOUR_TOKEN'},
json={
'url': 'https://example.com/products.csv',
'batch_size': 1000
}
)
job_id = response.json()['job_id']
# Check status
status_response = requests.get(
f'http://localhost:8000/import-status/{job_id}',
headers={'Authorization': 'Bearer YOUR_TOKEN'}
)
print(status_response.json())
```
### Product Search and Filtering
```bash
# Search in title and description
GET /products?search=laptop
# Filter by brand and category
GET /products?brand=Apple&category=Electronics
# Combine filters with pagination
GET /products?brand=Samsung&availability=in%20stock&search=phone&skip=0&limit=50
```
### Stock Management
```bash
# Set stock for a specific location
curl -X POST "http://localhost:8000/stock" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"gtin": "1234567890123",
"location": "WAREHOUSE_A",
"quantity": 100
}'
# Get stock summary
curl -X GET "http://localhost:8000/stock/1234567890123" \
-H "Authorization: Bearer YOUR_TOKEN"
```
## Database Schema
### Users Table
```sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR UNIQUE NOT NULL,
username VARCHAR UNIQUE NOT NULL,
hashed_password VARCHAR NOT NULL,
role VARCHAR DEFAULT 'user',
is_active BOOLEAN DEFAULT true,
last_login TIMESTAMP,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
### Products Table
- Full product catalog with Google Shopping compatibility
- Indexed fields: `gtin`, `brand`, `google_product_category`, `availability`
- Supports all Google Shopping feed attributes
### Stock Table
- Location-based inventory tracking
- GTIN-based product linking
- Unique constraint on GTIN+location combinations
### Import Jobs Table
- Track background import operations
- User ownership and access control
- Status monitoring and error handling
## Development
### Running Tests
```bash
# Install test dependencies
pip install pytest pytest-asyncio httpx
# Run all tests
pytest
# Run with coverage
pytest --cov=. tests/
```
### Development Server with Auto-reload
```bash
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```
## Production Deployment
### Security Checklist
- [ ] Change default admin password immediately
- [ ] Set strong JWT_SECRET_KEY (32+ random characters)
- [ ] Configure JWT_EXPIRE_MINUTES appropriately
- [ ] Set up HTTPS/TLS termination
- [ ] Configure CORS for your frontend domains only
- [ ] Set up database connection limits and pooling
- [ ] Enable request logging and monitoring
- [ ] Configure rate limiting per your needs
- [ ] Set up user account monitoring and alerting
- [ ] Regular security audits of user accounts
### Environment Variables for Production
```env
# Security
JWT_SECRET_KEY=your-very-long-random-secret-key-at-least-32-characters
JWT_EXPIRE_MINUTES=30
# Database (use PostgreSQL in production)
DATABASE_URL=postgresql://user:password@db-host:5432/ecommerce_prod
# Server
DEBUG=False
API_HOST=0.0.0.0
API_PORT=8000
# Optional: External services
REDIS_URL=redis://redis-host:6379/0
```
### Docker Deployment
```yaml
# docker-compose.yml
version: '3.8'
services:
db:
image: postgres:15
environment:
POSTGRES_DB: ecommerce
POSTGRES_USER: ecommerce_user
POSTGRES_PASSWORD: secure_password
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
api:
build: .
environment:
DATABASE_URL: postgresql://ecommerce_user:secure_password@db:5432/ecommerce
JWT_SECRET_KEY: your-production-secret-key
JWT_EXPIRE_MINUTES: 30
ports:
- "8000:8000"
depends_on:
- db
restart: unless-stopped
volumes:
postgres_data:
```
### Nginx Configuration
```nginx
server {
listen 80;
server_name your-api-domain.com;
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
## Troubleshooting
### Authentication Issues
**Invalid Token Errors:**
- Check token expiration (default 30 minutes)
- Verify JWT_SECRET_KEY is consistent
- Ensure Bearer token format: `Authorization: Bearer <token>`
**Login Failures:**
- Verify username/email and password
- Check if user account is active (`is_active: true`)
- Review user registration process
**Permission Denied:**
- Confirm user role permissions
- Check endpoint access requirements
- Verify token contains correct role information
### Database Issues
**Connection Errors:**
- Verify DATABASE_URL format and credentials
- Check database server accessibility
- Monitor connection pool limits
**Migration Issues:**
- Tables are created automatically on startup
- For schema changes, implement proper migrations
- Backup data before major updates
### Common API Issues
**CSV Import Failures:**
- Check file URL accessibility
- Verify CSV format and encoding
- Monitor import job status for detailed errors
- Ensure proper authentication token
**Rate Limiting:**
- Default limits: 100 requests/hour, 10 CSV imports/hour
- Check rate limit headers in responses
- Implement proper retry logic with backoff
### Logging and Monitoring
Application logs include:
- Authentication events (login, failed attempts)
- API request/response times
- Import job progress and errors
- Rate limiting events
- Database query performance
```bash
# View logs in development
python main.py # Logs to console
# Docker logs
docker-compose logs -f api
```
## Migration from v2.0
If upgrading from v2.0 to v2.1:
1. **Install new dependencies:**
```bash
pip install -r requirements.txt
```
2. **Update environment variables:**
```bash
echo "JWT_SECRET_KEY=your-secret-key" >> .env
echo "JWT_EXPIRE_MINUTES=30" >> .env
```
3. **Database migration:**
The application will automatically create the new `users` table and update the `import_jobs` table on startup.
4. **Update client code:**
- Add authentication to all product management API calls
- Implement login flow in your frontend
- Handle JWT token refresh
## Contributing
1. Fork the repository
2. Create a feature branch: `git checkout -b feature-name`
3. Make changes with proper tests
4. Run security and quality checks
5. Update documentation if needed
6. Submit a pull request
### Code Quality Standards
- All endpoints must have proper authentication
- Password handling must use bcrypt
- JWT tokens must have expiration
- Input validation using Pydantic models
- Comprehensive error handling
- Unit tests for authentication logic
## License
This project is licensed under the MIT License - see the LICENSE file for details.
## Security
For security issues, please email the maintainers directly instead of creating a public issue.
## Support
For issues and questions:
1. Check the troubleshooting section above
2. Review existing GitHub issues
3. Create a new issue with detailed information including:
- Authentication steps you've tried
- Error messages and logs
- API endpoint and request details
- Environment configuration (without secrets)