diff --git a/docs/proposals/security-audit-demo-poc-builder.md b/docs/proposals/security-audit-demo-poc-builder.md new file mode 100644 index 00000000..15b8219f --- /dev/null +++ b/docs/proposals/security-audit-demo-poc-builder.md @@ -0,0 +1,145 @@ +# Security Audit + Live Demo + POC Builder Integration + +## Context + +The prospecting module scans websites for tech stack, performance, and contacts — but not security posture. Meanwhile, `scripts/security-audit/` has a standalone audit tool (audit.py) that checks HTTPS, headers, exposed files, cookies, etc., and a demo tool (demo.py) that clones a site and lets you demonstrate XSS/cookie theft/defacement live. + +**Goal:** Integrate both into the platform as proper module features, plus architect a future POC site builder. + +## Progress + +- [x] Phase 1 foundation: `ProspectSecurityAudit` model, `last_security_audit_at` column, relationship on Prospect +- [ ] Phase 1 remaining: constants, service, migration, endpoints, frontend +- [ ] Phase 2: report generation +- [ ] Phase 3: live demo server +- [ ] Phase 4: POC builder (architecture only) + +## Phase 1: Security Audit in Enrichment Pipeline + +Slot security audit into the existing scan pipeline (alongside http-check, tech-scan, performance, contact-scrape). + +### New files to create + +| File | Purpose | +|---|---| +| `prospecting/models/security_audit.py` | **DONE** — `ProspectSecurityAudit` model (1:1 on Prospect) | +| `prospecting/services/security_audit_service.py` | `SecurityAuditService.run_audit(db, prospect)` — migrates `SecurityAudit` class logic from `scripts/security-audit/audit.py` (check_https, check_ssl, check_headers, check_exposed_files, check_cookies, check_server_info, check_technology) | +| `prospecting/services/security_audit_constants.py` | TRANSLATIONS, EXPOSED_PATHS, SECURITY_HEADERS, SEVERITY_SCORES, GRADE_COLORS moved from audit.py | +| `prospecting/schemas/security_audit.py` | Pydantic schemas: `SecurityAuditResponse`, `SecurityAuditSingleResponse`, `SecurityAuditBatchResponse` | +| `prospecting/migrations/versions/prospecting_002_security_audit.py` | Creates `prospect_security_audits` table, adds `last_security_audit_at` to `prospects` | + +### Files to modify + +| File | Change | +|---|---| +| `prospecting/models/prospect.py` | **DONE** — `last_security_audit_at` column + `security_audit` relationship | +| `prospecting/models/__init__.py` | **DONE** — Export `ProspectSecurityAudit` | +| `prospecting/services/prospect_service.py` | Add `get_pending_security_audit()` query | +| `prospecting/routes/api/admin_enrichment.py` | Add `/security-audit/batch` + `/security-audit/{prospect_id}` endpoints. Add to `full_enrichment` pipeline. | +| `prospecting/tasks/scan_tasks.py` | Add `batch_security_audit` Celery task + add to `full_enrichment` task | +| `prospecting/static/admin/js/scan-jobs.js` | Add `security_audit: 'security-audit'` to batchRoutes | +| `prospecting/templates/.../scan-jobs.html` | Add "Security Audit" batch button | +| `prospecting/templates/.../prospect-detail.html` | Add "Security" tab with grade badge, score bar, findings by severity, "Run Audit" button | +| `prospecting/static/admin/js/prospect-detail.js` | Add `runSecurityAudit()`, `severityColor()`, `gradeColor()` methods, `security_audit` tab handling | +| `prospecting/schemas/prospect.py` | Add `security_audit` field to `ProspectDetailResponse` | + +### Key design decisions +- Findings stored as JSON (same as `tech_stack_json`, `lighthouse_json`) — variable structure +- Denormalized severity counts for dashboard queries without JSON parsing +- Reuse same `ScanBatchResponse` schema for batch endpoint +- `SECURITY_AUDIT` already exists in `JobType` enum — no enum change needed + +## Phase 2: Security Report Generation + +Generate branded bilingual HTML reports from stored audit data. + +### New files + +| File | Purpose | +|---|---| +| `prospecting/services/security_report_service.py` | `generate_html_report(audit, prospect, language, contact)` → standalone HTML string. Migrates `generate_report()` from audit.py (~300 lines of HTML template). | + +### Files to modify + +| File | Change | +|---|---| +| `prospecting/routes/api/admin_enrichment.py` | Add `GET /security-audit/{prospect_id}/report?language=auto&download=false` → `HTMLResponse` | +| `prospecting/templates/.../prospect-detail.html` | "Generate Report" button + language selector on security tab | +| `prospecting/static/admin/js/prospect-detail.js` | `openSecurityReport(lang)` method | + +## Phase 3: Live Demo Server + +Integrate `demo.py`'s site cloner + hacker console as a managed background process. + +### New files + +| File | Purpose | +|---|---| +| `prospecting/models/demo_session.py` | `DemoSession` model — prospect_id, status, port, pid, target_url, target_domain, started_at, stopped_at, error_message | +| `prospecting/services/demo_service.py` | `DemoService` — start_demo (clone + spawn server), stop_demo, stop_all, get_active. Uses `multiprocessing.Process` to run HTTPServer. Port range 9000-9099. | +| `prospecting/services/demo_constants.py` | HACKER_JS, HACKER_HTML_TEMPLATE, VULNERABLE_SEARCH_BAR, SPA_SCRIPT_DOMAINS migrated from demo.py | +| `prospecting/schemas/demo_session.py` | `DemoSessionResponse` (includes computed hacker_url, site_url) | +| `prospecting/routes/api/admin_demo.py` | `POST /demo/start/{prospect_id}`, `POST /demo/stop/{session_id}`, `GET /demo/active`, `POST /demo/stop-all` | +| `prospecting/migrations/versions/prospecting_003_demo_sessions.py` | Creates `prospect_demo_sessions` table | + +### Files to modify + +| File | Change | +|---|---| +| `prospecting/routes/api/admin.py` | Include demo router | +| `prospecting/config.py` | Add `demo_port_range_start/end`, `demo_max_concurrent`, `demo_auto_stop_minutes` | +| `prospecting/templates/.../prospect-detail.html` | "Launch Demo" / "Stop Demo" buttons + active demo indicator on security tab | +| `prospecting/static/admin/js/prospect-detail.js` | `startDemo()`, `stopDemo()`, `openHackerConsole()`, periodic status check | + +### Key design decisions +- `multiprocessing.Process` (not subprocess) — can import SiteCloner/DemoHandler directly +- Auto-stop after 60 minutes (configurable) +- On app startup: cleanup stale RUNNING sessions from previous crashes +- Max 5 concurrent demos (configurable) + +## Phase 4: POC Site Builder (Architecture Only — No Implementation) + +Separate from security demo. Builds a better version of the client's site as proof-of-concept. + +### Where it fits +- Triggered FROM prospect detail page ("Build POC Site" button) +- Creates artifacts IN the hosting module (`HostedSite` with status=DRAFT → POC_READY) +- Uses `HostedSite.prospect_id` FK (already exists) to link back + +### Planned structure (future work) +- `hosting/services/poc_builder_service.py` — `create_poc(db, prospect_id, template, config)` +- `hosting/poc_templates/` — directory of starter templates with manifest +- `POST /admin/hosting/poc/build/{prospect_id}` — trigger POC generation +- `GET /admin/hosting/poc/templates` — list available templates +- HostedSite additions: `poc_template`, `poc_source_audit_id`, `poc_config_json` columns + +## Implementation Order + +``` +Phase 1 (security audit pipeline) ← IN PROGRESS + ↓ +Phase 2 (report generation) ← depends on Phase 1 model + ↓ (can parallel with Phase 3) +Phase 3 (live demo server) ← independent, but logically follows + ↓ +Phase 4 (POC builder) ← architecture only, deferred +``` + +## Verification + +```bash +# Phase 1 +python -m pytest app/modules/prospecting/tests/ -x -q --timeout=30 +# Manual: Run enrichment on prospect 1, check security tab shows grade/findings +curl -X POST .../enrichment/security-audit/1 # single +curl -X POST .../enrichment/security-audit/batch # batch + +# Phase 2 +# Open report in browser: +curl .../enrichment/security-audit/1/report > report.html && open report.html + +# Phase 3 +# Start demo, open hacker console, verify attacks work, stop demo +curl -X POST .../demo/start/1 # returns port + URLs +curl -X POST .../demo/stop/{session_id} +```