docs: add security audit + demo + POC builder proposal
4-phase plan for integrating scripts/security-audit/ into the prospecting module: security audit pipeline, report generation, live demo server, and POC site builder architecture. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
145
docs/proposals/security-audit-demo-poc-builder.md
Normal file
145
docs/proposals/security-audit-demo-poc-builder.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Security Audit + Live Demo + POC Builder Integration
|
||||
|
||||
## Context
|
||||
|
||||
The prospecting module scans websites for tech stack, performance, and contacts — but not security posture. Meanwhile, `scripts/security-audit/` has a standalone audit tool (audit.py) that checks HTTPS, headers, exposed files, cookies, etc., and a demo tool (demo.py) that clones a site and lets you demonstrate XSS/cookie theft/defacement live.
|
||||
|
||||
**Goal:** Integrate both into the platform as proper module features, plus architect a future POC site builder.
|
||||
|
||||
## Progress
|
||||
|
||||
- [x] Phase 1 foundation: `ProspectSecurityAudit` model, `last_security_audit_at` column, relationship on Prospect
|
||||
- [ ] Phase 1 remaining: constants, service, migration, endpoints, frontend
|
||||
- [ ] Phase 2: report generation
|
||||
- [ ] Phase 3: live demo server
|
||||
- [ ] Phase 4: POC builder (architecture only)
|
||||
|
||||
## Phase 1: Security Audit in Enrichment Pipeline
|
||||
|
||||
Slot security audit into the existing scan pipeline (alongside http-check, tech-scan, performance, contact-scrape).
|
||||
|
||||
### New files to create
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `prospecting/models/security_audit.py` | **DONE** — `ProspectSecurityAudit` model (1:1 on Prospect) |
|
||||
| `prospecting/services/security_audit_service.py` | `SecurityAuditService.run_audit(db, prospect)` — migrates `SecurityAudit` class logic from `scripts/security-audit/audit.py` (check_https, check_ssl, check_headers, check_exposed_files, check_cookies, check_server_info, check_technology) |
|
||||
| `prospecting/services/security_audit_constants.py` | TRANSLATIONS, EXPOSED_PATHS, SECURITY_HEADERS, SEVERITY_SCORES, GRADE_COLORS moved from audit.py |
|
||||
| `prospecting/schemas/security_audit.py` | Pydantic schemas: `SecurityAuditResponse`, `SecurityAuditSingleResponse`, `SecurityAuditBatchResponse` |
|
||||
| `prospecting/migrations/versions/prospecting_002_security_audit.py` | Creates `prospect_security_audits` table, adds `last_security_audit_at` to `prospects` |
|
||||
|
||||
### Files to modify
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `prospecting/models/prospect.py` | **DONE** — `last_security_audit_at` column + `security_audit` relationship |
|
||||
| `prospecting/models/__init__.py` | **DONE** — Export `ProspectSecurityAudit` |
|
||||
| `prospecting/services/prospect_service.py` | Add `get_pending_security_audit()` query |
|
||||
| `prospecting/routes/api/admin_enrichment.py` | Add `/security-audit/batch` + `/security-audit/{prospect_id}` endpoints. Add to `full_enrichment` pipeline. |
|
||||
| `prospecting/tasks/scan_tasks.py` | Add `batch_security_audit` Celery task + add to `full_enrichment` task |
|
||||
| `prospecting/static/admin/js/scan-jobs.js` | Add `security_audit: 'security-audit'` to batchRoutes |
|
||||
| `prospecting/templates/.../scan-jobs.html` | Add "Security Audit" batch button |
|
||||
| `prospecting/templates/.../prospect-detail.html` | Add "Security" tab with grade badge, score bar, findings by severity, "Run Audit" button |
|
||||
| `prospecting/static/admin/js/prospect-detail.js` | Add `runSecurityAudit()`, `severityColor()`, `gradeColor()` methods, `security_audit` tab handling |
|
||||
| `prospecting/schemas/prospect.py` | Add `security_audit` field to `ProspectDetailResponse` |
|
||||
|
||||
### Key design decisions
|
||||
- Findings stored as JSON (same as `tech_stack_json`, `lighthouse_json`) — variable structure
|
||||
- Denormalized severity counts for dashboard queries without JSON parsing
|
||||
- Reuse same `ScanBatchResponse` schema for batch endpoint
|
||||
- `SECURITY_AUDIT` already exists in `JobType` enum — no enum change needed
|
||||
|
||||
## Phase 2: Security Report Generation
|
||||
|
||||
Generate branded bilingual HTML reports from stored audit data.
|
||||
|
||||
### New files
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `prospecting/services/security_report_service.py` | `generate_html_report(audit, prospect, language, contact)` → standalone HTML string. Migrates `generate_report()` from audit.py (~300 lines of HTML template). |
|
||||
|
||||
### Files to modify
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `prospecting/routes/api/admin_enrichment.py` | Add `GET /security-audit/{prospect_id}/report?language=auto&download=false` → `HTMLResponse` |
|
||||
| `prospecting/templates/.../prospect-detail.html` | "Generate Report" button + language selector on security tab |
|
||||
| `prospecting/static/admin/js/prospect-detail.js` | `openSecurityReport(lang)` method |
|
||||
|
||||
## Phase 3: Live Demo Server
|
||||
|
||||
Integrate `demo.py`'s site cloner + hacker console as a managed background process.
|
||||
|
||||
### New files
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `prospecting/models/demo_session.py` | `DemoSession` model — prospect_id, status, port, pid, target_url, target_domain, started_at, stopped_at, error_message |
|
||||
| `prospecting/services/demo_service.py` | `DemoService` — start_demo (clone + spawn server), stop_demo, stop_all, get_active. Uses `multiprocessing.Process` to run HTTPServer. Port range 9000-9099. |
|
||||
| `prospecting/services/demo_constants.py` | HACKER_JS, HACKER_HTML_TEMPLATE, VULNERABLE_SEARCH_BAR, SPA_SCRIPT_DOMAINS migrated from demo.py |
|
||||
| `prospecting/schemas/demo_session.py` | `DemoSessionResponse` (includes computed hacker_url, site_url) |
|
||||
| `prospecting/routes/api/admin_demo.py` | `POST /demo/start/{prospect_id}`, `POST /demo/stop/{session_id}`, `GET /demo/active`, `POST /demo/stop-all` |
|
||||
| `prospecting/migrations/versions/prospecting_003_demo_sessions.py` | Creates `prospect_demo_sessions` table |
|
||||
|
||||
### Files to modify
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `prospecting/routes/api/admin.py` | Include demo router |
|
||||
| `prospecting/config.py` | Add `demo_port_range_start/end`, `demo_max_concurrent`, `demo_auto_stop_minutes` |
|
||||
| `prospecting/templates/.../prospect-detail.html` | "Launch Demo" / "Stop Demo" buttons + active demo indicator on security tab |
|
||||
| `prospecting/static/admin/js/prospect-detail.js` | `startDemo()`, `stopDemo()`, `openHackerConsole()`, periodic status check |
|
||||
|
||||
### Key design decisions
|
||||
- `multiprocessing.Process` (not subprocess) — can import SiteCloner/DemoHandler directly
|
||||
- Auto-stop after 60 minutes (configurable)
|
||||
- On app startup: cleanup stale RUNNING sessions from previous crashes
|
||||
- Max 5 concurrent demos (configurable)
|
||||
|
||||
## Phase 4: POC Site Builder (Architecture Only — No Implementation)
|
||||
|
||||
Separate from security demo. Builds a better version of the client's site as proof-of-concept.
|
||||
|
||||
### Where it fits
|
||||
- Triggered FROM prospect detail page ("Build POC Site" button)
|
||||
- Creates artifacts IN the hosting module (`HostedSite` with status=DRAFT → POC_READY)
|
||||
- Uses `HostedSite.prospect_id` FK (already exists) to link back
|
||||
|
||||
### Planned structure (future work)
|
||||
- `hosting/services/poc_builder_service.py` — `create_poc(db, prospect_id, template, config)`
|
||||
- `hosting/poc_templates/` — directory of starter templates with manifest
|
||||
- `POST /admin/hosting/poc/build/{prospect_id}` — trigger POC generation
|
||||
- `GET /admin/hosting/poc/templates` — list available templates
|
||||
- HostedSite additions: `poc_template`, `poc_source_audit_id`, `poc_config_json` columns
|
||||
|
||||
## Implementation Order
|
||||
|
||||
```
|
||||
Phase 1 (security audit pipeline) ← IN PROGRESS
|
||||
↓
|
||||
Phase 2 (report generation) ← depends on Phase 1 model
|
||||
↓ (can parallel with Phase 3)
|
||||
Phase 3 (live demo server) ← independent, but logically follows
|
||||
↓
|
||||
Phase 4 (POC builder) ← architecture only, deferred
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Phase 1
|
||||
python -m pytest app/modules/prospecting/tests/ -x -q --timeout=30
|
||||
# Manual: Run enrichment on prospect 1, check security tab shows grade/findings
|
||||
curl -X POST .../enrichment/security-audit/1 # single
|
||||
curl -X POST .../enrichment/security-audit/batch # batch
|
||||
|
||||
# Phase 2
|
||||
# Open report in browser:
|
||||
curl .../enrichment/security-audit/1/report > report.html && open report.html
|
||||
|
||||
# Phase 3
|
||||
# Start demo, open hacker console, verify attacks work, stop demo
|
||||
curl -X POST .../demo/start/1 # returns port + URLs
|
||||
curl -X POST .../demo/stop/{session_id}
|
||||
```
|
||||
Reference in New Issue
Block a user