4-phase plan for integrating scripts/security-audit/ into the prospecting module: security audit pipeline, report generation, live demo server, and POC site builder architecture. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7.6 KiB
Security Audit + Live Demo + POC Builder Integration
Context
The prospecting module scans websites for tech stack, performance, and contacts — but not security posture. Meanwhile, scripts/security-audit/ has a standalone audit tool (audit.py) that checks HTTPS, headers, exposed files, cookies, etc., and a demo tool (demo.py) that clones a site and lets you demonstrate XSS/cookie theft/defacement live.
Goal: Integrate both into the platform as proper module features, plus architect a future POC site builder.
Progress
- Phase 1 foundation:
ProspectSecurityAuditmodel,last_security_audit_atcolumn, relationship on Prospect - Phase 1 remaining: constants, service, migration, endpoints, frontend
- Phase 2: report generation
- Phase 3: live demo server
- Phase 4: POC builder (architecture only)
Phase 1: Security Audit in Enrichment Pipeline
Slot security audit into the existing scan pipeline (alongside http-check, tech-scan, performance, contact-scrape).
New files to create
| File | Purpose |
|---|---|
prospecting/models/security_audit.py |
DONE — ProspectSecurityAudit model (1:1 on Prospect) |
prospecting/services/security_audit_service.py |
SecurityAuditService.run_audit(db, prospect) — migrates SecurityAudit class logic from scripts/security-audit/audit.py (check_https, check_ssl, check_headers, check_exposed_files, check_cookies, check_server_info, check_technology) |
prospecting/services/security_audit_constants.py |
TRANSLATIONS, EXPOSED_PATHS, SECURITY_HEADERS, SEVERITY_SCORES, GRADE_COLORS moved from audit.py |
prospecting/schemas/security_audit.py |
Pydantic schemas: SecurityAuditResponse, SecurityAuditSingleResponse, SecurityAuditBatchResponse |
prospecting/migrations/versions/prospecting_002_security_audit.py |
Creates prospect_security_audits table, adds last_security_audit_at to prospects |
Files to modify
| File | Change |
|---|---|
prospecting/models/prospect.py |
DONE — last_security_audit_at column + security_audit relationship |
prospecting/models/__init__.py |
DONE — Export ProspectSecurityAudit |
prospecting/services/prospect_service.py |
Add get_pending_security_audit() query |
prospecting/routes/api/admin_enrichment.py |
Add /security-audit/batch + /security-audit/{prospect_id} endpoints. Add to full_enrichment pipeline. |
prospecting/tasks/scan_tasks.py |
Add batch_security_audit Celery task + add to full_enrichment task |
prospecting/static/admin/js/scan-jobs.js |
Add security_audit: 'security-audit' to batchRoutes |
prospecting/templates/.../scan-jobs.html |
Add "Security Audit" batch button |
prospecting/templates/.../prospect-detail.html |
Add "Security" tab with grade badge, score bar, findings by severity, "Run Audit" button |
prospecting/static/admin/js/prospect-detail.js |
Add runSecurityAudit(), severityColor(), gradeColor() methods, security_audit tab handling |
prospecting/schemas/prospect.py |
Add security_audit field to ProspectDetailResponse |
Key design decisions
- Findings stored as JSON (same as
tech_stack_json,lighthouse_json) — variable structure - Denormalized severity counts for dashboard queries without JSON parsing
- Reuse same
ScanBatchResponseschema for batch endpoint SECURITY_AUDITalready exists inJobTypeenum — no enum change needed
Phase 2: Security Report Generation
Generate branded bilingual HTML reports from stored audit data.
New files
| File | Purpose |
|---|---|
prospecting/services/security_report_service.py |
generate_html_report(audit, prospect, language, contact) → standalone HTML string. Migrates generate_report() from audit.py (~300 lines of HTML template). |
Files to modify
| File | Change |
|---|---|
prospecting/routes/api/admin_enrichment.py |
Add GET /security-audit/{prospect_id}/report?language=auto&download=false → HTMLResponse |
prospecting/templates/.../prospect-detail.html |
"Generate Report" button + language selector on security tab |
prospecting/static/admin/js/prospect-detail.js |
openSecurityReport(lang) method |
Phase 3: Live Demo Server
Integrate demo.py's site cloner + hacker console as a managed background process.
New files
| File | Purpose |
|---|---|
prospecting/models/demo_session.py |
DemoSession model — prospect_id, status, port, pid, target_url, target_domain, started_at, stopped_at, error_message |
prospecting/services/demo_service.py |
DemoService — start_demo (clone + spawn server), stop_demo, stop_all, get_active. Uses multiprocessing.Process to run HTTPServer. Port range 9000-9099. |
prospecting/services/demo_constants.py |
HACKER_JS, HACKER_HTML_TEMPLATE, VULNERABLE_SEARCH_BAR, SPA_SCRIPT_DOMAINS migrated from demo.py |
prospecting/schemas/demo_session.py |
DemoSessionResponse (includes computed hacker_url, site_url) |
prospecting/routes/api/admin_demo.py |
POST /demo/start/{prospect_id}, POST /demo/stop/{session_id}, GET /demo/active, POST /demo/stop-all |
prospecting/migrations/versions/prospecting_003_demo_sessions.py |
Creates prospect_demo_sessions table |
Files to modify
| File | Change |
|---|---|
prospecting/routes/api/admin.py |
Include demo router |
prospecting/config.py |
Add demo_port_range_start/end, demo_max_concurrent, demo_auto_stop_minutes |
prospecting/templates/.../prospect-detail.html |
"Launch Demo" / "Stop Demo" buttons + active demo indicator on security tab |
prospecting/static/admin/js/prospect-detail.js |
startDemo(), stopDemo(), openHackerConsole(), periodic status check |
Key design decisions
multiprocessing.Process(not subprocess) — can import SiteCloner/DemoHandler directly- Auto-stop after 60 minutes (configurable)
- On app startup: cleanup stale RUNNING sessions from previous crashes
- Max 5 concurrent demos (configurable)
Phase 4: POC Site Builder (Architecture Only — No Implementation)
Separate from security demo. Builds a better version of the client's site as proof-of-concept.
Where it fits
- Triggered FROM prospect detail page ("Build POC Site" button)
- Creates artifacts IN the hosting module (
HostedSitewith status=DRAFT → POC_READY) - Uses
HostedSite.prospect_idFK (already exists) to link back
Planned structure (future work)
hosting/services/poc_builder_service.py—create_poc(db, prospect_id, template, config)hosting/poc_templates/— directory of starter templates with manifestPOST /admin/hosting/poc/build/{prospect_id}— trigger POC generationGET /admin/hosting/poc/templates— list available templates- HostedSite additions:
poc_template,poc_source_audit_id,poc_config_jsoncolumns
Implementation Order
Phase 1 (security audit pipeline) ← IN PROGRESS
↓
Phase 2 (report generation) ← depends on Phase 1 model
↓ (can parallel with Phase 3)
Phase 3 (live demo server) ← independent, but logically follows
↓
Phase 4 (POC builder) ← architecture only, deferred
Verification
# Phase 1
python -m pytest app/modules/prospecting/tests/ -x -q --timeout=30
# Manual: Run enrichment on prospect 1, check security tab shows grade/findings
curl -X POST .../enrichment/security-audit/1 # single
curl -X POST .../enrichment/security-audit/batch # batch
# Phase 2
# Open report in browser:
curl .../enrichment/security-audit/1/report > report.html && open report.html
# Phase 3
# Start demo, open hacker console, verify attacks work, stop demo
curl -X POST .../demo/start/1 # returns port + URLs
curl -X POST .../demo/stop/{session_id}