orion/docs/proposals/end-to-end-prospecting-to-live-site.md

End-to-End Plan: Prospecting → Live Site

Context

Full pipeline from discovering a prospect to delivering a live hosted website. Three workstreams in dependency order, with AI content enhancement deferred to a later phase.

Work Order

Workstream 1: Fix Hosting Foundation          ← FIRST (unblocks everything)
Workstream 2: Security Audit Pipeline         ← sales tool, enriches prospect data
Workstream 3: POC Builder + Templates         ← the wow factor (AI deferred)
Workstream 4: AI Content Enhancement          ← DEFERRED (provider TBD)

The Full Journey (target state)

1. DISCOVER    → Prospect created (digital scan or manual capture)
2. ENRICH      → HTTP check, tech scan, performance, contacts, security audit
3. SCORE       → Opportunity score + security grade → prioritize leads
4. REPORT      → Generate branded security report for sales meeting
5. DEMO        → Launch live hacker console to demonstrate vulnerabilities
6. BUILD POC   → Apply industry template, populate with scraped content
7. PREVIEW     → Client sees POC at acme.hostwizard.lu
8. PROPOSE     → Send proposal with security report + POC link
9. ACCEPT      → Create merchant, assign store, create subscription
10. GO LIVE    → Assign custom domain (acme.lu), DNS + SSL configured

Steps 1-3 work today. Steps 4-10 need the work below.

---

Workstream 1: Fix Hosting Foundation

Goal: Make site creation work, fix the merchant/prospect requirement, clean up the lifecycle.

Proposal: docs/proposals/hosting-site-creation-fix.md

1.1 Fix Site Creation Schema + Service

File	Change
hosting/schemas/hosted_site.py	Add `merchant_id: int
hosting/services/hosted_site_service.py	Rewrite create(): if merchant_id → use it; if prospect_id → auto-create merchant from prospect. Remove system merchant hack. Remove create_from_prospect().
hosting/routes/api/admin_sites.py	Remove POST /from-prospect/{prospect_id} endpoint. Main POST /sites handles both paths.

1.2 Fix Site Creation Template

File	Change
hosting/templates/.../site-new.html	Add merchant autocomplete dropdown (search existing merchants), prospect autocomplete dropdown (search existing prospects). Validate at least one selected. Auto-fill business_name/contacts from selection.

1.3 Simplify accept_proposal

File	Change
hosting/services/hosted_site_service.py	accept_proposal() — merchant already exists at creation time, so remove merchant creation logic. Only needs: create subscription, mark prospect CONVERTED.

1.4 Update Tests

File	Change
hosting/tests/conftest.py	Update hosted_site fixture to pass merchant_id
hosting/tests/unit/test_hosted_site_service.py	Update create calls, add validation tests, remove TestHostedSiteFromProspect

Verification

python -m pytest app/modules/hosting/tests/ -x -q
# Manual: create site from /admin/hosting/sites/new with merchant + prospect

---

Workstream 2: Security Audit Pipeline

Goal: Add security scanning to enrichment pipeline + report generation + live demo.

Proposal: docs/proposals/security-audit-demo-poc-builder.md

Phase 2A: Security Audit Service (scan + store results)

Already done: ProspectSecurityAudit model, last_security_audit_at column, relationship.

File	Action
prospecting/services/security_audit_constants.py	CREATE — Migrate TRANSLATIONS, EXPOSED_PATHS, SECURITY_HEADERS, SEVERITY_SCORES from scripts/security-audit/audit.py
prospecting/services/security_audit_service.py	CREATE — SecurityAuditService.run_audit(db, prospect) with check_https, check_ssl, check_headers, check_exposed_files, check_cookies, check_server_info, check_technology
prospecting/schemas/security_audit.py	CREATE — Response schemas
prospecting/migrations/versions/prospecting_002_security_audit.py	CREATE — Table + column migration
prospecting/services/prospect_service.py	Add get_pending_security_audit()
prospecting/routes/api/admin_enrichment.py	Add /security-audit/batch + /security-audit/{prospect_id}. Add to full_enrichment.
prospecting/static/admin/js/scan-jobs.js	Add security_audit: 'security-audit' to batchRoutes
prospecting/templates/.../scan-jobs.html	Add "Security Audit" batch button
prospecting/templates/.../prospect-detail.html	Add "Security" tab: grade badge, score bar, findings by severity, "Run Audit" button
prospecting/static/admin/js/prospect-detail.js	Add runSecurityAudit(), tab handling

Phase 2B: Security Report Generation

File	Action
prospecting/services/security_report_service.py	CREATE — generate_html_report(audit, prospect, language) → standalone HTML. Migrate report template from scripts/security-audit/audit.py
prospecting/routes/api/admin_enrichment.py	Add GET /security-audit/{prospect_id}/report → HTMLResponse
prospecting/templates/.../prospect-detail.html	"Generate Report" button + language selector on security tab

Phase 2C: Live Demo Server

File	Action
prospecting/models/demo_session.py	CREATE — DemoSession model (prospect_id, status, port, pid)
prospecting/services/demo_service.py	CREATE — start_demo (clone + spawn), stop_demo, stop_all. Migrate SiteCloner + DemoHandler from scripts/security-audit/demo.py
prospecting/services/demo_constants.py	CREATE — HACKER_JS, HACKER_HTML_TEMPLATE, etc.
prospecting/routes/api/admin_demo.py	CREATE — start/stop/list endpoints
prospecting/templates/.../prospect-detail.html	"Launch Demo" / "Stop Demo" buttons on security tab

Verification

python -m pytest app/modules/prospecting/tests/ -x -q
# Manual: run security audit on prospect 1
curl -X POST .../enrichment/security-audit/1
# Manual: generate report
curl .../enrichment/security-audit/1/report > report.html
# Manual: launch demo
curl -X POST .../demo/start/1

---

Workstream 3: POC Builder + Industry Templates

Goal: One-click POC generation from prospect data using industry templates. Result is a near-final multi-page website.

Architecture decision: docs/proposals/hosting-architecture-decision.md — reuse CMS + Store + StoreDomain.

Phase 3A: Content Scraping Enhancement

Before building POCs, we need more content from prospect sites than just contacts.

File	Action
prospecting/models/prospect.py	Add scraped_content_json column (Text) — stores extracted page content
prospecting/services/enrichment_service.py	Add scrape_content(db, prospect) method: extract meta description, H1/H2 headings, first paragraphs, service/menu items, image URLs, business hours, social links using BeautifulSoup
prospecting/services/prospect_service.py	Add get_pending_content_scrape()
prospecting/routes/api/admin_enrichment.py	Add /content-scrape/batch + /content-scrape/{prospect_id}
Add to full_enrichment pipeline	After contact scrape, before scoring
Migration	Add last_content_scrape_at, scraped_content_json to prospects

Scraped content structure:

{
  "meta_description": "Bati Rénovation — Construction et rénovation à Strasbourg",
  "headings": ["Bati Rénovation", "Nos Services", "Nos Réalisations"],
  "paragraphs": ["Entreprise spécialisée dans la rénovation...", ...],
  "services": ["Rénovation intérieure", "Construction neuve", ...],
  "images": ["https://site.fr/hero.jpg", "https://site.fr/project1.jpg"],
  "social_links": {"facebook": "...", "instagram": "..."},
  "business_hours": "Lun-Ven 8h-18h",
  "languages_detected": ["fr"]
}

Phase 3B: Template Infrastructure

File	Action
hosting/templates_library/manifest.json	CREATE — Registry of all templates
hosting/templates_library/generic/	CREATE — Generic template (meta.json, pages/*.json, theme.json)
hosting/templates_library/restaurant/	CREATE — Restaurant template
hosting/templates_library/construction/	CREATE — Construction template
hosting/templates_library/auto-parts/	CREATE — Auto parts template
hosting/templates_library/professional-services/	CREATE — Professional services template
hosting/services/template_service.py	CREATE — list_templates(), get_template(id), validate_template()
hosting/schemas/template.py	CREATE — TemplateListResponse, TemplateDetailResponse
hosting/routes/api/admin_sites.py	Add GET /templates endpoint

Phase 3C: POC Builder Service

File	Action
hosting/services/poc_builder_service.py	CREATE — build_poc(db, prospect_id, template_id, merchant_id): 1) Load template, 2) Load scraped content from prospect, 3) Create HostedSite + Store, 4) Populate ContentPages from template with prospect data replacing placeholders, 5) Apply StoreTheme from template, 6) Return HostedSite
hosting/routes/api/admin_sites.py	Add POST /poc/build endpoint
hosting/templates/.../site-new.html	Add template selector (cards with previews) to creation flow

Placeholder replacement:

{{business_name}}  → prospect.business_name or scraped H1
{{city}}           → prospect.city or extracted from address
{{phone}}          → primary phone contact
{{email}}          → primary email contact
{{address}}        → address contact value
{{meta_description}} → scraped meta description
{{services}}       → scraped service items (for service grid sections)
{{hero_image}}     → first scraped image or template default

Phase 3D: CMS Section Types for Hosting

The existing CMS sections (hero, features, pricing, cta) cover basics. Hosting templates need a few more:

Section Type	Used By	Description
services_grid	construction, professional	Service cards with icons and descriptions
gallery	construction, restaurant	Image grid with lightbox
team_grid	professional	Team member cards (photo, name, role)
menu_display	restaurant	Menu categories with items and prices
testimonials	all	Customer review cards
contact_form	all	Configurable contact form (name, email, message)
map_embed	all	Google Maps embed from address
hours_display	restaurant, auto-parts	Business hours table

File	Action
cms/schemas/homepage_sections.py	Add new section type schemas
app/templates/storefront/sections/	CREATE — Jinja2 partials for each new section type
app/templates/storefront/content-page.html	Update section renderer to handle new types

Verification

python -m pytest app/modules/hosting/tests/ -x -q
python -m pytest app/modules/prospecting/tests/ -x -q
# Manual:
# 1. Create prospect for batirenovation-strasbourg.fr
# 2. Run full enrichment (includes content scrape)
# 3. Go to /admin/hosting/sites/new → select prospect + "construction" template
# 4. POC built → preview at subdomain
# 5. Accept → go live with custom domain

---

Workstream 4: AI Content Enhancement (DEFERRED)

Provider: TBD (Claude API or OpenAI) Scope: TBD (full rewrite vs enhance + fill gaps)

When ready, this adds an AI step to the POC builder:

1. After scraping content and before populating CMS pages
2. AI enhances scraped text: better headlines, professional descriptions, SEO meta
3. AI generates missing content: testimonials, about section, service descriptions
4. AI translates to additional languages (fr/de/en/lb)
5. Integrated as optional step — POC builder works without AI (just uses scraped text + template defaults)

File	Action (future)
app/core/config.py or hosting/config.py	Add AI provider config (API key, model, temperature)
hosting/services/ai_content_service.py	CREATE — enhance_content(scraped, template, language), generate_missing(template_sections, scraped), translate(content, target_langs)
hosting/services/poc_builder_service.py	Call AI service between scrape and populate steps
.env.example	Add AI_PROVIDER, AI_API_KEY, AI_MODEL

---

Dependency Graph

Workstream 1 (hosting fix)
    ├── 1.1 Schema + service     ← no dependencies
    ├── 1.2 Template form        ← needs 1.1
    ├── 1.3 Simplify accept      ← needs 1.1
    └── 1.4 Tests                ← needs 1.1-1.3

Workstream 2 (security audit)
    ├── 2A Service + endpoints   ← no dependencies (model done)
    ├── 2B Report generation     ← needs 2A
    └── 2C Live demo             ← needs 2A conceptually

Workstream 3 (POC builder)
    ├── 3A Content scraping      ← no dependencies
    ├── 3B Template files        ← no dependencies
    ├── 3C POC builder service   ← needs 1.1 + 3A + 3B
    └── 3D CMS section types     ← needs 3B (to know what sections templates use)

Workstream 4 (AI) ← DEFERRED
    └── needs 3C

Parallelizable: 1.1 + 2A + 3A + 3B can all start simultaneously.

---

Estimated Scope

Workstream	New Files	Modified Files	~Lines
1 (hosting fix)	0	5	~200
2A (security service)	4	6	~1200
2B (report)	1	2	~400
2C (demo)	4	3	~500
3A (content scrape)	0	4	~200
3B (templates)	~20 JSON	2	~500
3C (POC builder)	2	3	~300
3D (CMS sections)	~8	2	~400
Total	~40	~27	~3700