sboulahtit/orion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add end-to-end plan from prospecting to live site

Browse Source 
Master plan covering 4 workstreams:
1. Fix hosting foundation (merchant/prospect required)
2. Security audit pipeline + report + live demo
3. POC builder with industry templates (restaurant, construction,
   auto-parts, professional-services, generic)
4. AI content enhancement (deferred, provider TBD)

Target: 10-step journey from prospect discovery to live website.
Steps 1-3 work today, steps 4-10 need the work described.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Samir Boulahtit
2026-03-31 23:05:21 +02:00
parent 91963f3b87
commit 2bc03ed97c
1 changed files with 293 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

293
docs/proposals/end-to-end-prospecting-to-live-site.md Normal file
View File

@@ -0,0 +1,293 @@
# End-to-End Plan: Prospecting → Live Site
## Context
Full pipeline from discovering a prospect to delivering a live hosted website. Three workstreams in dependency order, with AI content enhancement deferred to a later phase.
## Work Order
```
Workstream 1: Fix Hosting Foundation ← FIRST (unblocks everything)
Workstream 2: Security Audit Pipeline ← sales tool, enriches prospect data
Workstream 3: POC Builder + Templates ← the wow factor (AI deferred)
Workstream 4: AI Content Enhancement ← DEFERRED (provider TBD)
```
## The Full Journey (target state)
```
1. DISCOVER → Prospect created (digital scan or manual capture)
2. ENRICH → HTTP check, tech scan, performance, contacts, security audit
3. SCORE → Opportunity score + security grade → prioritize leads
4. REPORT → Generate branded security report for sales meeting
5. DEMO → Launch live hacker console to demonstrate vulnerabilities
6. BUILD POC → Apply industry template, populate with scraped content
7. PREVIEW → Client sees POC at acme.hostwizard.lu
8. PROPOSE → Send proposal with security report + POC link
9. ACCEPT → Create merchant, assign store, create subscription
10. GO LIVE → Assign custom domain (acme.lu), DNS + SSL configured
```
Steps 1-3 work today. Steps 4-10 need the work below.
---
## Workstream 1: Fix Hosting Foundation
**Goal:** Make site creation work, fix the merchant/prospect requirement, clean up the lifecycle.
**Proposal:** `docs/proposals/hosting-site-creation-fix.md`
### 1.1 Fix Site Creation Schema + Service
| File | Change |
|---|---|
| `hosting/schemas/hosted_site.py` | Add `merchant_id: int | None`, `prospect_id: int | None`, `model_validator` requiring at least one |
| `hosting/services/hosted_site_service.py` | Rewrite `create()`: if merchant_id → use it; if prospect_id → auto-create merchant from prospect. Remove system merchant hack. Remove `create_from_prospect()`. |
| `hosting/routes/api/admin_sites.py` | Remove `POST /from-prospect/{prospect_id}` endpoint. Main `POST /sites` handles both paths. |
### 1.2 Fix Site Creation Template
| File | Change |
|---|---|
| `hosting/templates/.../site-new.html` | Add merchant autocomplete dropdown (search existing merchants), prospect autocomplete dropdown (search existing prospects). Validate at least one selected. Auto-fill business_name/contacts from selection. |
### 1.3 Simplify accept_proposal
| File | Change |
|---|---|
| `hosting/services/hosted_site_service.py` | `accept_proposal()` — merchant already exists at creation time, so remove merchant creation logic. Only needs: create subscription, mark prospect CONVERTED. |
### 1.4 Update Tests
| File | Change |
|---|---|
| `hosting/tests/conftest.py` | Update `hosted_site` fixture to pass `merchant_id` |
| `hosting/tests/unit/test_hosted_site_service.py` | Update create calls, add validation tests, remove `TestHostedSiteFromProspect` |
### Verification
```bash
python -m pytest app/modules/hosting/tests/ -x -q
# Manual: create site from /admin/hosting/sites/new with merchant + prospect
```
---
## Workstream 2: Security Audit Pipeline
**Goal:** Add security scanning to enrichment pipeline + report generation + live demo.
**Proposal:** `docs/proposals/security-audit-demo-poc-builder.md`
### Phase 2A: Security Audit Service (scan + store results)
**Already done:** `ProspectSecurityAudit` model, `last_security_audit_at` column, relationship.
| File | Action |
|---|---|
| `prospecting/services/security_audit_constants.py` | **CREATE** — Migrate TRANSLATIONS, EXPOSED_PATHS, SECURITY_HEADERS, SEVERITY_SCORES from `scripts/security-audit/audit.py` |
| `prospecting/services/security_audit_service.py` | **CREATE** — `SecurityAuditService.run_audit(db, prospect)` with check_https, check_ssl, check_headers, check_exposed_files, check_cookies, check_server_info, check_technology |
| `prospecting/schemas/security_audit.py` | **CREATE** — Response schemas |
| `prospecting/migrations/versions/prospecting_002_security_audit.py` | **CREATE** — Table + column migration |
| `prospecting/services/prospect_service.py` | Add `get_pending_security_audit()` |
| `prospecting/routes/api/admin_enrichment.py` | Add `/security-audit/batch` + `/security-audit/{prospect_id}`. Add to `full_enrichment`. |
| `prospecting/static/admin/js/scan-jobs.js` | Add `security_audit: 'security-audit'` to batchRoutes |
| `prospecting/templates/.../scan-jobs.html` | Add "Security Audit" batch button |
| `prospecting/templates/.../prospect-detail.html` | Add "Security" tab: grade badge, score bar, findings by severity, "Run Audit" button |
| `prospecting/static/admin/js/prospect-detail.js` | Add `runSecurityAudit()`, tab handling |
### Phase 2B: Security Report Generation
| File | Action |
|---|---|
| `prospecting/services/security_report_service.py` | **CREATE** — `generate_html_report(audit, prospect, language)` → standalone HTML. Migrate report template from `scripts/security-audit/audit.py` |
| `prospecting/routes/api/admin_enrichment.py` | Add `GET /security-audit/{prospect_id}/report` → HTMLResponse |
| `prospecting/templates/.../prospect-detail.html` | "Generate Report" button + language selector on security tab |
### Phase 2C: Live Demo Server
| File | Action |
|---|---|
| `prospecting/models/demo_session.py` | **CREATE** — DemoSession model (prospect_id, status, port, pid) |
| `prospecting/services/demo_service.py` | **CREATE** — start_demo (clone + spawn), stop_demo, stop_all. Migrate SiteCloner + DemoHandler from `scripts/security-audit/demo.py` |
| `prospecting/services/demo_constants.py` | **CREATE** — HACKER_JS, HACKER_HTML_TEMPLATE, etc. |
| `prospecting/routes/api/admin_demo.py` | **CREATE** — start/stop/list endpoints |
| `prospecting/templates/.../prospect-detail.html` | "Launch Demo" / "Stop Demo" buttons on security tab |
### Verification
```bash
python -m pytest app/modules/prospecting/tests/ -x -q
# Manual: run security audit on prospect 1
curl -X POST .../enrichment/security-audit/1
# Manual: generate report
curl .../enrichment/security-audit/1/report > report.html
# Manual: launch demo
curl -X POST .../demo/start/1
```
---
## Workstream 3: POC Builder + Industry Templates
**Goal:** One-click POC generation from prospect data using industry templates. Result is a near-final multi-page website.
**Architecture decision:** `docs/proposals/hosting-architecture-decision.md` — reuse CMS + Store + StoreDomain.
### Phase 3A: Content Scraping Enhancement
Before building POCs, we need more content from prospect sites than just contacts.
| File | Action |
|---|---|
| `prospecting/models/prospect.py` | Add `scraped_content_json` column (Text) — stores extracted page content |
| `prospecting/services/enrichment_service.py` | Add `scrape_content(db, prospect)` method: extract meta description, H1/H2 headings, first paragraphs, service/menu items, image URLs, business hours, social links using BeautifulSoup |
| `prospecting/services/prospect_service.py` | Add `get_pending_content_scrape()` |
| `prospecting/routes/api/admin_enrichment.py` | Add `/content-scrape/batch` + `/content-scrape/{prospect_id}` |
| Add to `full_enrichment` pipeline | After contact scrape, before scoring |
| Migration | Add `last_content_scrape_at`, `scraped_content_json` to prospects |
**Scraped content structure:**
```json
{
"meta_description": "Bati Rénovation — Construction et rénovation à Strasbourg",
"headings": ["Bati Rénovation", "Nos Services", "Nos Réalisations"],
"paragraphs": ["Entreprise spécialisée dans la rénovation...", ...],
"services": ["Rénovation intérieure", "Construction neuve", ...],
"images": ["https://site.fr/hero.jpg", "https://site.fr/project1.jpg"],
"social_links": {"facebook": "...", "instagram": "..."},
"business_hours": "Lun-Ven 8h-18h",
"languages_detected": ["fr"]
}
```
### Phase 3B: Template Infrastructure
| File | Action |
|---|---|
| `hosting/templates_library/manifest.json` | **CREATE** — Registry of all templates |
| `hosting/templates_library/generic/` | **CREATE** — Generic template (meta.json, pages/*.json, theme.json) |
| `hosting/templates_library/restaurant/` | **CREATE** — Restaurant template |
| `hosting/templates_library/construction/` | **CREATE** — Construction template |
| `hosting/templates_library/auto-parts/` | **CREATE** — Auto parts template |
| `hosting/templates_library/professional-services/` | **CREATE** — Professional services template |
| `hosting/services/template_service.py` | **CREATE** — `list_templates()`, `get_template(id)`, `validate_template()` |
| `hosting/schemas/template.py` | **CREATE** — `TemplateListResponse`, `TemplateDetailResponse` |
| `hosting/routes/api/admin_sites.py` | Add `GET /templates` endpoint |
### Phase 3C: POC Builder Service
| File | Action |
|---|---|
| `hosting/services/poc_builder_service.py` | **CREATE** — `build_poc(db, prospect_id, template_id, merchant_id)`: 1) Load template, 2) Load scraped content from prospect, 3) Create HostedSite + Store, 4) Populate ContentPages from template with prospect data replacing placeholders, 5) Apply StoreTheme from template, 6) Return HostedSite |
| `hosting/routes/api/admin_sites.py` | Add `POST /poc/build` endpoint |
| `hosting/templates/.../site-new.html` | Add template selector (cards with previews) to creation flow |
**Placeholder replacement:**
```
{{business_name}} → prospect.business_name or scraped H1
{{city}} → prospect.city or extracted from address
{{phone}} → primary phone contact
{{email}} → primary email contact
{{address}} → address contact value
{{meta_description}} → scraped meta description
{{services}} → scraped service items (for service grid sections)
{{hero_image}} → first scraped image or template default
```
### Phase 3D: CMS Section Types for Hosting
The existing CMS sections (hero, features, pricing, cta) cover basics. Hosting templates need a few more:
| Section Type | Used By | Description |
|---|---|---|
| `services_grid` | construction, professional | Service cards with icons and descriptions |
| `gallery` | construction, restaurant | Image grid with lightbox |
| `team_grid` | professional | Team member cards (photo, name, role) |
| `menu_display` | restaurant | Menu categories with items and prices |
| `testimonials` | all | Customer review cards |
| `contact_form` | all | Configurable contact form (name, email, message) |
| `map_embed` | all | Google Maps embed from address |
| `hours_display` | restaurant, auto-parts | Business hours table |
| File | Action |
|---|---|
| `cms/schemas/homepage_sections.py` | Add new section type schemas |
| `app/templates/storefront/sections/` | **CREATE** — Jinja2 partials for each new section type |
| `app/templates/storefront/content-page.html` | Update section renderer to handle new types |
### Verification
```bash
python -m pytest app/modules/hosting/tests/ -x -q
python -m pytest app/modules/prospecting/tests/ -x -q
# Manual:
# 1. Create prospect for batirenovation-strasbourg.fr
# 2. Run full enrichment (includes content scrape)
# 3. Go to /admin/hosting/sites/new → select prospect + "construction" template
# 4. POC built → preview at subdomain
# 5. Accept → go live with custom domain
```
---
## Workstream 4: AI Content Enhancement (DEFERRED)
**Provider:** TBD (Claude API or OpenAI)
**Scope:** TBD (full rewrite vs enhance + fill gaps)
When ready, this adds an AI step to the POC builder:
1. After scraping content and before populating CMS pages
2. AI enhances scraped text: better headlines, professional descriptions, SEO meta
3. AI generates missing content: testimonials, about section, service descriptions
4. AI translates to additional languages (fr/de/en/lb)
5. Integrated as optional step — POC builder works without AI (just uses scraped text + template defaults)
| File | Action (future) |
|---|---|
| `app/core/config.py` or `hosting/config.py` | Add AI provider config (API key, model, temperature) |
| `hosting/services/ai_content_service.py` | **CREATE** — `enhance_content(scraped, template, language)`, `generate_missing(template_sections, scraped)`, `translate(content, target_langs)` |
| `hosting/services/poc_builder_service.py` | Call AI service between scrape and populate steps |
| `.env.example` | Add `AI_PROVIDER`, `AI_API_KEY`, `AI_MODEL` |
---
## Dependency Graph
```
Workstream 1 (hosting fix)
├── 1.1 Schema + service ← no dependencies
├── 1.2 Template form ← needs 1.1
├── 1.3 Simplify accept ← needs 1.1
└── 1.4 Tests ← needs 1.1-1.3
Workstream 2 (security audit)
├── 2A Service + endpoints ← no dependencies (model done)
├── 2B Report generation ← needs 2A
└── 2C Live demo ← needs 2A conceptually
Workstream 3 (POC builder)
├── 3A Content scraping ← no dependencies
├── 3B Template files ← no dependencies
├── 3C POC builder service ← needs 1.1 + 3A + 3B
└── 3D CMS section types ← needs 3B (to know what sections templates use)
Workstream 4 (AI) ← DEFERRED
└── needs 3C
```
**Parallelizable:** 1.1 + 2A + 3A + 3B can all start simultaneously.
---
## Estimated Scope
| Workstream | New Files | Modified Files | ~Lines |
|---|---|---|---|
| 1 (hosting fix) | 0 | 5 | ~200 |
| 2A (security service) | 4 | 6 | ~1200 |
| 2B (report) | 1 | 2 | ~400 |
| 2C (demo) | 4 | 3 | ~500 |
| 3A (content scrape) | 0 | 4 | ~200 |
| 3B (templates) | ~20 JSON | 2 | ~500 |
| 3C (POC builder) | 2 | 3 | ~300 |
| 3D (CMS sections) | ~8 | 2 | ~400 |
| **Total** | **~40** | **~27** | **~3700** |