Skip to content

40.6. Phase 2 Ends With a Real Architectural Finding

Continuation of 40.5. Phase 2's job was three things:

  1. Stand up a vetting harness that classifies docker-compose targets as plug_and_play, needs_setup, or rejected.
  2. Run it across Vulhub short-lists, OWASP apps, and Docker Hub tagged versions until we had ~20 ready-to-go targets.
  3. Smoke the full runtime path: random target → spin up → nmap → corpus query → tear down.

The first two were grind. The third surfaced the entry's actual story.

The smoke's first run was a clean architectural failure

Pool state going in: - 20 plug_and_play targets (Vulhub + OWASP + Docker Hub) - 13 hand-vetted entries in cve_fingerprints - 26,946 CVEs in cve_intel - 33,243 exploit recipes

Phase 2.6 smoke (gemma_forge/harness/tools/pentest/lab_smoke.py) picks N random targets from the pool, spins each up on the lab network, runs nmap from inside, queries the corpus for each discovered service+version, and reports candidates per port.

First run:

Full pipeline succeeded on:    5/5 targets
Targets returning candidates:  0/5
Total CVE candidates returned: 0

Five-for-five on the pipeline (containers up, nmap ran, queries went out, containers torn down) and zero-for-five on the actual answer. Not a wiring bug — every layer worked. The corpus lookup was just returning empty sets for every observed banner.

The cause was obvious in retrospect and embarrassing in real-time: CorpusClient.lookup_candidates() matched only against cve_fingerprints, which had 13 entries. For random pool sampling to work, fingerprints would need to cover thousands of (service, version) pairs by hand. That doesn't scale. We'd built a Ferrari with a 5-gallon fuel tank.

The fix: a tiered lookup with a vendor-fallback rank multiplier

lookup_candidates(service, version) now tries two tiers:

  1. Precise fingerprint match (existing path) — cve_fingerprints exact-match on service + version pattern. Rank multiplier = 1.0.
  2. Vendor+product fallback — wildcard vendor/product match against cve_intel directly, scoped to entries that have at least one exploit recipe (msf_modules, exploitdb_paths, or vulhub_path non-empty). Rank multiplier = 0.5.

The 0.5× multiplier is the load-bearing piece. It keeps precise fingerprint matches preferred when both tiers return results, but lets the broader corpus do useful work when fingerprints are silent. Apache 2.4.49 → CVE-2021-41773 (KEV, rank=100, precise fingerprint). Tomcat 9.0.20 → multiple Tomcat CVEs at rank=50 (vendor-fallback).

The fallback's tightness depends on a small alias map in _vendor_product_fallback() that maps nmap service names to vendor/product tokens stored in cve_intel. First version mapped "tomcat" → ["apache", "tomcat"], which over-matched by pulling in every Apache product (Struts, ActiveMQ, HTTP Server, Airflow…) via the vendor token. Sample the data, see the products are stored as (vendor="Apache", product="Tomcat"), tighten to "tomcat" → ["tomcat"], and now product-name match is the gate, not vendor-name match. Clean separation.

What the second smoke showed

[1/5] target-struts2-s2-057 (Apache Tomcat 8.5.33)
  → 4 candidate(s): CVE-2025-24813, CVE-2017-12615, CVE-2017-12617 …
[2/5] target-owasp-dvwa (Apache httpd 2.4.25)
  → 4 candidate(s): CVE-2021-41773, CVE-2021-42013 …
[3/5] target-weblogic-cve-2017-10271 (Oracle WebLogic Server)
  → 5 candidate(s): CVE-2020-14882, CVE-2017-10271, CVE-2023-21839 …
[4/5] target-owasp-webgoat (zeus-admin?)            ← nmap blind
  → no corpus candidates
[5/5] target-elasticsearch-cve-2014-3120 (wap-wsp?) ← nmap blind
  → no corpus candidates

Three observations worth recording:

The WebLogic case is the ideal match. The target's cve_ids is exactly ['CVE-2017-10271']. The corpus query — run against the live nmap banner with no advance knowledge of what the target was — returned CVE-2017-10271 in the top-3 candidates. End-to-end blind: target spun up, banner harvested, corpus queried, exact intended CVE recovered. This is the loop the Worker will run in Phase 3.

Generic "http" service names are still a problem. Nmap reports service="http" for Tomcat, Jetty, Struts containers (because they all speak HTTP). The smoke harness now tries the product token parsed from the version string FIRST ("Apache Tomcat 9.0.20" → tomcat), then the vendor token, then the raw service name. That correctly routed the Tomcat target to Tomcat CVEs instead of generic HTTP Server CVEs. But the corpus client itself still treats the input opaquely — ideally lookup_candidates() would parse a full banner string and disambiguate internally. Leaving that for Phase 3.

Nmap goes blind on some services that vet_target.py handles fine. WebGoat presents as zeus-admin? on 9090; Elasticsearch 1.4.2 presents as wap-wsp? on 9200. The vetting harness already solved this with an HTTP-probe fallback (curl the port, parse Server header and body markers). The smoke doesn't use that path yet. The Worker WILL need it in Phase 3 — same code, reused.

The pattern that's emerging

Both the Phase 2 wins and the Phase 2 gaps are about the same thing: dynamic runtime lookup over a 27k-entry corpus is not the same problem as a hand-curated playbook of 50 entries. Every system layer needs to assume incomplete fingerprints, generic service names, vendor-name collisions, and ambiguous banners. The fallback path isn't a hack — it's the mainline. Fingerprints are an optimization on top of it.

ADR-0023 §1 said "shift from static playbook to runtime DB query." Phase 2 proved the second-order consequence: when the DB is large, the query has to be tolerant. Tiered lookups with rank multipliers are the mechanism. The 0.5× number is a magic constant for now — Phase 4's cross-run measurement will tell us if it's the right one (does fallback-tier exploitation succeed less often than fingerprint-tier exploitation, by what ratio?). If we get good telemetry there, the multiplier becomes a learnable parameter.

What's not done yet

  • Nmap-blind fallback in the Worker. Reuse vet_target.http_probe_open_ports() and derive_fingerprint_from_http_probe() from inside the Worker's recon loop, not just from inside vetting.
  • Banner-aware corpus lookup. Push the product-token parsing INTO CorpusClient.lookup_candidates() so callers don't have to do it. Currently it's in lab_smoke._product_alias() — fine for smoke, wrong place for the Worker.
  • Webgoat/Shiro/Jetty fingerprints. Either backfill the alias map or solve generically via the HTTP probe path.

These all roll into Phase 3.

Where we stand at end of Phase 2

  • Intel corpus: 26,946 CVEs, 33,243 recipes, 17s refresh ✓
  • Target pool: 20 plug_and_play, 3 rejected, 1 needs_setup ✓
  • Vetting harness: vulhub + OWASP + Docker Hub, three classifications ✓
  • Lab network: isolated pentest-lab bridge, override-injection works ✓
  • End-to-end smoke: 5/5 pipeline succeeds, 3/5 return candidates, WebLogic exact-CVE cross-match recovered blind ✓
  • Cross-run learning: Phase 4 ✗

Phase 3 is the skill code: 5 sub-runtimes (WorkQueue, Architect, Worker, Reflector, Auditor), msfrpcd wrapper, banner-aware lookup, nmap-blind HTTP-probe fallback in the Worker's recon. Estimated 3 days. The hard architecture work — corpus shape, fallback strategy, pool curation — is done. Phase 3 is engineering, not design.