seon
af37d19e2a
Experiment design 5 and 7 for the harness VM (10.20.0.11): vm/harness/apply.sh pulls the secrets from Secret Manager, compose runs runner:de076a7, measure-db, blackbox, Alloy (probe_success -> Mimir) and k6 under the campaign profile. Scenario files C-K1..X-M2 are loaded by the runner. Harness ClusterRole gains replicasets get/list for rollout undo.
28 lines
1.8 KiB
Markdown
28 lines
1.8 KiB
Markdown
# vm/harness
|
|
|
|
Harness VM (10.20.0.11) of the self-healing lab: experiment design 5 (runner, measure DB, blackbox, k6) plus Alloy.
|
|
|
|
| file | purpose |
|
|
|---|---|
|
|
| `apply.sh` | boot / `make vm-sync` entry: secrets from Secret Manager to `/opt/selfheal/harness/secrets`, env file, compose up |
|
|
| `compose.yaml` | runner (8080), measure-db, blackbox (9115), alloy (12345), k6 (profile `campaign`) |
|
|
| `image.tag` | runner image tag (`selfheal-public/harness:<sha>` built from `selfheal-lab-apps/harness`) |
|
|
| `scenarios/*.yaml` | the 12 scenarios of design 7, loaded by the runner at start |
|
|
| `config.alloy` | blackbox + runner scrape -> Mimir remote_write |
|
|
| `blackbox.yaml` | `http_2xx` (ALB `/api/orders`, label `target=alb`) and `tcp_connect` (`target=db-primary`) |
|
|
| `k6/load.js` | background load 20 rps (design 5.5) |
|
|
|
|
Runner API (design 5.2):
|
|
|
|
| endpoint | use |
|
|
|---|---|
|
|
| `GET /selfcheck` | measure DB, k8s (chaos-mesh pods), AWS OIDC caller identity, GCP instances, Mimir probe, ALB |
|
|
| `POST /runs` `{"campaign","scenario_ids":[...],"baseline_s","cooldown_s","observe_timeout_s","min_observe_s","drift_before"}` | start a run (one at a time) |
|
|
| `GET /runs`, `GET /runs/{id}`, `POST /runs/{id}/abort`, `POST /runs/{id}/finish {"drift_after"}` | inspect / stop / close |
|
|
| `POST /hooks/alert` | Grafana contact point |
|
|
| `POST /proposals` | AI VM registers a layer-2 proposal; returns the approval token |
|
|
| `POST /actions/execute {"action_id","target","token"}` | catalogue action with a human token or an approved proposal token |
|
|
| `GET /actions/catalog`, `GET /scenarios`, `GET /metrics` | reference |
|
|
|
|
Campaign load: `docker compose --env-file /opt/selfheal/harness.env --profile campaign up -d k6` (stop: `... stop k6`; exit code and `/logs/summary.json` in the `k6-logs` volume).
|