seon
3f686c1aea
第 3 回キャンペーンの実測を反映する。V-G1 では ILB が 1 分以内に api の接続を 遅いインスタンスから逃がすため 5 分窓の p95 が 0.5 秒を越えず、ApiLatencyHigh が 発火しなかった（1 分窓の p95 は 3 分半 2.3 秒）。式を 1 分窓にし 2 分継続に変える。 C-G2 は 200MB で working set が limit の 0.797〜0.803 を行き来して閾値 0.8 を 保てなかったため 225MB にし、ApiMemoryPressure の継続を 1 分にする。
vm/ai-env
AI VM (10.20.0.12) of the self-healing lab: experiment design 6 (diagnosis agent, pgvector RAG store, Discord HITL bot).
| file | purpose |
|---|---|
apply.sh |
boot / make vm-sync entry: secrets from Secret Manager to /opt/selfheal/ai/secrets, env file, compose up. Succeeds even when the OpenAI key or Discord token is not registered yet (services run degraded) |
compose.yaml |
agent (8080), hitl (8081), rag-db (pgvector 0.8.7 / PostgreSQL 16, internal) |
ai.env |
non-secret settings: Discord guild and channel ids, model ids, internal endpoints |
image.tag |
image tag (selfheal-public/ai-agent:<sha> and ai-hitl:<sha> built from selfheal-lab-apps/ai) |
prompts/system.md |
system prompt; its sha256 prefix is the prompt_version recorded with every session |
prompts/topology.md, prompts/slo.md |
context blocks of the user message (design 6.4) |
runbooks/*.md |
RAG documents: one per catalogue action, topology, layer-3 procedures. Re-ingested at agent start (hash based) |
Agent API: POST /diagnose {"alert": {...}, "run_id", "scenario_id"} (called by the harness) returns the strict diagnosis plus disposition (proposal | escalated | none). GET /selfcheck checks key presence, rag-db, Mimir, k8s, GCP, AWS and the harness.
HITL bot: POST /post (agent -> Discord message with Approve/Reject buttons), POST /escalate (agent -> escalation channel + runbook thread), GET /selfcheck. Buttons: approve -> harness /proposals/{token}/approve then /actions/execute; reject -> /proposals/{token}/reject; no decision within the window -> escalation channel. Gateway intents: GUILDS only (design 11.7); interactions need no intent.
Secrets (Secret Manager, VM service account): selfheal-openai-api-key, selfheal-discord-bot-token (operator registers the values), selfheal-k8s-ai-token and selfheal-k8s-ca-cert (make token). Verify presence only: gcloud secrets versions list selfheal-openai-api-key --project cms-d-414903.