seon
1e0e17b33c
DbReplicationLag の式は PromQL の or が左辺のサンプル（常に 0）を残すため pg_is_wal_replay_paused の分岐が評価されず、第 1・2 回とも発火しなかった。2 つの bool を加算して clamp する式に直す。api コンテナのメモリが limit の 80% を超えて 2 分続いたときの ApiMemoryPressure（第 2 層）を追加する。C-G2 の StressChaos は size "80%"（ノード基準）で即 OOM になっていたため 200MB に、V-G1 の対象は トラフィックを受けているインスタンス legacy:busy に変更する。slo.md に新ルールを追記。
vm/ai-env
AI VM (10.20.0.12) of the self-healing lab: experiment design 6 (diagnosis agent, pgvector RAG store, Discord HITL bot).
| file | purpose |
|---|---|
apply.sh |
boot / make vm-sync entry: secrets from Secret Manager to /opt/selfheal/ai/secrets, env file, compose up. Succeeds even when the OpenAI key or Discord token is not registered yet (services run degraded) |
compose.yaml |
agent (8080), hitl (8081), rag-db (pgvector 0.8.7 / PostgreSQL 16, internal) |
ai.env |
non-secret settings: Discord guild and channel ids, model ids, internal endpoints |
image.tag |
image tag (selfheal-public/ai-agent:<sha> and ai-hitl:<sha> built from selfheal-lab-apps/ai) |
prompts/system.md |
system prompt; its sha256 prefix is the prompt_version recorded with every session |
prompts/topology.md, prompts/slo.md |
context blocks of the user message (design 6.4) |
runbooks/*.md |
RAG documents: one per catalogue action, topology, layer-3 procedures. Re-ingested at agent start (hash based) |
Agent API: POST /diagnose {"alert": {...}, "run_id", "scenario_id"} (called by the harness) returns the strict diagnosis plus disposition (proposal | escalated | none). GET /selfcheck checks key presence, rag-db, Mimir, k8s, GCP, AWS and the harness.
HITL bot: POST /post (agent -> Discord message with Approve/Reject buttons), POST /escalate (agent -> escalation channel + runbook thread), GET /selfcheck. Buttons: approve -> harness /proposals/{token}/approve then /actions/execute; reject -> /proposals/{token}/reject; no decision within the window -> escalation channel. Gateway intents: GUILDS only (design 11.7); interactions need no intent.
Secrets (Secret Manager, VM service account): selfheal-openai-api-key, selfheal-discord-bot-token (operator registers the values), selfheal-k8s-ai-token and selfheal-k8s-ca-cert (make token). Verify presence only: gcloud secrets versions list selfheal-openai-api-key --project cms-d-414903.