Files
seon 7ce2dc68fe feat: AI VM stack (WP-12) and Alloy telemetry on k8s, legacy and DB VMs (WP-15)
WP-12(실험 설계서 6장) vm/ai-env: apply.sh(시크릿 4종을 Secret Manager에서 파일로, 미등록이면
성능 저하 모드로 기동), compose(agent 8080·hitl 8081·rag-db pgvector 0.8.7-pg16), ai.env(Discord
서버 Selfheal Lab 1557424985793101824, approvals 1557425140302872698, escalation 1557425282363691048,
OPENAI_MODEL gpt-6.1-sol 후보), 프롬프트 3종, 러너북 11종(조치별·토폴로지·3층 절차).

WP-15(실험 설계서 4.1·4.3): apps/alloy.yaml(Alloy Helm 1.13.0 DaemonSet, 클러스터링으로 KSM·Argo CD·
cadvisor 스크랩 분산, demo 등 파드 로그 → Loki), apps/kube-state-metrics.yaml(Helm 8.6.0).
legacy-api compose에 Alloy 사이드카(up{job="legacy-api"}, 컨테이너 로그). DB VM은
vm/postgres-common/alloy.sh로 apt 설치 + prometheus.exporter.postgres(pg_replication_lag_seconds)
+ custom query pg_is_wal_replay_paused + PostgreSQL 로그.

vm/harness/apply.sh: 준비 대기 루프의 curl에 || true(러너 재기동 순간 curl 56으로 중단되던 문제,
2026-10-08 harness-sync 실측).
2026-10-08 01:47:38 +09:00

2.0 KiB

vm/ai-env

AI VM (10.20.0.12) of the self-healing lab: experiment design 6 (diagnosis agent, pgvector RAG store, Discord HITL bot).

file purpose
apply.sh boot / make vm-sync entry: secrets from Secret Manager to /opt/selfheal/ai/secrets, env file, compose up. Succeeds even when the OpenAI key or Discord token is not registered yet (services run degraded)
compose.yaml agent (8080), hitl (8081), rag-db (pgvector 0.8.7 / PostgreSQL 16, internal)
ai.env non-secret settings: Discord guild and channel ids, model ids, internal endpoints
image.tag image tag (selfheal-public/ai-agent:<sha> and ai-hitl:<sha> built from selfheal-lab-apps/ai)
prompts/system.md system prompt; its sha256 prefix is the prompt_version recorded with every session
prompts/topology.md, prompts/slo.md context blocks of the user message (design 6.4)
runbooks/*.md RAG documents: one per catalogue action, topology, layer-3 procedures. Re-ingested at agent start (hash based)

Agent API: POST /diagnose {"alert": {...}, "run_id", "scenario_id"} (called by the harness) returns the strict diagnosis plus disposition (proposal | escalated | none). GET /selfcheck checks key presence, rag-db, Mimir, k8s, GCP, AWS and the harness.

HITL bot: POST /post (agent -> Discord message with Approve/Reject buttons), POST /escalate (agent -> escalation channel + runbook thread), GET /selfcheck. Buttons: approve -> harness /proposals/{token}/approve then /actions/execute; reject -> /proposals/{token}/reject; no decision within the window -> escalation channel. Gateway intents: GUILDS only (design 11.7); interactions need no intent.

Secrets (Secret Manager, VM service account): selfheal-openai-api-key, selfheal-discord-bot-token (operator registers the values), selfheal-k8s-ai-token and selfheal-k8s-ca-cert (make token). Verify presence only: gcloud secrets versions list selfheal-openai-api-key --project cms-d-414903.