fix(alerting): ApiLatencyHigh on the 1-minute p95, ApiMemoryPressure for 1m, C-G2 at 225MB

第 3 回キャンペーンの実測を反映する。V-G1 では ILB が 1 分以内に api の接続を
遅いインスタンスから逃がすため 5 分窓の p95 が 0.5 秒を越えず、ApiLatencyHigh が
発火しなかった(1 分窓の p95 は 3 分半 2.3 秒)。式を 1 分窓にし 2 分継続に変える。
C-G2 は 200MB で working set が limit の 0.797〜0.803 を行き来して閾値 0.8 を
保てなかったため 225MB にし、ApiMemoryPressure の継続を 1 分にする。
This commit is contained in:
seon committed 2026-10-08 21:27:24 +09:00
1 parent a4f4b3a53a
commit 3f686c1aea
3 files changed
+10 -9

No files matched your search

+4 -3
View File
@@ -5,7 +5,7 @@ asset: container
expected_layer: 2
ground_truth_label: memory_pressure_api
ground_truth_action: k8s.rollout_restart
description: StressChaos memory workers 2 size 200MB (api limit 256Mi), app=api, duration 10m
description: StressChaos memory workers 2 size 225MB (api limit 256Mi, working set about 0.9), app=api, duration 10m
notes:
open_item: "design 11.3: whether the stress re-attaches to the restarted pods for the remaining duration"
inject:
@@ -26,8 +26,9 @@ inject:
workers: 2
# 2026-10-08: "80%" is a share of the node memory (memStress reached 4.4 GB), so the cgroup OOM killer took memStress
# and the api process within a second: one restart per pod, no sustained pressure, no alert. 200MB stays under the
# 256Mi limit and keeps the working set near 90% for the whole duration (ApiMemoryPressure fires after 2m).
size: "200MB"
# 256Mi limit. 200MB left the working set at 0.797-0.803 of the limit (campaign 3), flapping around the 0.8
# threshold; 225MB keeps it near 0.9 so ApiMemoryPressure holds.
size: "225MB"
duration: "10m"
revert:
method: chaos_delete