seon
3f686c1aea
第 3 回キャンペーンの実測を反映する。V-G1 では ILB が 1 分以内に api の接続を 遅いインスタンスから逃がすため 5 分窓の p95 が 0.5 秒を越えず、ApiLatencyHigh が 発火しなかった（1 分窓の p95 は 3 分半 2.3 秒）。式を 1 分窓にし 2 分継続に変える。 C-G2 は 200MB で working set が limit の 0.797〜0.803 を行き来して閾値 0.8 を 保てなかったため 225MB にし、ApiMemoryPressure の継続を 1 分にする。
35 lines
1.3 KiB
YAML
35 lines
1.3 KiB
YAML
# design 7 C-G2: grey zone, container. StressChaos memory on api pods; latency rises, intermittent 5xx. Correct: k8s.rollout_restart after approval.
|
|
id: C-G2
|
|
path: gray
|
|
asset: container
|
|
expected_layer: 2
|
|
ground_truth_label: memory_pressure_api
|
|
ground_truth_action: k8s.rollout_restart
|
|
description: StressChaos memory workers 2 size 225MB (api limit 256Mi, working set about 0.9), app=api, duration 10m
|
|
notes:
|
|
open_item: "design 11.3: whether the stress re-attaches to the restarted pods for the remaining duration"
|
|
inject:
|
|
method: chaos
|
|
manifest: |
|
|
apiVersion: chaos-mesh.org/v1alpha1
|
|
kind: StressChaos
|
|
metadata:
|
|
namespace: chaos-mesh
|
|
spec:
|
|
mode: all
|
|
selector:
|
|
namespaces: [demo]
|
|
labelSelectors:
|
|
app: api
|
|
stressors:
|
|
memory:
|
|
workers: 2
|
|
# 2026-10-08: "80%" is a share of the node memory (memStress reached 4.4 GB), so the cgroup OOM killer took memStress
|
|
# and the api process within a second: one restart per pod, no sustained pressure, no alert. 200MB stays under the
|
|
# 256Mi limit. 200MB left the working set at 0.797-0.803 of the limit (campaign 3), flapping around the 0.8
|
|
# threshold; 225MB keeps it near 0.9 so ApiMemoryPressure holds.
|
|
size: "225MB"
|
|
duration: "10m"
|
|
revert:
|
|
method: chaos_delete
|