fix(alerting): ApiLatencyHigh on the 1-minute p95, ApiMemoryPressure for 1m, C-G2 at 225MB

第 3 回キャンペーンの実測を反映する。V-G1 では ILB が 1 分以内に api の接続を
遅いインスタンスから逃がすため 5 分窓の p95 が 0.5 秒を越えず、ApiLatencyHigh が
発火しなかった(1 分窓の p95 は 3 分半 2.3 秒)。式を 1 分窓にし 2 分継続に変える。
C-G2 は 200MB で working set が limit の 0.797〜0.803 を行き来して閾値 0.8 を
保てなかったため 225MB にし、ApiMemoryPressure の継続を 1 分にする。
This commit is contained in:
seon committed 2026-10-08 21:27:24 +09:00
1 parent a4f4b3a53a
commit 3f686c1aea
3 files changed
+10 -9

No files matched your search

@@ -41,13 +41,13 @@ groups:
- uid: api-latency-high
title: ApiLatencyHigh
condition: C
for: 3m
for: 2m
noDataState: OK
execErrState: Error
labels:
layer: '2'
annotations:
summary: demo api p95 latency above 500ms (grey zone)
summary: api p95 over the last minute above 500 ms for 2 minutes
data:
- refId: A
relativeTimeRange:
@@ -57,7 +57,7 @@ groups:
model:
refId: A
instant: true
expr: histogram_quantile(0.95, sum by (le) (rate(demo_http_request_duration_seconds_bucket{route=~"orders|price"}[5m]))) > bool 0.5
expr: histogram_quantile(0.95, sum by (le) (rate(demo_http_request_duration_seconds_bucket{route=~"orders|price"}[1m]))) > bool 0.5
- refId: C
datasourceUid: __expr__
model:
@@ -261,13 +261,13 @@ groups:
- uid: api-memory-pressure
title: ApiMemoryPressure
condition: C
for: 2m
for: 1m
noDataState: OK
execErrState: Error
labels:
layer: '2'
annotations:
summary: an api container holds more than 80% of its memory limit for 2 minutes (grey zone; rollout restart after approval)
summary: an api container holds more than 80% of its memory limit for 1 minute (grey zone; rollout restart after approval)
data:
- refId: A
relativeTimeRange: