Incident

Goal: restore service fast, communicate, then learn. Safety first: a building’s heat does not depend on this platform — the controller keeps heating on last known good config if the cloud is down. So a platform outage is not a heating emergency; a controller/field issue is (see device-down.md).

1. Detect & declare

• Triggers: Sentry error spike, uptime probe on GET /health failing, p95 latency / error-rate alert from GET /metrics, or a customer report.
• Declare an incident; assign an incident lead. Pick severity (SEV1 = platform down / data risk, SEV2 = degraded, SEV3 = minor).

2. Assess

• GET /health → is the DB reachable? GET /metrics → error rate / latency.
• Recent deploy? Check the Deploy workflow run and the last migration.
• Logs (structured JSON) and Sentry for the failing path.

3. Mitigate

• Bad deploy → roll back (rollback.md).
• DB issue → check the managed Postgres dashboard / failover; if data loss risk, see backup-restore.md.
• Overload → scale the backend/web service replicas; the rate limiter sheds abuse.

4. Communicate

• Status updates to stakeholders at a fixed cadence for the severity. Note customer impact and ETA.

5. Recover & verify

• Confirm GET /health ok, error rate normal, a real user flow works (login → dashboard).

6. Post-incident

• Within 48h: blameless write-up — timeline, root cause, what to fix. File the action items.