Skip to content

Add relay Service Bus Slack alert monitor #419

@linear

Description

@linear

Goal

Add internal Slack alerting for the relay Service Bus topic so we are notified when relay delivery is unhealthy.

Scope

  • Monitor bildrelaybus / relay-events subscriptions: bnj-dev, bnj-prod, op-dev, op-prod.
  • Alert in #eva-ops-alerts when a subscription has dead-letter messages.
  • Alert when the oldest active message is older than 30 minutes.
  • Keep the implementation simple and stateless: every hourly run may report the same still-open issue again.
  • Avoid Blob-backed alert state, dedupe storage, and recovery notices.
  • Do not include message bodies, SAS URLs, connection strings, or Slack secrets in alert payloads.

Verification

  • Unit coverage for stale-active alerts, dead-letter alerts, stateless repeated reporting, missing Slack config, and monitor failure handling.
  • Service Bus smoke test with a temporary relay-alert-smoke subscription.
  • Live settings verifier confirms exactly one monitor host is enabled.

Related

PR: https://github.com/bild-engineering/bild-ia/pull/299

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions