Start a project

Bots and alerts

The easy half is sending a message. The half that decides whether the thing is useful or muted within a fortnight is deciding when not to.

Start a project

Quoted per build.

What you get

  • A bot in Discord, Slack or Telegram that posts only when something has actually happened
  • Conditions written down and testable, rather than a threshold somebody guessed
  • Replay against historical data, so you can see what it would have said last month before you trust it
  • Quiet by default: status chatter off, so a real alert is not buried in noise
  • A watchdog, so a bot that dies tells you it died
  • Scheduled jobs that run without a machine being left switched on

Alert fatigue is the whole design problem

A monitoring bot built here sent 471 messages over its first run. Every single one was status chatter: started, still running, nothing to report. The signal it existed to send was somewhere in there and nobody would ever have found it.

Status messages are now off by default and the bot posts when a condition is met. That is a one line change and it is the difference between a tool people read and a channel people mute.

Prove it against the past before trusting it

Any alert rule can be made to look good by describing it. The way to know is to replay it over historical data and count what it would actually have said, and how often it would have been wrong.

On that same bot, replaying the real path is what found the reason it had never fired at all. A test harness with a fake data source had been passing the whole time, which is the failure mode of most monitoring: the tests check the code, not the thing.

This work, done

Tell me what is not working.

Twenty minutes, no charge, and a straight answer about what it would take and what it would cost.

Start a project