Browse resources

Use knotel

Alerts

Get told when error rate, latency, traffic or matching log lines cross a line, by webhook, Slack or email.

An alert is a threshold on a number the dashboard already shows — error rate, p95, requests, or how many log lines matched — checked on a schedule and sent to a webhook, Slack or email when it crosses. There's no second query language: the filters are the same ones the Traces page puts in its URL.

Create an alert

Open a project's Alerts page and choose New alert. Each alert watches one of two things:

WatchMetricsNarrowed by
SpansError rate, errors, requests, p50, p95, p99, average latency, slowest requestAny span filter (service, route, attribute). Entry spans counts requests into a service rather than every span under them.
LogsMatching linesA substring of the body (case-folded unless you ask otherwise), a minimum level, one service, and log attribute filters.

Turn on Apply to all projects and one rule covers every project, with its own state in each: one broken project is one incident to announce and to recover, not a number averaged with nine healthy ones until it stops crossing the line.

When it fires

SettingDefaultWhat it does
Look back15 min (logs: 5)The window each check measures. 5 minutes to a day.
Must hold for0 minHow long the breach must last before anyone hears of it. The alert sits in pending until then, so one bad minute doesn't wake anyone.
Check every5 minHow often it's evaluated. The Worker's cron runs each minute and checks what's due.
Repeat at most every60 minWhile it stays broken, it re-sends on this cadence rather than every check. Without it a tripped alert sends 288 messages a day.
  • No data is not a breach. A window with nothing in it is skipped, not read as zero, so a "requests below 10" alert doesn't fire on an idle weekend. To catch a service going silent, watch a busier scope, or rely on Home's Went quiet signal.
  • A resolve is sent once, when a firing alert comes back under the line. A breach that never got past pending was never announced, so it isn't resolved either.
  • Muting stops sending, not checking. Mute for an hour across a deploy and an alert that broke and recovered inside it comes out quiet, rather than announcing a fire that's already over.

Destinations

Where alerts go is set up once, under Webhooks and Email, and shared by every project. Each has a Send test that goes through the real delivery path, and the page remembers whether the last test worked.

Slack

Create an incoming webhook in Slack, add it as a webhook destination and choose the Slack format. Messages carry the condition, the measured value, the threshold and a button back to the project.

Your own receiver

The JSON format posts:

{
  "alert": "Checkout error rate",
  "state": "fired",            // or "resolved"
  "project": "shop",
  "projectId": "p_…",
  "condition": "error rate above 2% over 15 min",
  "value": 4.7,                // null in a test send
  "threshold": 2,
  "at": "2026-09-22T03:14:00.000Z",
  "link": "https://trace.example.com/p/p_…"
}

Generate a signing secret and each request carries x-knotel-signature: sha256=<hex>, the HMAC-SHA256 of the raw body. Check it before trusting the payload:

Node
import { createHmac, timingSafeEqual } from "node:crypto";

function verified(rawBody: string, header: string | undefined, secret: string) {
  const expected = "sha256=" + createHmac("sha256", secret).update(rawBody).digest("hex");
  return !!header && header.length === expected.length &&
    timingSafeEqual(Buffer.from(header), Buffer.from(expected));
}

Extra headers, such as an Authorization your receiver wants, can be added to the destination. A request times out after 10 seconds; a 5xx or 429 is retried once, and any other error is not, since it won't fix itself.

Email

Email goes through Cloudflare's Email Service, via the SEND_EMAIL binding. It needs Workers Paid and a sender address on a domain in the same account with Email Routing turned on; without them the Email page says it's unavailable rather than failing at send time. Up to 50 recipients per destination.

Subject and body are templates with {{name}} placeholders and nothing else — no loops or conditions, so a template can't fail to render at 3am. The preview is the same function the real send uses.

PlaceholderValue
{{alert}}The alert's name
{{state}}fired or resolved
{{project}}The project's name
{{condition}}The condition, in words
{{value}}What was measured
{{threshold}}The line it crossed
{{time}}When, in UTC
{{link}}The project in knotel
Set APP_URL
Links in notifications use the APP_URL variable. Set it to your instance's address, or every link points at knotel.dev. See Variables and secrets.

What alerts cost

Checking is the cost; sending is rounding error. Alerts due in the same minute that share a scope share one query, and an all-projects alert is one query grouped by project, not one per project. Checks only read the hot window in D1: a scheduled query against R2 SQL, which bills at least 10 MB per query, would be the most expensive thing in the product. A log alert at WARN and above reads a smaller index than any level, so it's cheaper and faster.

Alerts worth starting with

AlertSettings
Checkout is failingSpans · error rate above 2% · filter http.route = /checkout · look back 10 min · hold 5 min
API got slowSpans · entry spans · p95 above 1500 ms · look back 15 min · hold 10 min
A worker is erroringLogs · level ERROR · service billing-worker · matching lines above 20 · look back 5 min
A known failure is backLogs · body contains "payment declined" · matching lines above 0
Any project breaksSpans · all projects · entry spans · error rate above 5% · hold 5 min