Receive incident notifications

In Analytics → Alerts → Notification channels, create per-environment destinations for email, Slack, or PagerDuty. Select All, Threshold, or Anomaly incidents. Readers can view channels; members with write permission manage them.

Threshold rules and the anomaly detector produce incidents. The detector is opt-in and has its own warmup and algorithm. A channel receives opening and closing/expiration notifications; creating a channel does not subscribe it to previous incidents. Signed webhooks are another way to integrate data.incident.opened, closed, and expired events.

Add and preserve the destination

• Email: one ASCII address per channel, with SMTP configured by the operator.
• Slack: an incoming HTTPS webhook from hooks.slack.com or hooks.slack-gov.com.
• PagerDuty: a 32-character Events v2 integration key.

Credentials are write-only and never shown again. Leaving the destination empty while editing preserves the current one. There is no test-send button. Forms preserve drafts on failure; a conflict requires explicitly loading the current version before replacing it.

Manage through the SDK

With the server client and data:write permission:

import { randomUUID } from "node:crypto";

const channel = await orilla.data.notificationChannels.create({
  request_id: randomUUID(),
  name: "On call",
  type: "pagerduty",
  kind: "threshold",
  enabled: true,
  destination: process.env.PAGERDUTY_INTEGRATION_KEY!,
});
await orilla.data.notificationChannels.update(channel.id, {
  version: channel.version,
  name: channel.name,
  kind: "all",
  enabled: false,
});
const channels = await orilla.data.notificationChannels.list();

Save request_id before creation and reuse it with the same body if the response is lost. An identical repeat retrieves the resource; changing it produces 409, while repeating a deleted creation produces 410. Updating or deleting requires the current version; the service type is immutable. Each environment supports up to 50 active channels. Queries use data:read.

To see an incident’s audit trail:

const incident = await orilla.data.incidents.retrieve(incidentId);
console.log(incident.data.orilla.channel_notifications);

Pausing, retries, and outcomes

Pausing retains pending notifications; resuming continues them. Deleting cancels pending notifications and preserves the audit trail. A request already sent cannot be recalled. Each subscription preserves the original version and destination at opening: editing a channel does not change how an earlier incident is resolved.

The API dispatcher uses a durable queue, leases, and up to twelve attempts with exponential backoff up to one hour. It preserves opening/closing order per incident and channel. A lost ACK can duplicate email or Slack messages; PagerDuty preserves its deduplication key. The audit trail distinguishes queued, paused, canceled, retrying, failed, and unconfirmed outcomes.

SMTP acceptance, Slack’s ok, or PagerDuty success confirms provider acceptance, not reading or final delivery. Audit responses do not expose destinations or provider error text. Continuous Data export to external destinations remains outstanding.

Configuration and scope

The operator must apply the schema and configure a dedicated, stable NOTIFICATION_ENCRYPTION_KEY containing 32 bytes encoded as hexadecimal. Destinations are stored encrypted; backing up and migrating this key is an operational task. Without it, creation/replacement and dispatch are blocked, but reading, pausing, deleting, and metadata edits remain available. Email reuses the SMTP variables described in account.

The code is included in the recorded deployed version, with bounded remote reads. Sending was verified with synthetic local SMTP/HTTP; each installation’s external providers still need configuration and validation. See implementation status.