Skip to main content

Observability Webhooks

Webhooks enable your applications to receive real-time notifications when events occur in your monitoring and observability systems. This eliminates the need for polling and ensures your systems stay synchronized with alerts, metrics, and infrastructure changes across all integrated platforms.

Unizo normalizes webhook events from Datadog, New Relic, Splunk, Prometheus, Grafana, and other observability providers into a consistent format. This means you write your webhook handler once and it works with all supported platforms.

Webhook Configuration

To set up webhooks for your integration, visit the Unizo Console Webhooks section for step-by-step configuration guide.

Supported Event Types

These are the event types currently supported by Unizo's Observability webhooks. The list keeps growing as we add support for more events across different platforms.

Event TypeDescriptionTrigger Conditions
alert:triggeredAn alert has been triggeredThreshold breach, anomaly detection, or custom conditions
alert:resolvedAn alert has been resolvedMetrics return to normal or manual resolution
alert:acknowledgedAn alert has been acknowledgedUser acknowledges the alert
monitor:createdA new monitor has been createdMonitor creation via UI or API
monitor:updatedMonitor configuration has been modifiedThreshold, condition, or notification changes
monitor:deletedA monitor has been deletedMonitor removal from the system
dashboard:createdA new dashboard has been createdDashboard creation via UI or API
dashboard:updatedDashboard has been modifiedWidget additions, layout changes, or filter updates
metric:anomaly_detectedAnomaly detected in metricsML-based anomaly detection or statistical deviation

Webhook Security

All webhooks from Unizo include security headers to verify authenticity:

Headers

HeaderDescription
x-unizo-event-typeThe type of event that triggered the webhook
x-unizo-signatureHMAC SHA-256 signature for request validation
x-unizo-timestampUnix timestamp when the event was sent
x-unizo-delivery-idUnique identifier for this webhook delivery

Signature Verification

Verify the authenticity of incoming webhooks using HMAC SHA-256:

const crypto = require('crypto');

function verifyWebhookSignature(payload, signature, secret) {
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');
  
  return crypto.timingSafeEqual(
    Buffer.from(signature, 'hex'),
    Buffer.from(expectedSignature, 'hex')
  );
}

Event Details

Alert Events

Event TypeDescriptionTrigger Conditions
alert:triggeredAn alert has been triggeredThreshold breach, anomaly detection, or custom conditions
alert:resolvedAn alert has been resolvedMetrics return to normal or manual resolution
alert:acknowledgedAn alert has been acknowledgedUser acknowledges the alert

Alert Triggered

alert:triggered

Triggered when an alert condition is met in the monitoring system
POSThttps://api.yourapp.com/webhooks/unizo/observability
Best Practice: Use a dedicated webhook endpoint that can handle multiple event types. You have two architectural options:
• Single endpoint: https://api.yourapp.com/webhooks/unizo - Route all events to one handler
• Category-based endpoints: https://api.yourapp.com/webhooks/unizo/observability - Route by category (observability, ticketing, etc.) for microservices architecture
Headers
NameTypeRequiredDescription
Content-TypestringYesAlways application/json
x-unizo-event-typestringYesEvent type: alert:triggered
x-unizo-webhook-idstringYesUnique webhook configuration ID
x-unizo-delivery-idstringYesUnique delivery ID for idempotency
x-unizo-signaturestringYesHMAC SHA-256 signature
Request Body Schema
PropertyTypeRequiredDescription
typestringYesEvent type identifier
versionstringYesWebhook payload version
alert.idstringYesUnique alert identifier
alert.namestringYesAlert name
alert.descriptionstringNoAlert description
alert.severitystringYesSeverity level: critical, high, medium, low
alert.statusstringYesCurrent status: triggered, acknowledged, resolved
alert.metricobjectYesMetric that triggered the alert
alert.thresholdobjectYesThreshold configuration
alert.triggeredDateTimestringYesISO 8601 timestamp
alert.tagsarrayNoAssociated tags
integrationobjectYesIntegration details
Example Payload
{
"type": "alert:triggered",
"version": "1.0.0",
"alert": {
"id": "alert-123456",
"name": "High CPU Usage - Production Server",
"description": "CPU usage exceeded 90% for more than 5 minutes",
"severity": "critical",
"status": "triggered",
"metric": {
"name": "system.cpu.usage",
"value": 92.5,
"unit": "percentage",
"host": "prod-web-01"
},
"threshold": {
"operator": "greater_than",
"value": 90,
"duration": "5m"
},
"triggeredDateTime": "2024-01-15T14:00:00Z",
"tags": [
"production",
"web-server",
"critical"
]
},
"integration": {
"type": "OBSERVABILITY",
"id": "int_123456",
"name": "Datadog Production",
"provider": "datadog"
}
}
Response
200 OKWebhook processed successfully
400 Bad RequestInvalid webhook payload
401 UnauthorizedInvalid or missing signature

Alert Resolved

alert:resolved

Triggered when an alert condition is no longer met
POSThttps://api.yourapp.com/webhooks/unizo/scm
Best Practice: Use a dedicated webhook endpoint that can handle multiple event types. You have two architectural options:
• Single endpoint: https://api.yourapp.com/webhooks/unizo - Route all events to one handler
• Category-based endpoints: https://api.yourapp.com/webhooks/unizo/scm - Route by category (scm, ticketing, etc.) for microservices architecture
Headers
NameTypeRequiredDescription
Content-TypestringYesAlways application/json
x-unizo-event-typestringYesEvent type: alert:resolved
x-unizo-webhook-idstringYesUnique webhook configuration ID
x-unizo-delivery-idstringYesUnique delivery ID for idempotency
x-unizo-signaturestringYesHMAC SHA-256 signature
Request Body Schema
PropertyTypeRequiredDescription
typestringYesEvent type identifier
versionstringYesWebhook payload version
alert.idstringYesUnique alert identifier
alert.namestringYesAlert name
alert.severitystringYesSeverity level
alert.statusstringYesCurrent status: resolved
alert.durationstringYesHow long the alert was active
alert.resolvedDateTimestringYesISO 8601 timestamp
alert.resolvedBystringYesResolution method: auto, manual
integrationobjectYesIntegration details
Example Payload
{
"type": "alert:resolved",
"version": "1.0.0",
"alert": {
"id": "alert-123456",
"name": "High CPU Usage - Production Server",
"severity": "critical",
"status": "resolved",
"duration": "15m30s",
"resolvedDateTime": "2024-01-15T14:15:30Z",
"resolvedBy": "auto",
"metric": {
"name": "system.cpu.usage",
"value": 75.2,
"unit": "percentage",
"host": "prod-web-01"
}
},
"integration": {
"type": "OBSERVABILITY",
"id": "int_123456",
"name": "Datadog Production",
"provider": "datadog"
}
}
Response
200 OKWebhook processed successfully
400 Bad RequestInvalid webhook payload
401 UnauthorizedInvalid or missing signature

Alert Acknowledged

alert:acknowledged

Triggered when a user acknowledges an alert
POSThttps://api.yourapp.com/webhooks/unizo/scm
Best Practice: Use a dedicated webhook endpoint that can handle multiple event types. You have two architectural options:
• Single endpoint: https://api.yourapp.com/webhooks/unizo - Route all events to one handler
• Category-based endpoints: https://api.yourapp.com/webhooks/unizo/scm - Route by category (scm, ticketing, etc.) for microservices architecture
Headers
NameTypeRequiredDescription
Content-TypestringYesAlways application/json
x-unizo-event-typestringYesEvent type: alert:acknowledged
x-unizo-webhook-idstringYesUnique webhook configuration ID
x-unizo-delivery-idstringYesUnique delivery ID for idempotency
x-unizo-signaturestringYesHMAC SHA-256 signature
Request Body Schema
PropertyTypeRequiredDescription
typestringYesEvent type identifier
versionstringYesWebhook payload version
alert.idstringYesUnique alert identifier
alert.namestringYesAlert name
alert.statusstringYesCurrent status: acknowledged
acknowledgment.notestringNoAcknowledgment note
acknowledgment.acknowledgedByobjectYesUser who acknowledged
acknowledgment.acknowledgedDateTimestringYesISO 8601 timestamp
integrationobjectYesIntegration details
Example Payload
{
"type": "alert:acknowledged",
"version": "1.0.0",
"alert": {
"id": "alert-123456",
"name": "High CPU Usage - Production Server",
"status": "acknowledged"
},
"acknowledgment": {
"note": "Investigating the issue, likely due to traffic spike",
"acknowledgedBy": {
"id": "user-789",
"email": "ops@example.com",
"name": "John Ops"
},
"acknowledgedDateTime": "2024-01-15T14:05:00Z"
},
"integration": {
"type": "OBSERVABILITY",
"id": "int_123456",
"name": "PagerDuty",
"provider": "pagerduty"
}
}
Response
200 OKWebhook processed successfully
400 Bad RequestInvalid webhook payload
401 UnauthorizedInvalid or missing signature

Monitor Events

Event TypeDescriptionTrigger Conditions
monitor:createdA new monitor has been createdMonitor creation via UI or API
monitor:updatedMonitor configuration has been modifiedThreshold, condition, or notification changes
monitor:deletedA monitor has been deletedMonitor removal from the system

Monitor Created

monitor:created

Triggered when a new monitor is created in the observability system
POSThttps://api.yourapp.com/webhooks/unizo/scm
Best Practice: Use a dedicated webhook endpoint that can handle multiple event types. You have two architectural options:
• Single endpoint: https://api.yourapp.com/webhooks/unizo - Route all events to one handler
• Category-based endpoints: https://api.yourapp.com/webhooks/unizo/scm - Route by category (scm, ticketing, etc.) for microservices architecture
Headers
NameTypeRequiredDescription
Content-TypestringYesAlways application/json
x-unizo-event-typestringYesEvent type: monitor:created
x-unizo-webhook-idstringYesUnique webhook configuration ID
x-unizo-delivery-idstringYesUnique delivery ID for idempotency
x-unizo-signaturestringYesHMAC SHA-256 signature
Request Body Schema
PropertyTypeRequiredDescription
typestringYesEvent type identifier
versionstringYesWebhook payload version
monitor.idstringYesUnique monitor identifier
monitor.namestringYesMonitor name
monitor.typestringYesMonitor type: metric, log, synthetic
monitor.querystringYesMonitor query or condition
monitor.thresholdsobjectYesAlert thresholds
monitor.createdDateTimestringYesISO 8601 timestamp
monitor.createdByobjectYesUser who created the monitor
integrationobjectYesIntegration details
Example Payload
{
"type": "monitor:created",
"version": "1.0.0",
"monitor": {
"id": "monitor-456",
"name": "API Response Time Monitor",
"type": "metric",
"query": "avg(last_5m):avg:api.response_time{service:payment} > 1000",
"thresholds": {
"critical": 1000,
"warning": 500
},
"createdDateTime": "2024-01-15T14:00:00Z",
"createdBy": {
"id": "user-123",
"email": "devops@example.com",
"name": "DevOps Team"
}
},
"integration": {
"type": "OBSERVABILITY",
"id": "int_123456",
"name": "New Relic Production",
"provider": "newrelic"
}
}
Response
200 OKWebhook processed successfully
400 Bad RequestInvalid webhook payload
401 UnauthorizedInvalid or missing signature

Dashboard Events

Event TypeDescriptionTrigger Conditions
dashboard:createdA new dashboard has been createdDashboard creation via UI or API
dashboard:updatedDashboard has been modifiedWidget additions, layout changes, or filter updates

Dashboard Created

dashboard:created

Triggered when a new dashboard is created
POSThttps://api.yourapp.com/webhooks/unizo/scm
Best Practice: Use a dedicated webhook endpoint that can handle multiple event types. You have two architectural options:
• Single endpoint: https://api.yourapp.com/webhooks/unizo - Route all events to one handler
• Category-based endpoints: https://api.yourapp.com/webhooks/unizo/scm - Route by category (scm, ticketing, etc.) for microservices architecture
Headers
NameTypeRequiredDescription
Content-TypestringYesAlways application/json
x-unizo-event-typestringYesEvent type: dashboard:created
x-unizo-webhook-idstringYesUnique webhook configuration ID
x-unizo-delivery-idstringYesUnique delivery ID for idempotency
x-unizo-signaturestringYesHMAC SHA-256 signature
Request Body Schema
PropertyTypeRequiredDescription
typestringYesEvent type identifier
versionstringYesWebhook payload version
dashboard.idstringYesUnique dashboard identifier
dashboard.namestringYesDashboard name
dashboard.descriptionstringNoDashboard description
dashboard.widgetsarrayYesList of widgets
dashboard.createdDateTimestringYesISO 8601 timestamp
dashboard.createdByobjectYesUser who created the dashboard
integrationobjectYesIntegration details
Example Payload
{
"type": "dashboard:created",
"version": "1.0.0",
"dashboard": {
"id": "dashboard-789",
"name": "Production Overview",
"description": "Main production environment metrics",
"widgets": [
{
"id": "widget-1",
"type": "timeseries",
"title": "CPU Usage"
},
{
"id": "widget-2",
"type": "heatmap",
"title": "Request Distribution"
}
],
"createdDateTime": "2024-01-15T14:00:00Z",
"createdBy": {
"id": "user-456",
"email": "admin@example.com",
"name": "Admin User"
}
},
"integration": {
"type": "OBSERVABILITY",
"id": "int_123456",
"name": "Grafana Cloud",
"provider": "grafana"
}
}
Response
200 OKWebhook processed successfully
400 Bad RequestInvalid webhook payload
401 UnauthorizedInvalid or missing signature

Anomaly Events

Event TypeDescriptionTrigger Conditions
metric:anomaly_detectedAnomaly detected in metricsML-based anomaly detection or statistical deviation

Anomaly Detected

metric:anomaly_detected

Triggered when an anomaly is detected in metrics
POSThttps://api.yourapp.com/webhooks/unizo/scm
Best Practice: Use a dedicated webhook endpoint that can handle multiple event types. You have two architectural options:
• Single endpoint: https://api.yourapp.com/webhooks/unizo - Route all events to one handler
• Category-based endpoints: https://api.yourapp.com/webhooks/unizo/scm - Route by category (scm, ticketing, etc.) for microservices architecture
Headers
NameTypeRequiredDescription
Content-TypestringYesAlways application/json
x-unizo-event-typestringYesEvent type: metric:anomaly_detected
x-unizo-webhook-idstringYesUnique webhook configuration ID
x-unizo-delivery-idstringYesUnique delivery ID for idempotency
x-unizo-signaturestringYesHMAC SHA-256 signature
Request Body Schema
PropertyTypeRequiredDescription
typestringYesEvent type identifier
versionstringYesWebhook payload version
anomaly.idstringYesUnique anomaly identifier
anomaly.metricstringYesMetric name
anomaly.deviationnumberYesDeviation from normal
anomaly.confidencenumberYesConfidence score (0-1)
anomaly.detectedDateTimestringYesISO 8601 timestamp
anomaly.contextobjectYesAdditional context
integrationobjectYesIntegration details
Example Payload
{
"type": "metric:anomaly_detected",
"version": "1.0.0",
"anomaly": {
"id": "anomaly-123",
"metric": "api.request_count",
"deviation": 3.5,
"confidence": 0.95,
"detectedDateTime": "2024-01-15T14:00:00Z",
"context": {
"expected": 1000,
"actual": 3500,
"service": "payment-api",
"environment": "production"
}
},
"integration": {
"type": "OBSERVABILITY",
"id": "int_123456",
"name": "Splunk Enterprise",
"provider": "splunk"
}
}
Response
200 OKWebhook processed successfully
400 Bad RequestInvalid webhook payload
401 UnauthorizedInvalid or missing signature

Webhook Delivery & Retries

Unizo implements automatic retry logic for failed webhook deliveries:

  1. Initial Delivery: Immediate
  2. First Retry: After 1 minute
  3. Second Retry: After 5 minutes
  4. Third Retry: After 15 minutes
  5. Final Retry: After 1 hour

Webhooks are considered failed if:

  • Your endpoint returns a non-2xx status code
  • Connection timeout (30 seconds)
  • SSL/TLS errors

Best Practices

1. Idempotency

Idempotent Webhook Handler

async function handleWebhook(request) {
const deliveryId = request.headers['x-unizo-delivery-id'];

// Check if already processed
if (await isProcessed(deliveryId)) {
  return { status: 200, message: 'Already processed' };
}

// Process webhook
await processWebhook(request.body);

// Mark as processed
await markProcessed(deliveryId);

return { status: 200 };
}

2. Async Processing

Asynchronous Processing

app.post('/webhooks/observability', (req, res) => {
// Validate signature
if (!verifySignature(req)) {
  return res.status(401).send('Invalid signature');
}

// Queue for processing
alertQueue.add(req.body);

// Return immediately
res.status(200).send('OK');
});

3. Alert Routing

Intelligent Alert Routing

async function processAlert(payload) {
const { alert, integration } = payload;

// Route based on severity and tags
if (alert.severity === 'critical') {
  await notifyPagerDuty(alert);
  await createIncident(alert);
} else if (alert.tags.includes('database')) {
  await notifyDatabaseTeam(alert);
} else {
  await notifySlackChannel(alert);
}

// Log for audit
await logAlert(payload);
}

Need Help?

For webhook-related support: