An integration’s most expensive failure isn’t the noisy one, it’s the silent one: the servers are up, the endpoints return 200, but stock hasn’t been updated for three days, or a few orders never reach the ERP every day. The problem is usually noticed by the customer or accounting. At that point, it’s no longer a technical failure but a data inconsistency that needs fixing. Catching it early requires a separate monitoring layer, from correlation IDs to business-level metrics and from alert thresholds to a periodic reconciliation report.
The difference between technical uptime and business flow health
Classic uptime monitoring answers the question “is the service responding?” In an integration, the real question is “is the data reaching the other side within the expected time, completely and correctly?” The difference shows up in these silent failures we see again and again in the field:
- A scheduled job never runs because the cron definition was lost after the server was rebuilt or moved; it doesn’t produce an error either.
- The queue consumer is up, but it retries every message because of a locked record in the ERP; the queue has been growing for hours.
- The webhook provider doesn’t know the signing secret was changed; all events are rejected with 401, and after a while the provider stops trying.
- The API call succeeds, but because the field mapping changed, product prices are written as zero.
- The batch transfer file arrives but is empty; the job ends with “0 records processed, successful”.
No HTTP uptime check catches any of these scenarios. Everything on the dashboard is green. That’s why monitoring is set up at three levels: infrastructure (server, queue broker, database), technical flow (error rate, latency, queue depth) and business outcome (is the expected data on the other side). For infrastructure-side metrics and thresholds, see the server monitoring metrics and thresholds article; here we focus on the other two levels.
Correlation IDs and structured logging
On its way from the website to the ERP, an order passes through several components: the web application, the outbox publisher, the queue, the consumer and the ERP connector. When something fails, the answer to “what happened to order 10452?” should be findable with a single search. Grepping five separate log files on five servers isn’t an investigation, it’s guesswork. Two practices make this possible:
- Correlation ID: An ID (or business key) generated at the start of the flow is carried with every message, every HTTP call and every log line. In HTTP calls, the
traceparentheader from the W3C Trace Context specification is a standard carrier for this; tools such as OpenTelemetry can propagate this header automatically. - Structured logs: Log lines are written as fixed-field JSON instead of free text. This makes filtering and counting by field possible in the log management tool.
{
"ts": "2026-11-10T09:14:22.481Z",
"level": "error",
"flow": "order-erp",
"step": "erp_post",
"correlation_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"business_key": "WEB-10452",
"attempt": 3,
"http_status": 422,
"error_code": "ACCOUNT_NOT_FOUND",
"duration_ms": 812
}
In this structure, the flow and step fields are the basis for generating metrics, and the business_key field is the basis for the support team answering a customer question. Personal data (name, phone, address) and credentials (tokens, passwords) aren’t written to the log; the business key is enough for most investigations. When the “log the full body” setting turned on for debugging is forgotten in production, the log file turns into a personal data store. Don’t forget to turn it off.
Integration metrics to monitor
For each flow, the following metrics make most silent failures visible:
| Metric | What it catches | Source |
|---|---|---|
| Last successful sync time | A job that never runs or gets stuck | A timestamp written at the end of the job |
| Number of records processed (per flow) | Empty file, successful run with zero records | Job summary |
| Queue depth | The consumer can’t keep up | Broker or queue table |
| Age of the oldest waiting message | Stuck message, slow consumer | Broker or queue table |
| Error count (by code and type) | Increase in permanent and temporary errors | Structured log |
| Dead-letter record count | Messages that have used up their retries | DLQ |
| End-to-end latency | Time between the event and the write on the other side | Correlated timestamps |
Queue depth on its own is misleading: a deep but fast-flowing queue on a campaign day is healthy; a shallow queue where a single message has been stuck for hours isn’t. Monitor depth and the age of the oldest message together. Error type classification and the dead-letter setup are explained in the API error handling, retry and backoff guide.
Alert thresholds and noise management
A system that raises an alert for every error soon becomes a system nobody looks at. The operator writes an email rule, the alerts flow into a folder, and the real failure goes with them. These principles help in alert design:
- Alert on the symptom, not the cause. Conditions that show business impact, such as “the last successful transfer in the order flow is older than 30 minutes”, should be alerts rather than “the ERP API returned 503 once”. Individual errors stay in the logs and dashboards.
- Set the threshold according to the business calendar. No orders at night may be normal; zero orders for an hour during business hours shows a problem either on the website or in the flow. Use different thresholds by time window or a comparison with expected volume.
- Define a persistence period. Waiting for the condition to last a few minutes filters out alerts caused by short fluctuations.
- Monitor the absence of the metric too. If the process producing the metric dies, the threshold condition never fires. “Metric not arriving” should be a separate alert.
- Give every alert an owner and a short response note. The alert text should include the flow name, possible causes and the first check step. The person who opens the alert at three in the morning shouldn’t have to search for what to look at.
In an environment using Prometheus, the “last successful sync” rule can be written as below. The metric name is an example; the job must publish its timestamp at the end of every successful run. Details of the rule syntax are in the Prometheus alerting rules document.
groups:
- name: integration
rules:
- alert: StockSyncDelayed
expr: time() - integration_last_success_timestamp_seconds{flow="stock"} > 1800
for: 5m
labels:
severity: warning
annotations:
summary: "Stock sync hasn't completed successfully for 30 minutes"
- alert: StockSyncMetricMissing
expr: absent(integration_last_success_timestamp_seconds{flow="stock"})
for: 15m
labels:
severity: warning
In environments without Prometheus, the same logic can be set up with a scheduled SQL check and an email/Teams notification. The tool is secondary. The condition should measure the business outcome. An externally running “heartbeat” service can also be used: the job sends a request to a specific address on every successful run, and if the request doesn’t arrive within the expected time, the service raises an alert. This is the cheapest way to catch a lost cron definition.
Checking data consistency with a reconciliation report
Metrics show that the flow is running; only comparing the two sides proves that the data is correct. Reconciliation is the job of periodically matching records in the source and target systems and reporting the differences. It’s also the most reliable way to catch missed events in webhook- and queue-based flows; the role of these patterns is explained in the ERP and web integration patterns article.
An example reconciliation query for the order flow: records created on the web in the last 48 hours, with payment completed but no order number returned from the ERP. The last 30 minutes are out of scope to exclude normal processing delay.
SELECT o.id, o.created_at, o.status
FROM orders o
LEFT JOIN erp_order_map m ON m.web_order_id = o.id
WHERE o.created_at BETWEEN now() - interval '48 hours'
AND now() - interval '30 minutes'
AND o.status NOT IN ('cancelled', 'pending_payment')
AND m.erp_order_no IS NULL
ORDER BY o.created_at;
This query trusts the web side’s own mapping table. A stronger check is to pull the list of “orders with a web reference” from the ERP for the same time range and compare it with the web list in both directions: records on the web but not in the ERP (missing) and records found twice in the ERP (duplicates) are reported separately. For master data such as stock and prices, quantity or price is compared by product code; if there are many products, totals by category are compared first, and only the groups that differ are examined at row level.
Write a process for differences too: which difference is fixed automatically (for example, re-queuing a missing order), which goes to human review, and who is responsible. A difference report without an owner doesn’t get read.
Integration monitoring checklist
- Are each flow’s name, owner, source and target systems, expected run frequency and longest acceptable delay documented?
- Do the correlation ID and business key appear with the same field name in the logs of all components?
- Are logs collected centrally in a structured format, and are they free of personal data?
- Are the last successful run time and the number of records processed published as metrics for each flow?
- Are queue depth and the age of the oldest message monitored together?
- Is a missing metric defined as a separate alert?
- Do alerts have an owner, a response note and thresholds based on business hours?
- Is a periodic reconciliation report running for critical flows, and are results without differences recorded too?
- Is a correction process defined for dead-letter records and reconciliation differences?
Use this list as an acceptance criterion when a new integration goes live; monitoring then stops being something added afterwards. For receiver-side indicators in webhook-based flows, also see the webhook receiver design guide. For setting up monitoring and alerting for your integration flows, you can review the scope on our web and uptime monitoring and ERP, CRM and API integrations pages.
Frequently asked questions
Why doesn’t uptime monitoring catch integration errors?
Uptime monitoring only measures whether a service responds. Integration errors, however, mostly occur while the services are up: the job never runs, messages get stuck in the queue or data is written with the wrong mapping. Catching these requires business-level checks such as the last successful sync time, the number of records processed and reconciliation.
How often should the reconciliation report run?
The frequency is determined by the flow’s longest acceptable delay. For critical flows such as orders, hourly or several times a day; for master data such as product records, daily or weekly reconciliation is enough in most scenarios. Even if the report finds no differences, it should record its result on every run.
What’s the difference between a correlation ID and a business key?
A correlation ID is a technical identifier generated to trace a single flow run or request across components. A business key is a value the business side recognizes, such as an order number. Both are logged together; the technical team searches by correlation ID, the support team by business key.
How can alert noise be reduced?
Alerts are defined on conditions that show business impact rather than individual technical errors, the condition is required to persist for a while, and thresholds are adjusted to business hours. Every alert has an owner and a response note. Alerts that haven’t been acted on for a long time are reviewed and either fixed or removed.