Configuring a monitoring system to reduce redundant or unnecessary alerts reduces notification fatigue, eliminates frustrating off-hours alerts, and ensures that legitimate alerts receive proper attention. Below are some recommendations that can reduce false alerts.
Adjust Sensitivity Settings
Sensitivity defines the number of locations that must report an error before a check moves into the Down state. For example, if a check is assigned 3 locations with a sensitivity of 2, then 2 of those locations must report an error before the check is considered Down. However, if the configured sensitivity is 1, then a single location reporting an error will cause the entire check to report Down. This increases the chances of false alarms due to a single probe failure, which may simply be an intermittent routing issue or some other problem that doesn't necessarily reflect a legitimate outage.
Uptime.com recommends a sensitivity of at least 2, except for special use cases.
Apply Maintenance Windows
Maintenance windows define a period when a check is expected to report outages due to a known technical issue or routine maintenance to the resource. Be sure that maintenance windows are scheduled in advance when possible to reduce false alerts, especially during off-hours when maintenance typically occurs and alerts could be unnecessary and frustrating.
Consolidate Contacts
While increased granularity in contacts can help when different alerting groups require mixed alert notification methods, too much granularity can also cause overlap in alerting, causing some contacts to receive multiple notifications for single outages. This can be especially detrimental when SMS or phone alerting credits are used to unnecessarily alert multiple contacts. Consider the best combination of granularity and consolidation for your alerting groups as best suits your use case, and consider using escalation policies so that serious outages are routed to the appropriate team members at the most effective time intervals.
Comments
0 comments