Private Location Monitoring (PLM) brings the all-in-one monitoring, alerting, and reporting suite of the Uptime.com platform to a containerized Docker instance, which can be safely and securely deployed to monitor internal network resources that are otherwise blocked from external access.
Private Location probe servers work just like a normal Location, and communicate encrypted data from internal checks to the Uptime.com central check servers so that it can be viewed, reported on, and managed in line with other external checks in the same Uptime.com account.
This tutorial will walk through the entire Private Location process, from requesting the probe servers themselves to troubleshooting techniques and use cases. For more technical information, such as hardware requirements and Command Line Interface (CLI) usage, check out our Github.
- Private Location Monitoring At A Glance
- Prerequisites
- General Overview of Container Setup
- Adding a Private Location to a Check
- Special Configurations
- Use Cases
- Troubleshooting a Private Location
- Final Thoughts
Private Location Monitoring At A Glance
Monitor services on internal networks without sacrificing security measures.
Manage internal and external metrics, reports, and alerts from one account.
Protect against internal monitoring gaps to guarantee full stack availability.
Prerequisites
Account Prerequisites
Private Location Monitoring is a feature that Premium-level subscribers can request as an add-on to their existing subscription plan. However, we also offer a limited Free Trial to accounts of all subscription levels.
To request a Private Location, send us an email with your Docker username and the preferred name of the probe server(s) (eg MyServer01) that you’d like to configure.
If you have any questions, please contact support@uptime.com.
Technical Prerequisites
Private Locations require a registered Docker account as well as a Linux system of Ubuntu 20.04 (or higher) using kernel 4.x or 5.x.
Please note: Windows machines are not supported, neither as a Docker host nor via any Linux virtual machines running on a Windows host (eg WSL or VirtualBox).
For a full list of technical and hardware requirements, please see our Github.
Pre-Container Prerequisites
Before the Docker container initialization, you will need the following things:
- An invite and access to the Uptime.com private repository in Docker
- An API Key (provided by Uptime.com Support in the initial request ticket). Please note: Each configured Private Location will need its own API Key.
General Overview of Container Setup
Full instructions for Container Setup, including downloading the seccomp-config.json file, Docker commands for “login” and “pull image”, and "Starting the Container", are found on our Github under Installation Instructions.
Please note: Private Location probe servers (and their container) require write permissions to the machine’s hard drive.
Adding a Private Location to a Check
Once the container is started and running, it’s time to add a new (or edit an existing) check to use this Private Location.
From the Edit Check window, Click the “Locations” drop down to select the configured Private Location probe server(s). If adding checks programmatically via our REST API, GET /api/v1/checks/locations/ will provide the list of available locations in the account.
Before finalizing your check, use the Run Test button to verify your settings are configured properly and returning the expected results.
Select the appropriate Private Location from the “Run Test” drop down, then click Run Test.
For help with setting up an Uptime.com check generally, see our article Overview of Checks.
Special Configurations
Private Locations can utilize the full range of Uptime.com check types and features, just like a normal Location. Generally, the only difference is that the checks in question need to have the correct Private Location probe server selected from the “Locations” menu (see above).
However, there are a few specific instances that may require additional configuration:
Kubernetes (k8s)
While technically unsupported, please see our Github for a sample k8s configuration, in the file k8s-sample.yaml for reference.
Monitoring via Proxy
Private Locations support monitoring resources via proxy connections on the host machine, assuming that the proxy is configured in accordance with the official Docker Proxy Server guide.
Once-per-day Checks
Certain Check types, such as SSL Certificate Expiry, WHOIS/Domain Expiry, Domain Blacklist, and Malware/Virus checks, run on a 24 hour interval. To utilize a Private Location to monitor one of these sources, select the appropriate Private Location probe server from the “Locations” drop down menu (the default is Auto).
Traceroute
Traceroute (if enabled with a Premium subscription) can also utilize a Private Location probe server as the starting address. To do so, go to the Real-Time Analysis page for a given check (click Monitoring > Checks, then Actions > Analysis), and then scroll to the bottom. Select the appropriate Private Location from the “Location” drop down, and hit Submit.
Use Cases
In a general sense, Private Locations allow monitoring checks to access services or URLs that aren’t normally publicly accessible. There are any number of reasons and types of services that are protected from external access that still require monitoring to confirm that everything is up and running as expected. Below are two examples of use cases where Private Locations allow for monitoring without a lapse in security.
Firewalls
Under normal circumstances, protected services or URLs behind a firewall or WAF require IP address whitelisting to allow our Uptime.com Locations access to run the specific check. Depending on the context, service, or organizational security standards, IP whitelisting may not be allowed. In this scenario, Private Locations on an internal Docker instance can exist entirely within the protected system, with only the encrypted check and synchronization data sent to our external Uptime.com check servers.
Intranet Endpoints
Any organization — such as a school, business, or government organization — may use an Intranet server to host resources that don’t need to be accessible to the wider public. Private Locations would allow these private tools, such as a bulletin board, internally hosted metrics, or other employee-only resources, to be monitored just the same as any other external URL or resource. This way, such internal systems can still be monitored by an external party to ensure they’re up and running, without the hassle of exposing those internal endpoints.
Troubleshooting a Private Location
Private Location monitoring is a complex system with many moving parts. Some things may be self-diagnosable, while others may require a ticket to support@uptime.com.
Please note: Confirm that your container is using the most recent image (2.5) to ensure that checks run properly.
To help in the troubleshooting process, it will be important to keep the following things in mind:
Expected Behavior
Docker and Private Location containers can return a status output in JSON format, which will provide details on the Private Location itself, and any associated errors.
These outputs are called via the Command Line Interface (CLI), and are explained in detail on our Github under “Troubleshooting (via CLI)”.
Below is an example of a log file that may seem like an error, but is actually fine:
Adding password for user nagiosadmin ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR:systemctl: Oops, 1 unsupported directory settings. You need to create those before using the service. ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR:systemctl: Oops, 1 unsupported directory settings. You need to create those before using the service. ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING:systemctl: nagios-nrpe-server.service: the use of /bin/kill is not recommended for ExecReload as it is asychronous. That means all the dependencies will perform the reload simultanously / out of order.
In this scenario, there are extra configurations that containers may have, but our slimmed-down image doesn't need.
For further assistance with understanding these outputs, please contact support@uptime.com.
Another message to note is Graceful Shutdown. This message will occasionally appear in the logs to inform you of a past event. This operation is intended to prevent memory leaks, and is completely normal.
In the scenario where a container restart is required, it may take up to an hour for tasks to settle and return to normal. Some checks may fail during this period, but they should correct automatically; checks that remain down should be specifically investigated.
For further help understanding any logs generated by a Private Location, or if checks remain down over an hour after container restart, please reach out to support@uptime.com.
Understanding Container Status Outputs
Docker and Private Location containers can return a status output in JSON format, which will provide details on the Private Location itself, and any associated errors.
These outputs are called via the Command Line Interface (CLI), and are explained in detail on our Github under “Troubleshooting (via CLI)”.
For further assistance with understanding these outputs, please contact support@uptime.com.
Creating backup.tar
Backup.tar is a crucial log file that is important for the Uptime.com Support team to troubleshoot problems with a Private Location. If possible, please generate this file and attach it to your support ticket to expedite the troubleshooting process.
These are the instructions for generating the backup.tar file:
-
Use the command
docker psto retrieve the running container’s PID. -
With the container’s PID, run the command
docker run --rm --volumes-from RUNNING_CONTAINER_PID -v $(pwd):/backup ubuntu:latest tar -cvf /backup/backup.tar /home/webapps/uptime /usr/local/nagiosto create thebackup.tarfile in the listed directory.
Save this file from the directory, and attach it to the ticket with support@uptime.com for further assistance.
General Settings
Private Location monitoring does not change or adjust the following network settings:
- SSH
- Firewall (or Firewall rules)
- Timezones
- Hostname
- DNS
- Hostname
- DHCP
Please confirm these settings during any troubleshooting efforts.
Tips and Tricks
Private Locations need monitoring of their own to make sure they’re up and running too. Uptime.com advises using Custom process checks to monitor the Docker container itself. This can be done in (at least) two ways:
- Use the URL provided by an incoming webhook or heartbeat check as part of an external script on the host. Run this script periodically to make sure that your Docker instance, or the specific Private Location container is running. This is the most robust method, and our primary recommendation.
- Create a HTTP(s) check that monitors an internal resource via the given Private Location probe server. If this check ever fails, an alert will be sent to the configured contact to signal that something is wrong with the Private Location probe server.
Final Thoughts
Private Locations allow for internal resources, endpoints, and URLs to be securely and safely monitored without external exposure. To get started, contact support@uptime.com, or request a Private Location directly!