Private Location Monitoring (PLM) brings the all-in-one monitoring, alerting, and reporting suite of the Uptime.com platform to a containerized Docker instance or Kubernetes (k8s) deployment, which can be safely and securely deployed to monitor internal network resources that are otherwise blocked from external access.
Private Location probe servers work just like a normal Location, and communicate encrypted data from internal checks to the Uptime.com central check servers so that it can be viewed, reported on, and managed in line with other external checks in the same Uptime.com account.
This tutorial will walk through the entire Private Location process, from requesting the probe servers themselves to troubleshooting techniques and use cases. For more technical information, such as hardware requirements and Command Line Interface (CLI) usage, check out our Github.
Please note: These instructions are specific to the Private Location image version 3.X. For information on using the old 2.5 (or lower) images, please click here.
- Private Location Monitoring At A Glance
- Prerequisites
- General Overview of Container Setup
- Adding a Private Location to a Check
- Private Location Status
- Special Configurations
- Use Cases
- Troubleshooting a Private Location
- Final Thoughts
Private Location Monitoring At A Glance
Monitor services on internal networks without sacrificing security measures.
Manage internal and external metrics, reports, and alerts from one account.
Protect against internal monitoring gaps to guarantee full stack availability.
Prerequisites
Account Prerequisites
Private Location Monitoring is a feature that Premium-level subscribers can request as an add-on to their existing subscription plan. However, we also offer a limited Free Trial to accounts of all subscription levels.
To request a Private Location, send us an email with the preferred name of the probe server(s) (e.g., MyServer01) that you’d like to configure.
If using a Docker container, please also provide your Docker username for access to the Uptime.com repository.
If you have any questions, please contact support@uptime.com.
Technical Prerequisites
Private Locations deployed to a Docker container require a registered Docker account as well as a Linux system of Ubuntu 20.04 (or higher) using kernel 4.x or 5.x.
Please note: Windows machines are not supported, neither as a Docker host nor via any Linux virtual machines running on a Windows host (eg WSL or VirtualBox).
For a full list of technical and hardware requirements, please see our Github.
Pre-Container Prerequisites
Before the Docker container initialization, you will need the following things:
- An invite and access to the Uptime.com private repository in Docker
- An API Key (provided by Uptime.com Support in the initial request ticket). Please note: Each configured Private Location will need its own API Key.
General Overview of Container Setup
Full instructions for Container Setup, including Docker commands for “login” and “pull image”, and "Starting the Container", are found on our Github under Installation Instructions.
For Kubernetes deployments, a sample yaml file k8s-sample.yaml is available from our Github here.
Please note: Private Location probe servers (and their containers) require write permissions to the machine’s hard drive.
Upgrading from Private Location image v2.X
There are substantial differences and improvements made in the v3.0 image, so it is highly recommended to upgrade as soon as possible.
For full information regarding these changes, please refer to our Github, specifically the section on "Upgrading from 2.X".
Adding a Private Location to a Check
Once the container is started and running, it’s time to add a new (or edit an existing) check to use this Private Location.
From the Edit Check window, Click the “Locations” drop down to select the configured Private Location probe server(s). If adding checks programmatically via our REST API, GET /api/v1/checks/locations/ will provide the list of available locations in the account.
Before finalizing your check, use the Run Test button to verify your settings are configured properly and returning the expected results.
Select the appropriate Private Location from the “Run Test” drop down, then click Run Test.
For help with setting up an Uptime.com check generally, see our article Overview of Checks.
Private Location Status
The Private Location Status displays your private location deployments across various environments.
This dedicated page ensures that you have a comprehensive overview of your private locations, enabling you to track their health, availability, and any potential issues.
Within the Uptime.com UI, navigate to Support > Private Locations Status.
Use the edit button to rename the current private location’s display name.
Please note: Private location servers may show warnings or errors during the first hour of operation following a restart. The Version column is only available on Private Location images v3.0 and above.
Special Configurations
Private Locations can utilize the full range of Uptime.com check types and features, just like a normal Location. Generally, the only difference is that the checks in question need to have the correct Private Location probe server selected from the “Locations” menu (see above).
However, there are a few specific instances that may require additional configuration:
Kubernetes (k8s)
Private Location containers can be added to and managed via a wider Kubernetes (k8s) deployment as needed.
Please see our Github for instructions and a sample config file k8s-sample.yaml for reference.
Monitoring via Proxy
Private Locations support monitoring resources via proxy connections on the host machine, assuming that the proxy is configured in accordance with the official Docker Proxy Server guide.
Once-per-day Checks
Certain Check types, such as SSL Certificate Expiry, WHOIS/Domain Expiry, Domain Blacklist, and Malware/Virus checks, run on a 24 hour interval. To utilize a Private Location to monitor one of these sources, select the appropriate Private Location probe server from the “Locations” drop down menu (the default is Auto).
Traceroute
Traceroute (if enabled with a Premium subscription) can also utilize a Private Location probe server as the starting address. To do so, go to the Real-Time Analysis page for a given check (click Monitoring > Checks, then Actions > Analysis), and then scroll to the bottom. Select the appropriate Private Location from the “Location” drop down, and hit Submit.
Support for Private Location image v2.5 (and lower)
Users running an outdated (v2.5-) image of Private Locations will have slightly different troubleshooting and management steps, as new syntax and commands have been added in v3.0.
For troubleshooting and commands specific to the v2.5 build, please see the appropriate branch of our Github documentation here.
Use Cases
In a general sense, Private Locations allow monitoring checks to access services or URLs that aren’t normally publicly accessible. There are any number of reasons and types of services that are protected from external access that still require monitoring to confirm that everything is up and running as expected. Below are two examples of use cases where Private Locations allow for monitoring without a lapse in security.
Firewalls
Under normal circumstances, protected services or URLs behind a firewall or WAF require IP address whitelisting to allow our Uptime.com Locations access to run the specific check. Depending on the context, service, or organizational security standards, IP whitelisting may not be allowed. In this scenario, Private Locations on an internal Docker instance can exist entirely within the protected system, with only the encrypted check and synchronization data sent to our external Uptime.com check servers.
Intranet Endpoints
Any organization — such as a school, business, or government organization — may use an Intranet server to host resources that don’t need to be accessible to the wider public. Private Locations would allow these private tools, such as a bulletin board, internally hosted metrics, or other employee-only resources, to be monitored just the same as any other external URL or resource. This way, such internal systems can still be monitored by an external party to ensure they’re up and running, without the hassle of exposing those internal endpoints.
Troubleshooting a Private Location
Private Location monitoring is a complex system with many moving parts. Some things may be self-diagnosable, while others may require a ticket to support@uptime.com.
Please note: Confirm that your container is using the most recent image (3.X) to ensure that checks run properly.
To help in the troubleshooting process, it will be important to keep the following things in mind:
Expected Behavior
Docker and Private Location containers can return a status output in JSON format, which will provide details on the Private Location itself, and any associated errors.
These outputs are called via the Command Line Interface (CLI), and are explained in detail on our Github under “Troubleshooting (via CLI)”.
Below is an example of a log file that may seem like an error, but is actually fine:
Adding password for user nagiosadmin ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR:systemctl: Oops, 1 unsupported directory settings. You need to create those before using the service. ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ERROR:systemctl: Oops, 1 unsupported directory settings. You need to create those before using the service. ERROR:systemctl: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING:systemctl: nagios-nrpe-server.service: the use of /bin/kill is not recommended for ExecReload as it is asychronous. That means all the dependencies will perform the reload simultanously / out of order.
In this scenario, there are extra configurations that containers may have, but our slimmed-down image doesn't need.
For further assistance with understanding these outputs, please contact support@uptime.com.
Another message to note is Graceful Shutdown. This message will occasionally appear in the logs to inform you of a past event. This operation is intended to prevent memory leaks, and is completely normal.
In the scenario where a container restart is required, it may take up to an hour for tasks to settle and return to normal. Some checks may fail during this period, but they should correct automatically; checks that remain down should be specifically investigated.
For further help understanding any logs generated by a Private Location, or if checks remain down over an hour after container restart, please reach out to support@uptime.com.
Understanding Container Status Outputs
Docker and Private Location containers can return a status output in JSON format, which will provide details on the Private Location itself, and any associated errors.
These outputs are called via the Command Line Interface (CLI), and are explained in detail on our Github under “Troubleshooting (via CLI)”.
For further assistance with understanding these outputs, please contact support@uptime.com.
Creating backup.tgz
Backup.tgz is a crucial log file that is important for the Uptime.com Support team to troubleshoot problems with a Private Location. If possible, please generate this file and attach it to your support ticket to expedite the troubleshooting process.
These are the instructions for generating the backup.tar file:
- Use the command
docker psto retrieve the running container’s PID. - With the container’s PID, run the command
docker run --rm --volumes-from <RUNNING_CONTAINER_PID> -v $(pwd):/backup ubuntu:latest tar -zcvf /backup/backup.tgz /home/uptime/logs /usr/local/nagios/etc /usr/local/nagios/varto create thebackup.tgzfile in the listed directory.
Save this file from the directory, and attach it to the ticket with support@uptime.com for further assistance.
General Settings
Private Location monitoring does not change or adjust the following network settings:
- SSH
- Firewall (or Firewall rules)
- Timezones
- Hostname
- DNS
- Hostname
- DHCP
Please confirm these settings during any troubleshooting efforts.
Tips and Tricks
Private Locations need monitoring of their own to make sure they’re up and running too. Uptime.com advises using heartbeat check as well as an HTTPS check to monitor the Docker container itself.
To do this, create a heartbeat check and take note of the URL provided in its setup.
Next, create an HTTPS check that points to the URL from your heartbeat check and specify your Private Location as the location for this check.
Finally, for your HTTPS check to successfully transmit data to your heartbeat check, you will need to utilize the optional “String to Post” field since HTTPS uses GET by default. For example, adding “{"response_time": 0.8}” to this field.
By doing this, your HTTPS check will ping your heartbeat check, letting it know that the HTTPS check is operational, and in turn, your Private Location is working as expected.
Final Thoughts
Private Locations allow for internal resources, endpoints, and URLs to be securely and safely monitored without external exposure. To get started, contact support@uptime.com, or request a Private Location directly!