Skip to content
Hyperion

Check the health of your nodes

The Nodes page lists the master and every worker node, with one line that says whether anything needs attention. Each worker sends the master a heartbeat every 60 seconds, and the page builds its verdict from those heartbeats and from the settings sync.

Before you start

Section titled “Before you start”
  • Sign in as an Owner. The Nodes page, its actions and node settings are Owner-only.
  • A worker must already be enrolled. To add one, use the install guide for new nodes. The master always has a row, marked this server.

Steps

Section titled “Steps”
  1. Open Nodes in the sidebar, under Cluster.
  2. Read the line at the top. With no problems it says “Only this server so far — add a node below to spread sites across machines”, “The worker node is checking in, nothing needs a look” or “All N worker nodes checking in, nothing needs a look”. Otherwise it names up to three nodes with their problems, separated by ·, worst first, then +N more.
  3. Scan the rows. Each row shows:
    • a green dot when the node is online and a grey dot when it is not. A worker is online when its last heartbeat is no more than 5 minutes old. The master is always online.
    • the label, with flags: this server, Test for a test node and Drained for a drained one.
    • the public IP, the agent version, and seen <time> ago for a worker.
    • live facts that load as you scroll: the number of sites, the round trip in milliseconds, and the operating system update state.
  4. Click a row to open it. The facts strip shows Node ID, Public IP, Agent, Enrolled, Last heartbeat, Settings sync and links to Stats, Services and Settings for that node.
  5. For a worker, under Connection, click Test connection. The result appears beside the button.
  6. Under Operating system updates, read the state. Click Check now to refresh the package index. This runs apt-get update on the node and installs nothing.

What the facts and problems mean

Section titled “What the facts and problems mean”
You see Meaning
N sites, N ms The node answered a signed call. The time is the round trip.
not answering The node did not answer. Hover for the reason.
up to date / updates not checked The node’s package index says nothing is pending, or nobody has checked yet.
N updates, N security Packages are waiting. Security updates come from the distribution’s security suite.
reboot required A package asked for a reboot.
no heartbeat since <time> The worker has not checked in for more than 5 minutes.
has never checked in The node is enrolled but no heartbeat has arrived.
agent <version> differs from the master's Run Update on the node. Older agents may not support newer features.
no TLS pin on file The master has not recorded the node’s certificate pin yet.
no response-signing key The master has not recorded the node’s reply-signing key yet.
settings from the master did not arrive The settings sync reached the node and failed. See Keep node settings in sync.
not taking the master's settings — update it The node runs an agent too old for the sync.

The test result reads reachable · <version> · N hostings · N ms when it works. Otherwise it shows the error text.

Check that it worked

Section titled “Check that it worked”
  • The line at the top says “checking in, nothing needs a look”, and every worker’s dot is green.
  • Test connection shows reachable, and the version matches the master’s.
  • Operating system updates shows no updates pending with a recent checked time. A count above 0 is shown with a Show N packages list. To install them, see Update a node.

If it fails

Section titled “If it fails”
  • A worker shows no heartbeat since …: confirm the machine is up and hyperion-agent is running there (sudo systemctl status hyperion-agent). The heartbeat goes from the node to the master, so also check that the master’s URL is reachable from the node.
  • Test connection shows an error but the dot is green: the heartbeat travels from the node to the master, and the test goes the other way, from the master to the node’s agent port. The port is 9443 by default (the bind key in the [remote_rpc] section of /etc/hyperion/agent.toml). Open it on the node’s firewall for the master.
  • no TLS pin on file: restart hyperion-agent on the node and wait one heartbeat, which is up to 60 seconds. While pinning is enforced the master refuses a node without a pin. See Lock down the master-to-node channel.
  • no response-signing key: run Update on the node, then wait one heartbeat.
  • Operating system updates reads Could not read update status: …: the node returned an error. Try Check now.
  • “This node runs an older Hyperion that does not report operating-system updates yet. Update Hyperion on it to see them.”: update the node’s Hyperion.
  • The Nodes page redirects you to the dashboard: you are not an Owner.
Section titled “Related”