Monitoring network devices and server hardware in Kunobi with the SNMP and Redfish extensions

A Kubernetes cluster runs on hardware that Kubernetes cannot see. The nodes talk to each other through switches, reach the outside world through routers, and sit behind firewalls, and they run on physical servers with their own power supplies, drives, and cooling. When a node goes NotReady or a volume mount stalls, the cause is often one layer down - a saturated uplink, a port left administratively shut after a change nobody reverted, a failed power supply, or a server that overheated and throttled itself.

There are two layers of that hardware, and each is monitored somewhere else, if at all. Network gear speaks SNMP, read by tools like LibreNMS, Zabbix, or Observium. Servers expose a management controller that speaks Redfish, reached through per-vendor consoles like Dell iDRAC or HPE iLO. None of them is the Kubernetes resource browser a platform engineer keeps open all day, so both layers become tabs you check by hand, on the days you remember to.

Two extensions pull those layers into Kunobi. The SNMP extension polls your network devices; the Redfish extension talks to your servers' management controllers. Both render the result through the same drilldown tables and detail tabs we already use for Pods and Deployments, so devices, servers, their interfaces, and their components all become resource kinds you browse like any other. SNMP gives a read view of the network. Redfish reads the servers and can also act on them.

What SNMP is

If you have not worked with it: SNMP is how network devices expose their internal state. An agent runs on the device and answers queries about a tree of values - interface counters, system uptime, error rates - each addressed by a numeric object identifier, or OID, that looks like 1.3.6.1.2.1.1.1.0. A MIB is the dictionary that maps those numbers to readable names, so 1.3.6.1.2.1.1.1.0 becomes sysDescr, the device's description of itself.

Two operations do most of the work. A GET fetches one value; a WALK reads a whole subtree by stepping through it. Each value that comes back is a varbind, an OID paired with its value. The extension speaks SNMPv2c, where a device authenticates a request by a shared community string.

Connecting your devices

Everything starts on the Sources screen, under SNMP Devices. A device here is a host, a port, and a community string, plus the profile that decides which parts of the tree get polled. SNMP has no separate session to open, so the device row is the connection: the one you mark active is the one whose live status the rest of the app reflects.

The Test action checks a device before it's saved. It sends a single sysDescr GET and waits for a reply, so a wrong host or a mistyped community fails right there in the dialog, before it becomes an empty screen an hour later. Register a rack of switches and the routers and firewalls in front of them, and they all live here, a keystroke away from making any one active.

Every device, at a glance

In the Data workspace, SNMP devices show up as a resource kind called SnmpDevice, listed the same way Pods are.

The Status column shows a computed verdict. The extension polls each device on its own interval and counts failures; three consecutive misses flip a device to offline, and until then it stays unknown, so a single dropped packet never cries wolf. Here edge-fw-02 has crossed that line, carrying its last error and a failure count of 4 on the row, while the two that answer stay online. The device's own name and profile come from the same polls, so the table sorts and filters like any other resource list - sort by Status or Failures and the box that stopped answering comes to the top.

What the device says about itself

Drilling into a device opens its detail tabs. The Overview reads the SNMP system group - the handful of values every agent is required to expose.

sysDescr is the device telling you what it is: vendor, model, and software version in one string, here Cisco IOS XE Software, Version 17.9. Next to it are the system name and the uptime, and below them the poll status and the last error if there was one. It's the fastest answer to "what is this box and is it healthy" without opening a console.

Raw OIDs, turned into something readable

Every poll stores its varbinds untouched. The Poll Results tab is where they become legible.

The left column is the raw OID the device returned. The next two are resolved against the MIB database that ships inside the extension - around 3,200 standard IETF and IANA OIDs, embedded in the binary and always loaded - so 1.3.6.1.2.1.1.1.0 reads as SNMPv2-MIB::sysDescr.0 with a type of OctetString, and the interface counters carry their IF-MIB names and Counter64 types. Running snmpwalk from a shell gives you the left column; the right two are the difference between a wall of dotted numbers and a table you can read, and getting them from the CLI means installing the matching MIB files system-wide first.

Interfaces, without a second poll

The values that matter most for a network device are per-interface, and they already arrive in the poll. The extension parses the standard interface tables out of the varbinds it has and presents each interface as its own resource kind, SnmpInterface, with no additional queries to the device.

Each row carries the interface name and description, its speed, MTU and error counters, and two separate status columns. ifAdminStatus is the configured intent and ifOperStatus is the live reality. Here Gi0/1 is admin-up but oper-down with a climbing error count - a cable or a peer problem - while Gi0/2 reads admin-down, a port shut on purpose. Reading the two together is how you tell a fault from a deliberate change. The interfaces are keyed back to their device, so a saturated uplink or a flapping port is a sortable row next to the switch it belongs to.

Asking a device something directly

Scheduled polling covers the profile, but sometimes you want to ask a specific question now. The OID browser runs a WALK or a GET against a chosen device on demand: pick a device, type a numeric OID or a MIB name - sysDescr, IF-MIB::ifTable - and the extension resolves the name to a number before it queries. The suggestions are drawn from OIDs the device has actually returned in recent polls, so they point you at branches this device answers. It saves dropping to a terminal and remembering snmpwalk's flags for a one-off lookup.

Bringing your own vendor MIBs

The standard MIBs name the common tree, but every vendor hangs its own metrics off the private branch at 1.3.6.1.4.1, and those need the vendor's MIB files to read. The extension imports them per directory and tracks each import as a resource kind of its own, SnmpMibSource.

Each source shows how many modules and OIDs it contributed, its size, and whether it is enabled. Toggling a source off is instant and does not re-parse anything, so you can mute a noisy vendor tree and bring it back later without re-importing; removing one deletes its parsed file. Here the Cisco and Arista sources are enabled and the Juniper one is switched off, and the OIDs from the enabled sources flow straight into the same name resolution the Poll Results tab uses.

The MIB inventory, per device

The same inventory is summarized on each device's MIB Sources tab, alongside the standard set.

The totals across the top count everything currently loaded, the standard MIBs sit in their own always-on row, and the imported sources follow. It's the quick confirmation of which dictionary is naming the values you're reading on that device.

Where the imports come from

Importing a vendor directory is the one place the extension reaches outside itself. It shells out to gomib, an external tool that parses MIB text files into a structured form, compresses the result, and loads it.

The dialog checks that gomib is reachable before it offers to scan, so a missing dependency shows up as a clear message up front. Point it at a directory of .mib files, give the source a name, and its OIDs join the database.

The knobs

Settings holds the defaults that shape polling.

A profile is a named set of OID subtrees - generic, router, server ship built in - and it decides how much of the tree each device is asked for. The default poll interval and default profile apply to any device that does not override them, and the gomib path is there for the import step above. A device polling a wide profile every fifteen seconds and one polling a narrow profile every five minutes are both just rows with different settings.

Adding a device

The add form gathers the same pieces the Sources list shows, in one place.

Host, port, a community string, and a profile, with v2c assumed and a Test Connection button that checks reachability before the device is saved. Fill it in, and the device joins the list and the next poll cycle picks it up.

That covers the network the cluster runs across. The servers the cluster runs on are the second invisible layer, and reaching them takes a different protocol and a second extension.

What Redfish is, and why it needs no MIBs

Where SNMP exposes a tree of numbered values that only a MIB can name, Redfish answers over HTTPS with structured JSON that already carries its own field names. The extension talks to a server's baseboard management controller, the small always-on computer on the board that runs independently of the host operating system. Because that controller has its own processor and network port, it answers whether or not the machine has booted, which is what makes it reachable for a node that has dropped off the cluster. Redfish is a REST API standardized by the DMTF, and its data model has the same shape across vendors, so there is no per-vendor dictionary to assemble the way SNMP needs its MIBs. On the first connection Kunobi reads the service root, detects the vendor, and loads a matching provider (Dell iDRAC, HPE iLO, or a generic one for any compliant controller) which adds the vendor's own fields, such as a Dell service tag, on top of the standard set.

Registering a server

A Redfish server lives on the same Sources screen as an SNMP device, so registering one is the motion already shown above: the Add Server form takes a name, a host and port, credentials, and a polling profile. The part worth stopping on is Test Connection. For an SNMP device the equivalent sends a single sysDescr GET; for a Redfish server it opens the service root and reports the vendor and model it found, so running it before saving confirms the endpoint is reachable and actually speaks Redfish, and surfaces a bad address or wrong credential right there in the dialog. Behind the form, the servers already registered on this workspace are listed with the model and power state each one last reported.

The server list

In the Data workspace, Redfish servers appear as a resource kind called RedfishServer, listed the same way Pods are. The columns are the ones specific to a physical machine: its power state and its health, side by side, next to the vendor and model.

Here all three servers are connected. The Dell is powered on and healthy, the Lenovo is powered off, and the HPE is powered on while reporting an overall health of Critical. Power state and health sit in the same row, so a server that is powered on but unhealthy is easy to spot, and sorting by health brings it to the top of the list.

Down to the board

You usually open a server to answer something specific, like ruling out a failing drive or a bad memory module on a node that keeps flapping. Redfish reads the hardware as structured objects. Each component type is its own resource kind, keyed back to the server, so you browse it like any other resource list.

Processors are one such kind, each listed with its model, core and thread counts, and clock speed.

Memory modules, network interfaces, and storage are separate kinds alongside it, each browsable the same way. Drives, for example, carry media type, capacity, protocol, and the controller's own status, while volumes alongside them carry the RAID type.

This is inventory that normally lives only in a vendor BMC UI or an asset spreadsheet. Here each kind sorts and filters next to the server it belongs to, and a drive or RAID controller sitting in a warning state is visible without logging into the controller.

Sensors and the server's own logs

A server also reports how it is doing physically. Temperatures are their own kind, each sensor listed with its current reading and the warning and critical thresholds the hardware defines for it.

Fans and power supplies are sibling health kinds read from the same polls. A server that overheats will throttle or reset, and a node that reboots without an obvious cause is often a thermal or power fault visible in these readings.

The server also keeps its own records, and the extension reads two of them. Firmware inventory lists each component and its installed version, so version drift across your servers becomes visible.

The System Event Log is the controller's own record of hardware events, each carrying a severity, a message, and a timestamp.

Acting on a server

Everything so far is a read view, like the SNMP half of this post. Redfish can also change a server's state. From a selected server you can run a power action:

• On
• Graceful Shutdown
• Force Off
• Force Restart
• Power Cycle

You can also set a one-shot next-boot device so the machine comes up on PXE, its disk, or straight into BIOS setup, and switch the chassis indicator LED on, off, or blinking to find the physical box in a rack.

The actions that abruptly cut power are guarded. Force Off, Force Restart, and Power Cycle require typing the server's name to confirm before they run.

These are ordinary out-of-band operations, the kind you would otherwise open a vendor console for, like power-cycling a wedged node or reinstalling one over PXE. Redfish puts them on the server's row.

What they are, and aren't

Together the two extensions are a view of the hardware under your cluster, rendered as the sortable tables and detail tabs you already use for Kubernetes. On the SNMP side that means device and interface status, the system group, every poll's varbinds named through a built-in MIB layer, on-demand GET and WALK, and vendor MIBs you bring yourself. Redfish adds the servers themselves: inventory, health sensors, firmware and event logs, plus the power, boot, and LED control that only it offers.

Both have limits. SNMP speaks v2c today; SNMPv3, with per-user authentication and encryption, is modeled in the configuration but not yet wired to a live session. Redfish covers monitoring and control through standard endpoints, with Dell and HPE enrichment and a generic fallback for any compliant controller. And in both cases the correlation to Kubernetes is still manual: devices, servers, and interfaces are published into Kunobi's topology graph, but no rules yet draw an edge from an interface or a server to the node behind it, so tracing a NotReady node down to the uplink or the power supply is a step you still make yourself.

Even with those limits, the switches and routers between the nodes and the servers beneath them stop being separate tools you open on a good day. They become more resource types in the window you already have in front of you.