Skip to content

feat(panos): configure HA configuration for panos - #2633

Open
khackworth wants to merge 1 commit into
mainfrom
PUC-2127_panos_ha_config
Open

khackworth wants to merge 1 commit into
mainfrom
PUC-2127_panos_ha_config

Conversation

@khackworth

@khackworth khackworth commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

What this does

This will build the HA configuration on the Palo Altos early in its ironic lifecycle. The primary reason for this component moving forward into the inspect process is that we need to identify cabling issues early before devices are deployed. Unfortunately, with these, there is no method to query the ha interfaces unless they are fully brought up (build the HA config). The NetSec standards for building HA devices, which has not changed for years, has been integrated into code. These items may be better handled as vars passed in from undercloud_deploy, but due to the static nature, I've placed in code here.

    if role == "primary":
        return {
            "ha1_ip": "10.2.2.1",
            "ha1_backup_ip": "10.2.2.5",
            "ha2_ip": "10.2.2.9",
            "peer_ha1": "10.2.2.2",
            "peer_ha1_backup": "10.2.2.6",
            "priority": "50",  # Lower = active
        }
    else:  # secondary
        return {
            "ha1_ip": "10.2.2.2",
            "ha1_backup_ip": "10.2.2.6",
            "ha2_ip": "10.2.2.10",
            "peer_ha1": "10.2.2.1",
            "peer_ha1_backup": "10.2.2.5",
            "priority": "100",  # Higher = passive
        }

Basic Discovery

  • Get system info (model, serial, version, uptime)
  • Get management interface config (IP, netmask, gateway)
  • Discover LLDP neighbors on all interfaces
  • Create/update ports in Ironic (cleanup stale ports)
  • Store data in node.extra

HA Configuration (if has HA role trait)

  • Get HA role from traits (PRIMARY/SECONDARY)
  • Calculate HA addresses based on role
  • Configure HA interfaces (ha1-a, ha1-b, hsci)
  • Configure peer-ip, group-id, priority
  • Commit configuration
  • Store ha_role in node.extra

HA Validation (if has mate_serial)

  • Find mate node by serial
  • Pre-flight checks:
    • Mate serial matches peer serial
    • Both have management_ip
    • Matching resource_class
    • Opposite HA roles
    • Query device HA status
  • Post-config checks:
    • HA interfaces up (ha1-a, ha1-b, hsci)
    • Peer connection up
    • Peer serial matches mate_serial
    • Query mate device for bidirectional validation
    • Store ha_operational, ha_peer_serial in node.extra

Auto-transition (if validation passed)

  • RPC call: transition mate to AVAILABLE
  • RPC call: transition self to AVAILABLE

Exit States

  • AVAILABLE: Full HA validation passed
  • MANAGEABLE: No HA config, or config without mate, or validation failed

--

Testing performed

Inspected a device WITHOUT a trait & with a mate_serial

Expected

  • LLDP discovery to work successfully
  • No HA configuration applied
  • Device will NOT auto-promote to available (Deploying standalone devices is outside of our currently defined scope, as such, automation should probably not take part in moving devices forward that do not meet the criteria. We should not fail, but should not promote either)
2026-11-06 13:51:12.007 14 DEBUG ironic_understack.drivers.panos_inspect [None req-92061a99-47e9-4245-bc0e-30f8b74e2f13 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Updated port pa1410-dev-fw5:ethernet1/19 _update_ports_from_neighbors /var/lib/openstack/lib/python3.12/site-packages/ironic_understack/drivers/panos_inspect.py:411
2026-11-06 13:51:12.022 14 DEBUG ironic_understack.drivers.panos_inspect [None req-92061a99-47e9-4245-bc0e-30f8b74e2f13 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Updated port pa1410-dev-fw5:ethernet1/20 _update_ports_from_neighbors /var/lib/openstack/lib/python3.12/site-packages/ironic_understack/drivers/panos_inspect.py:411
2026-11-06 13:51:12.023 14 INFO ironic_understack.drivers.panos_inspect [None req-92061a99-47e9-4245-bc0e-30f8b74e2f13 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Firewall validation passed: 2 ports discovered, categories=['inside', 'outside']
⚠️ 2026-11-06 13:51:12.023 14 WARNING ironic_understack.drivers.panos_inspect [None req-92061a99-47e9-4245-bc0e-30f8b74e2f13 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Has mate_serial 026701009945 but no traits
⚠️ 2026-11-06 13:51:12.024 14 INFO ironic_understack.drivers.panos_inspect [None req-92061a99-47e9-4245-bc0e-30f8b74e2f13 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Not part of HA pair, skipping HA configuration

inspected a device with a trait & without a mate_serial

  • LLDP discovery to work successfully
  • No HA configuration applied
  • Device will NOT auto-promote to available (Deploying standalone devices is outside of our currently defined scope, as such, automation should probably not take part in moving devices forward that do not meet the criteria. We should not fail, but should not promote either)
2026-11-06 13:58:39.773 14 DEBUG ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Updated port pa1410-dev-fw5:ethernet1/19 _update_ports_from_neighbors /var/lib/openstack/lib/python3.12/site-packages/ironic_understack/drivers/panos_inspect.py:411
2026-11-06 13:58:39.789 14 DEBUG ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Updated port pa1410-dev-fw5:ethernet1/20 _update_ports_from_neighbors /var/lib/openstack/lib/python3.12/site-packages/ironic_understack/drivers/panos_inspect.py:411
2026-11-06 13:58:39.789 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Firewall validation passed: 2 ports discovered, categories=['inside', 'outside']
2026-11-06 13:58:39.790 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Not part of HA pair, skipping HA configuration

inspected a device with a trait & with a mate_serial both properly configured

  • LLDP discovery to work successfully
  • HA configuration applied
  • Both devices auto-promote
2026-11-06 14:11:07.790 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] HA interfaces operational
2026-11-06 14:11:08.007 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] HA peer connected: 026701009945
2026-11-06 14:11:08.030 14 DEBUG ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Found mate node 7787db8b-ed9f-4713-8c13-df075903bc4f with serial 026701009945 _find_mate_node /var/lib/openstack/lib/python3.12/site-packages/ironic_understack/drivers/panos_inspect.py:584
2026-11-06 14:11:08.030 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Pre-flight checks passed: roles=primary/secondary, class=pa1410

2026-11-06 14:11:12.253 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Bidirectional HA validation passed
2026-11-06 14:11:12.254 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] HA pair fully validated: 026701009584 (primary) <-> 026701009945 (secondary)
2026-11-06 14:11:12.254 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Transitioning mate 7787db8b-ed9f-4713-8c13-df075903bc4f to AVAILABLE
2026-11-06 14:11:12.255 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Using RPC to transition mate 7787db8b-ed9f-4713-8c13-df075903bc4f
2026-11-06 14:11:12.270 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] RPC request sent to transition mate 7787db8b-ed9f-4713-8c13-df075903bc4f
2026-11-06 14:11:12.271 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Transitioning self to AVAILABLE
2026-11-06 14:11:12.279 14 INFO ironic_understack.drivers.panos_inspect [None req-7509a38d-8bf2-4f6e-aeb6-1a26defd94d3 - - - - - -] [node:98fcc016-de95-47f8-bade-0c9b03bcb6a7] Both nodes transition to AVAILABLE requested
  {
    "extra": {
      "external_cmdb_id": 1354804,
      "serial": "026701009945",
      "mate_serial": "026701009584",
      "ha_configured": true,
      "ha_operational": true,
      "ha_peer_serial": "026701009584"
    },
    "name": "PA-026701009945-kenny",
    "provision_state": "available",
    "traits": [
      "CUSTOM_SECONDARY"
    ],
    "uuid": "7787db8b-ed9f-4713-8c13-df075903bc4f"
  },
  {
    "extra": {
      "mate_serial": "026701009945",
      "external_cmdb_id": 1354803,
      "serial": "026701009584",
      "ha_operational": true,
      "ha_peer_serial": "026701009945"
    },
    "name": "pa1410-dev-fw5",
    "provision_state": "available",
    "traits": [
      "CUSTOM_PRIMARY"
    ],
    "uuid": "98fcc016-de95-47f8-bade-0c9b03bcb6a7"
  }

What does this change do?

Upgrade impact

  • This change requires operator action to upgrade. If checked, add the
    upgrade-impact label and a release note: run scriv create from the
    repository root and describe the required action in the generated
    changelog.d/ file. See RELEASING.md.

Operator action means anything a deployment has to do beyond a normal resync:
deploy repo or values changes, new or removed secrets, enabling or disabling a
component, or a manual one-time step.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Python test results

155 tests   155 ✅  13s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 2882b77.

♻️ This comment has been updated with latest results.

@khackworth
khackworth force-pushed the PUC-2127_panos_ha_config branch from 89ee4a1 to 5851fdc Compare November 6, 2026 15:09
- Configure HA interfaces, peer-ip, group-id, and priority during inspection
- Validate HA pairing bidirectionally when both nodes ready
- Auto-transition both nodes to AVAILABLE state via Ironic RPC when validated
- Add comprehensive pre-flight and post-config validation checks
- Handle independent configuration (each device configures HA regardless of mate state)
- Fully idempotent and backward compatible (only activates with HA traits + mate_serial)

Fixes: PUC-2127
@khackworth
khackworth force-pushed the PUC-2127_panos_ha_config branch from 5851fdc to 2882b77 Compare November 6, 2026 15:19
@khackworth
khackworth marked this pull request as ready for review November 6, 2026 15:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant