Skip to content

Interfaces - #77

Merged
kpetremann merged 11 commits into
criteo:mainfrom
djothi:interfaces
Oct 20, 2026
Merged

Interfaces#77
kpetremann merged 11 commits into
criteo:mainfrom
djothi:interfaces

Conversation

@djothi

@djothi djothi commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Generate interface configuration from the CMDB

This PR adds interfaces to the generated OpenConfig device config. That covers physical and logical interfaces, IP addressing, VRF bindings and management static routes.

Depends on criteo/netbox-network-cmdb#74 for the datacenter filters

What's included

  • YANG models: adds openconfig-interfaces and openconfig-if-ip, plus a new afk-interfaces-ext module. The new module holds the SONiC settings OpenConfig has no field for: alias, lanes, index, management, IPv6 link-local only, forced routes, the remote end of each link, and the FEC_AUTO / FEC_RS FEC modes.
  • CMDB ingestion: new ingestors and models for device interfaces, logical interfaces, port layouts, links and management routes, each with unit tests. Like the other ingestors, they are filtered by datacenter wit device__site__…, and so are DHCP relays (the prefixed filter helper is removed).
  • DCIM: decodes the device type and role, which select a device's port layout.
    The role is a single Role field, read from role (NetBox 3.6 and later) or device_role (before 3.6). This also fixes the SONiC type mapping, which only read device_role.
  • Precompute: cmdb.ResolveInterfaces resolves each device's interfaces once,
    when the device is created:
    • name on the device, from the port layout;
    • neighbor;
    • logical interfaces;
    • management routes.
      CMDB data no configuration can be generated from drops only that device, with an
      error. Examples:
    • duplicate or dangling references;
    • an interface matching a port layout name that several ports share, such as
      UNUSED;
    • use_ipv6_link_local_only on a unit other than 0;
    • a management route that isn't on the management interface.
  • Convertor: a new internal/convertor/interfaces package converts the resolved
    interfaces:
    • into openconfig-interfaces, with port names and the remote end of each link;
    • management routes into static routes in the matching network instance.
  • Pipeline: the new data is fetched, precomputed per device and merged into the generated config.

Housekeeping

  • update_yang_models.sh: fixes the order of the rm -rf arguments, and passes
    -trimpath so local paths don't end up in the generated files.

@djothi
djothi requested a review from a team October 14, 2026 15:18
@djothi
djothi force-pushed the interfaces branch 2 times, most recently from ef35a0b to 4a5ce04 Compare October 15, 2026 14:24
kpetremann
kpetremann reviewed Oct 19, 2026
View reviewed changes
Comment thread internal/convertor/device/device.go Outdated
kpetremann
kpetremann reviewed Oct 19, 2026
View reviewed changes
Comment thread internal/convertor/interfaces/interfaces.go Outdated
Comment thread internal/ingestor/cmdb/query_param.go Outdated
Comment thread internal/ingestor/cmdb/port_layout.go Outdated
Comment thread internal/model/dcim/device.go Outdated
Comment thread internal/convertor/interfaces/interfaces.go Outdated
Comment thread internal/convertor/interfaces/interfaces.go Outdated
Comment thread internal/ingestor/cmdb/link.go Outdated
djothi added 11 commits October 20, 2026 12:15
@djothi
chore: fix rm argument order in update_yang_models.sh
cdf12e1 
Put the -rf flag before the operand: BSD rm (macOS) rejects options
placed after the first operand.
@djothi
feat(yang): add openconfig-interfaces, openconfig-if-ip and AFK inter…
83fb929 
…face extensions

Generate openconfig-if-ip (which pulls openconfig-interfaces,
openconfig-if-ethernet and openconfig-vlan) so that interfaces,
subinterfaces, addresses and VLANs can be emitted.

Add afk-interfaces-ext, the AFK augmentations for SONiC configuration
OpenConfig has no leaf for: management flag, use-ipv6-link-local-only,
alias, lanes, port index, forced-routes, neighbor-name/neighbor-port,
and the FEC_AUTO / FEC_RS fec-mode identities.

Bump criteo-oc-deviations to 0.2.0 to prune the openconfig-if-ip
subtrees AFK does not generate (VRRP, static neighbors, IPv6 router
advertisements).

Run the generator with -trimpath so that the generated headers no
longer embed the local Go module cache path.
@djothi
feat(dcim): expose device type and role identifiers
8590ae2 
Decode the device_type and role references of the NetBox device
inventory, which identify the port layout of a device. NetBox exposes
the role as "role" since 3.6 and as "device_role" before: both are
accepted through RoleID().
@djothi
feat(cmdb): ingest device interfaces, logical interfaces, port layout…
85c9bd9 
…s and links

Add the models and ingestors of the Network CMDB interface objects:

- device interfaces (/api/plugins/cmdb/device-interfaces/): the physical
  interfaces of a device with their Ethernet attributes;
- logical interfaces (/api/plugins/cmdb/logical-interfaces/): the L1/L2/L3
  units of a device interface with addresses, VRF and VLANs. They are
  filtered through their parent interface, hence the prefixed datacenter
  filter;
- port layouts (/api/plugins/cmdb/port-layouts/): the front-panel ports
  of a hardware model in a network role, with their NOS names and ASIC
  lanes. They are precomputed per (device type, role), not per device;
- links (/api/plugins/cmdb/links/): point-to-point links between device
  interfaces, precomputed as the neighbor table of each device with the
  remote port resolved through the remote device port layout.
@djothi
feat(interfaces): convert CMDB interfaces to openconfig-interfaces
a1a91ea 
Convert the device and logical interfaces of a device to
/interfaces/interface/:

- front-panel ports are renamed to the vendor name of their port layout
  entry and carry its alias, lanes and, for AFK managed devices, the
  port index derived from the port short name;
- the Ethernet attributes (speed, FEC, autonegotiation) are mapped to
  the OpenConfig identities, auto and rs using the afk-interfaces-ext
  ones;
- logical interfaces become subinterfaces with their IPv4/IPv6
  addresses (loopbacks exposed as host prefixes), VRF binding, 802.1Q
  mode and VLANs, and use-ipv6-link-local-only;
- the remote end of each link is emitted as neighbor-name/neighbor-port;
- eth0 is flagged as the management interface.

The network instance bindings of the subinterfaces are returned
alongside the interfaces so that the caller can attach them to their
VRF.
@djothi
feat: wire interfaces, links and management routes into the aggregati…
a222cbd 
…on pipeline

Fetch the device interfaces, logical interfaces, port layouts, links and
management routes from the CMDB alongside the existing ingestors,
precompute them per device (port layouts per hardware model and network
role, resolved per device through the inventory) and convert them in
the per-device configuration generation:

- /interfaces/interface/ is filled from InterfacesToOpenconfig;
- the subinterfaces bound to a VRF create or complete the matching
  network instance, typed as L3VRF;
- the management routing table is generated from the CMDB management
  routes: the static default route becomes a STATIC protocol instance
  of the network instance the management subinterface is bound to, with
  the subinterface as interface-ref, and forced routes are emitted on
  the IPv4 container of the management subinterface
  (afk-interfaces-ext forced-routes). Devices without management routes
  are the normal case (not SONiC).

ingestorNumber grows from 7 to 13: 5 new ingestors here, plus the NTP
one that was added without bumping it.
@djothi
chore: add Djothi Carpentier to AUTHORS
afa03be
@djothi
refactor(dcim): decode the NetBox device role into a single Role field
6910c6b 
NetBox exposes the device role as "role" since 3.6 and as "device_role"
before. Both were decoded in two fields and read through RoleID(), while
the SONiC type lookup read DeviceRole directly, so it never matched on
NetBox >= 3.6.

Decode the role once into Role in a custom UnmarshalJSON, which falls
back to "device_role" when "role" is absent, and drop the DeviceTypeID()
and RoleID() getters in favor of DeviceType.ID and Role.ID.
@djothi
refactor(interfaces): resolve the interfaces of a device once, at dev…
62b6c32 
…ice creation

The interface convertors received the raw CMDB device interfaces, logical
interfaces, management routes, port layout and neighbors, and each of
them resolved the names and checked the references again. An invalid
interface was only detected while generating the configurations, which
fails the whole build.

Resolve them once per device in cmdb.ResolveInterfaces, called from
NewDevice: each device interface becomes an Interface carrying its name
on the device, its port layout entry, its neighbor, its logical
interfaces and its management routes. The CMDB data no configuration
can be generated from is rejected there, which only drops the device:

- duplicate device or logical interfaces, interfaces sharing a port, or
  an interface matching several ports of the port layout;
- logical interfaces without index or referencing an unknown parent,
  and use_ipv6_link_local_only set on a unit other than 0;
- management routes of an unsupported kind, not attached to an IPv4
  logical interface of the management interface.

The convertors then build each interface, subinterface and static
route in one place from this model.

A logical name shared by several ports of a port layout (e.g. the
UNUSED placeholder) is kept as an ambiguous entry instead of the last
port: only the devices whose interfaces use it fail. The neighbor port
is named through PortLayoutTable.PortName, the vendor name of the
remote port, or its CMDB name when it has no port layout entry.

No change in the generated configurations.
@djothi
feat(cmdb): filter links, logical interfaces and management routes by…
4cc7490 
… datacenter

The links were fetched for every datacenter, as their endpoint had no
datacenter filter. The logical interfaces and the management routes were
filtered through the path to their device (e.g.
parent_interface__device__site__name), hence a prefixed datacenter
filter.

netbox-network-cmdb now accepts the device location filters under their
usual names (device__site__name, ...) on these endpoints, a link
matching when one of its ends does. Use the device datacenter filter for
the three of them, like the other ingestors.

A CMDB without these filters ignores them and returns every
datacenter: the generated configurations stay the same, the devices of
other datacenters being dropped, only the fetched volume grows.
@djothi
refactor(dhcp-relay): filter the DHCP relays by datacenter like the o…
00f3229 
…ther ingestors

The DHCP relays were filtered through the path to their device
(logical_interface__parent_interface__device__site__name), the last
user of datacenterFilterWithPrefix.

netbox-network-cmdb now accepts the device location filters under their
usual names on the DHCP relay endpoint too: use the device datacenter
filter and drop datacenterFilterWithPrefix.

A CMDB without these filters ignores them and returns the DHCP relays of
every datacenter: the generated configurations stay the same.
@djothi djothi mentioned this pull request Oct 20, 2026
Merged
@kpetremann
kpetremann merged commit 2d25f47 into criteo:main Oct 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kpetremann kpetremann kpetremann left review comments
@arn-alt arn-alt arn-alt approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@djothi @kpetremann @arn-alt