Skip to content

[dpd,dpd-api,dpd-types] make external multicast constraints self-enforcing - #396

Open
zeeshanlakhani wants to merge 10 commits into
zl/mcast-inner-ttl-guardfrom
zl/mcast-type-validation
Open

[dpd,dpd-api,dpd-types] make external multicast constraints self-enforcing#396
zeeshanlakhani wants to merge 10 commits into
zl/mcast-inner-ttl-guardfrom
zl/mcast-type-validation

Conversation

@zeeshanlakhani

@zeeshanlakhani zeeshanlakhani commented Oct 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

This began as a normal, ol' doc fix (I've kept the history here, including #364, which has been closed). The external multicast group API endpoints had documented scope that didn't match the handlers. Group and source validation already ran in the handlers, but this moves the checks into the request types themselves, at the deserialization boundary, and revisits the address rules for consistency. This includes relaxing the IPv6 scope rules from the earlier validation branch, preserving flexibility vs forcing some kind of extra Oxide-driven policy.

Then, this review question came along:

Could these documented constraints also be made self-enforcing via newtypes? Your decision whether doing so is worth the API churn and code complexity, but imo it's worth considering.

My response to this grew into newtypes for group addresses, sources, and NAT targets; plus, ASM/SSM request variants. The rollback work
and cleanup account for much of this diff.

Creation requests now check group/source constraints during
deserialization. Updates validate addresses at parse time; source family and SSM validations still run against the group address within the handler(s).

Group addresses (typed as ExternalMulticastIp):

Group addr / condition Outcome Spec / qualification
Not multicast Reject Group addresses must be multicast.
224.0.0.0 Reject RFC 1112 §4: "guaranteed not to be assigned to any group", not a MUST NOT; rejected because it can never name a group.
232.0.0.0 Reject RFC 4607 §4.3: SSM null address.
232.0.0.1–232.0.0.255 Accept RFC 4607 §4.3: reserved for IANA allocation.
232/8 SSM RFC 4607 §1.
ff0x::, with all-zero group ID Reject RFC 4291 §2.7.1: "reserved and shall never be assigned to any multicast group"; rejected on the same reasoning as 224.0.0.0.
IPv6 scope 0 Reject RFC 4291 §2.7: reserved; packets must be discarded.
IPv6 scopes 1, 2 Reject RFC 4291 §2.7: interface-local and link-local traffic cannot be forwarded past those scopes.
IPv6 scopes 3–f Accept, subject to other checks RFC 4291 §2.7; RFC 7346. Scope f is reserved; received packets are treated as global scope.
ff04::/64 Reject Oxide underlay reservation.
Rest of ff04::/16 Accept, subject to other checks Outside the reserved /64.
Flags ~ R=0, P=T=1, plen=0, either X value SSM RFC 4607 §1; RFC 7371 §4.1.2: ff3x and ffbx flag patterns; reserved fields ignored.
IPv6 SSM with nonzero network prefix Accept RFC 4607 §1: a system should treat all of ff3x::/32 as SSM. RFC 3306 §6 requires a zero prefix only of the node that's forming the address.
ff3x::4000:0 Reject RFC 4607 §4.3: reserved SSM null address.
IPv6 SSM group IDs 0–3fff:ffff Accept RFC 4607 §1: invalid as SSM addresses (assigned with P=T=0, RFC 3307 §4.1), but a router may drop them, so we don't reject them at the API.
IPv6 SSM group IDs 4000:1–7fff:ffff Accept RFC 3307 §4.2; RFC 4607 §4.3: IANA allocation range.
IPv6 SSM group IDs 8000:0–ffff:ffff Accept RFC 3307 §4.3.1, §4.3.2 and RFC 10028 §3: dynamic range where host SSM allocation is f000:0–fcff:ffff.
Flags ~ P=1, T=0 Reject RFC 7371 §4.1.1: P requires T.
Flags ~ R=1 without both P and T Reject RFC 7371 §4.2.2: R requires P and T.
Flags ~ R=1 with plen outside range 1–64 Reject RFC 3956 §4: embedded-Rendezvous Point (RP) prefix-length bounds.
Flags ~ R=1 with RIID=0 (RP Interface ID) Reject RFC 3956 §6.3: zero RIID is excluded.
Flags ~ R=1 with derived RP in ::/16, fe80::/10, or ff00::/8 Reject RFC 3956 §4: unusable embedded RP address.
Prefix-based ff2 bits Ignored RFC 7371 §4.1.1, §4.2.1: reserved bits ignored on receipt.
Prefix-based, R=0: rsvd bits Ignored RFC 3956 §3: reserved bits ignored on receipt.
Flags ~ R=1: low rsvd nibble becomes RIID Checked RFC 3956 §3, §6.3: reject RIID=0.
IANA assignment or an embedded-prefix scope comparison Not checked No registry lookup or scope comparison with the embedded prefix.

Exact sources (typed as ExactSource) and request constraints:

Exact source / condition Outcome Spec / qualification
Multicast source Reject RFC 1112 §6.2; RFC 4291 §2.7: multicast addresses cannot be packet sources.
0.0.0.0, :: Reject RFC 1812 §5.3.7 and RFC 6890 for 0.0.0.0, as with 0.0.0.0/8 specified below; RFC 4291 §2.5.2: an unspecified IPv6 source is never forwarded.
127/8, ::1 Reject RFC 1122 §3.2.1.3; RFC 4291 §2.5.3: loopback sources cannot be forwarded outside the host.
255.255.255.255 Reject RFC 919 §7; RFC 1122 §3.2.1.3: broadcast addresses cannot be source.
169.254/16, fe80::/10 Reject RFC 3927 §2.7; RFC 4291 §2.5.6: link-local traffic cannot be forwarded off-link.
0.0.0.0/8 Reject RFC 1122 §3.2.1.3 permits initialization use; RFC 1812 §5.3.7 recommends against forwarding these sources.
240/4 Reject RFC 1112 §4: reserved Class E space; the special-purpose registry marks Source=False (RFC 6890 §2.2.2).
::ffff:0:0/96, ::/96 Reject RFC 6890 §2.2.3 marks the mapped form Source=False; RFC 4291 §2.5.5.1 deprecates the compatible form. This matches OPTE.
100.64/10 Accept RFC 6598 §4: shared address space is usable within an operator's network.
Source family differs from group family Reject Same-family (S,G) check; matches OPTE.
SSM create or supplied update source list: no exact source, or Any Reject RFC 4607 §2: SSM requires a source-specific channel.

Includes:

  • Exact-source rejection set and same-family checks match OPTE Include subscriptions.
  • In the new API version, omitted sources on the update/modification preserve the source filter, while the older endpoints convert this omission to Any (wildcard), which SSM will reject.
  • External groups use caller-supplied addresses. No IANA-assignment lookup or embedded-prefix scope check occurs there.
  • Regarding rollback, DpdError::Unwind now collects multiple rollback failures. HTTP errors preserve the initial status and use a special ROLLBACK_FAILURE_ERROR_CODE in propagation. Restoration can re-add missing bitmap, replication, and route entries.
  • chaos-feature tests inject ASIC failures and check usage counts after rollback, including failures during the rollback op itself.
  • dpd-client now maps ExternalMulticastIp and UnderlayMulticastIpv6 to the dpd-types newtypes (via progenitor replace), allowing clients to reject bad addresses when parsing, before a request goes out.

@zeeshanlakhani
[dpd, mcast] Tighten multicast group and source validation
7c765d7 
This work constrains group and SSM source validation based on a variety
of specs, linked throughout the PR. Previously, DPD was accepting groups and
(S, G) sources that no packet could actually exercise, including ones
that can never leave a host/link or, worse, sources a switch can't forward.

Here's the list (the first classifier one was the most obvious):

| address                        | verdict | spec                          |
|--------------------------------|---------|-------------------------------|
| ff3x::/32 in ff30::/12 only    | is SSM  | RFC 4607 §1; RFC 3306 is ASM  |
| ff3x:: group ID > 0x40000000   | accept  | RFC 4607 §4.3                 |
| v6 scopes other than 4/5/8/e   | reject  | RFC 7346; no Ethernet realm   |
| class E (240.0.0.0/4) source   | reject  | RFC 6890 "Source: False"      |
| 0.0.0.0/8 source               | reject  | RFC 6890 "Forwardable: False" |
| 100.64.0.0/10 source           | accept  | RFC 6598, operator-internal   |
| source/group family mismatch   | reject  | (local choice)                |
| 232.0.0.0 (SSM null)           | reject  | RFC 4607 §4.3                 |
| v4-mapped/compat v6 source     | reject  | RFC 4291 §2.5.5, aliases IPv4 |
@zeeshanlakhani
[dpd-api] correct external multicast group scope docs
aeb2d6c 
The external-group docs say “IPv4 and non-admin-local IPv6 multicast traffic,”
but external groups can use admin-local IPv6 too. The exception,
however, is the `ff04::/64` underlay range, which is reserved for the
underlay mcast group mapped from the external mcast group. This makes
the information explicit.

The old wording is in a blessed spec, so this needs a new API version:
`MCAST_EXTERNAL_SCOPE_DOCS`.
@zeeshanlakhani
[dpd,dpd-api,dpd-types] make external multicast constraints self-enfo…
aa4087d 
…rcing

This began as a normal, ol' doc fix. The external multicast group API endpoints
had documented scope that didn't match the handlers. Group and source validation
already ran in the handlers, but this moves the checks into the request
types themselves, at the deserialization boundary, and revisits the
address rules for consistency. This includes relaxing the IPv6 scope
rules from the earlier validation branch, preserving flexibility vs forcing
some kind of extra Oxide-driven policy.

Then, this review question came along:

> Could these documented constraints also be made self-enforcing via
> newtypes? Your decision whether doing so is worth the API churn and
> code complexity, but imo it's worth considering.

My response to this grew into newtypes for group addresses, sources, and
NAT targets; plus, ASM/SSM request variants. The rollback work
and cleanup account for much of this diff.

Creation requests now check group/source constraints during
deserialization. Updates validate addresses at parse time; source-family
and SSM validations still run against the group address within the handler(s).

Group addresses (typed as `ExternalMulticastIp`):

| Group addr / condition | Outcome | Spec / qualification |
| --- | --- | --- |
| Not multicast | Reject | Group addresses must be multicast. |
| `224.0.0.0` | Reject | RFC 1112 §4: "guaranteed not to be assigned to any group", not a MUST NOT; rejected because it can never name a group. |
| `232.0.0.0` | Reject | RFC 4607 §4.3: SSM null address. |
| `232.0.0.1`–`232.0.0.255` | Accept | RFC 4607 §4.3: reserved for IANA allocation. |
| `232/8` | SSM | RFC 4607 §1. |
| `ff0x::`, with all-zero group ID | Reject | RFC 4291 §2.7.1: "reserved and shall never be assigned to any multicast group"; rejected on the same reasoning as `224.0.0.0`. |
| IPv6 scope `0` | Reject | RFC 4291 §2.7: reserved; packets must be discarded. |
| IPv6 scopes `1`, `2` | Reject | RFC 4291 §2.7: interface-local and link-local traffic cannot be forwarded past those scopes. |
| IPv6 scopes `3`–`f` | Accept, subject to other checks | RFC 4291 §2.7; RFC 7346. Scope `f` is reserved; received packets are treated as global scope. |
| `ff04::/64` | Reject | Oxide underlay reservation. |
| Rest of `ff04::/16` | Accept, subject to other checks | Outside the reserved `/64`. |
| Flags ~ `R=0`, `P=T=1`, `plen=0`, either `X` value | SSM | RFC 4607 §1; RFC 7371 §4.1.2: `ff3x` and `ffbx` flag patterns; reserved fields ignored. |
| IPv6 SSM with nonzero network prefix | Accept | RFC 4607 §1: a system should treat all of `ff3x::/32` as SSM. RFC 3306 §6 requires a zero prefix only of the node that's forming the address. |
| `ff3x::4000:0` | Reject | RFC 4607 §4.3: reserved SSM null address. |
| IPv6 SSM group IDs `0`–`3fff:ffff` | Accept | RFC 4607 §1: invalid as SSM addresses (assigned with `P=T=0`, RFC 3307 §4.1), but a router *may* drop them, so we don't reject them at the API. |
| IPv6 SSM group IDs `4000:1`–`7fff:ffff` | Accept | RFC 3307 §4.2; RFC 4607 §4.3: IANA allocation range. |
| IPv6 SSM group IDs `8000:0`–`ffff:ffff` | Accept | RFC 3307 §4.3.1, §4.3.2 and RFC 10028 §3: dynamic range where host SSM allocation is `f000:0`–`fcff:ffff`. |
| Flags ~ `P=1`, `T=0` | Reject | RFC 7371 §4.1.1: `P` requires `T`. |
| Flags ~ `R=1` without both `P` and `T` | Reject | RFC 7371 §4.2.2: `R` requires `P` and `T`. |
| Flags ~ `R=1` with `plen` outside range `1`–`64` | Reject | RFC 3956 §4: embedded-Rendezvous Point (RP) prefix-length bounds. |
| Flags ~ `R=1` with `RIID=0` (RP Interface ID) | Reject | RFC 3956 §6.3: zero RIID is excluded. |
| Flags ~ `R=1` with derived RP in `::/16`, `fe80::/10`, or `ff00::/8` | Reject | RFC 3956 §4: unusable embedded RP address. |
| Prefix-based `ff2` bits | Ignored | RFC 7371 §4.1.1, §4.2.1: reserved bits ignored on receipt. |
| Prefix-based, `R=0`: `rsvd` bits | Ignored | RFC 3956 §3: reserved bits ignored on receipt. |
| Flags ~ `R=1`: low `rsvd` nibble becomes `RIID` | Checked | RFC 7371 §4.2.2; RFC 3956 §6.3: reject `RIID=0`. |
| IANA assignment or an embedded-prefix scope comparison | Not checked | No registry lookup or scope comparison with the embedded prefix. |

Exact sources (typed as `ExactSource`) and request constraints:

| Exact source / condition | Outcome | Spec / qualification |
| --- | --- | --- |
| Multicast source | Reject | RFC 1112 §6.2; RFC 4291 §2.7: multicast addresses cannot be packet sources. |
| `0.0.0.0`, `::` | Reject | RFC 1812 §5.3.7 and RFC 6890 for `0.0.0.0`, as with `0.0.0.0/8` specified below; RFC 4291 §2.5.2: an unspecified IPv6 source is never forwarded. |
| `127/8`, `::1` | Reject | RFC 1122 §3.2.1.3; RFC 4291 §2.5.3: loopback sources cannot be forwarded outside the host. |
| `255.255.255.255` | Reject | RFC 1122 §3.2.1.3: broadcast addresses cannot be sources. |
| `169.254/16`, `fe80::/10` | Reject | RFC 3927 §2.7; RFC 4291 §2.5.6: link-local traffic cannot be forwarded off-link. |
| `0.0.0.0/8` | Reject | RFC 1122 §3.2.1.3 permits initialization use; RFC 1812 §5.3.7 recommends against forwarding these sources. |
| `240/4` | Reject | RFC 1112 §4: reserved Class E space; the special-purpose registry marks `Source=False` (RFC 6890 §2.2.2). |
| `::ffff:0:0/96`, `::/96` | Reject | RFC 6890 §2.2.3 marks the mapped form `Source=False`; RFC 4291 §2.5.5.1 deprecates the compatible form. This matches OPTE. |
| `100.64/10` | Accept | RFC 6598 §4: shared address space is usable within an operator's network. |
| Source family differs from group family | Reject | Same-family `(S,G)` check; matches OPTE. |
| SSM create or supplied update source list: no exact source, or `Any` | Reject | RFC 4607 §2: SSM requires a source-specific channel. |

Includes:

- Exact-source rejection set and same-family checks match OPTE `Include`
  subscriptions.
- In the new API version, omitted sources on the update/modification
  preserve the source filter, while the older endpoints convert
  this omission to `Any` (wildcard), which SSM will reject.
- External groups use caller-supplied addresses. No IANA-assignment
  lookup or embedded-prefix scope check occurs there.
- Regarding rollback, `DpdError::Unwind` now collects multiple rollback
  failures. HTTP errors preserve the initial status and use a special
  `ROLLBACK_FAILURE_ERROR_CODE` in propagation. Restoration can re-add
  missing bitmap, replication, and route entries.
- `chaos`-feature tests inject ASIC failures and check usage counts
  after rollback, including failures during the rollback op itself.
- `dpd-client` now maps `ExternalMulticastIp` and
  `UnderlayMulticastIpv6` to the `dpd-types` newtypes (via progenitor
  `replace`), allowing clients to reject bad addresses when parsing, before a
  request goes out.
@zeeshanlakhani zeeshanlakhani self-assigned this Oct 7, 2026
@zeeshanlakhani
zeeshanlakhani added this pull request to stack #397 October 7, 2026 10:17
@zeeshanlakhani zeeshanlakhani added the multicast Multicast-related feature/impl label Oct 7, 2026
This was referenced Oct 8, 2026
Open
Merged
zeeshanlakhani added a commit to oxidecomputer/thundermuffin that referenced this pull request Oct 8, 2026
@zeeshanlakhani
[post-review, ssm] follow dendrite's validation + IfAddr and comment …
f8413dc 
…cleanup

Our SSM validation now matches oxidecomputer/dendrite#396, which is the
updated read on the RFCs for both group and source checking.

We move the interface lookups onto `IfAddrs`/`IfAddr`, and add tests
for source parsing and the interface-name handling.

Also, we do a full run over the documentation to make this match up with
the current stack of PRs going into prod.
zeeshanlakhani added a commit to oxidecomputer/thundermuffin that referenced this pull request Oct 8, 2026
@zeeshanlakhani
[multicast, smf] validate SSM joins and rework the joiner service (#14)
98bb2f5 
We (now) reject a source-less join on an address classified as SSM at CLI validation, and require `--multicast-iface` alongside `--multicast-source` for an SSM join to always pin to a specific interface. Classification uses 232.0.0.0/8 for IPv4, and `ff3x::/32` or `ffbx::/32` with a zero prefix length for IPv6 (RFC 4607 §1, RFC 7371 §4.1.2).

We also check SSM destinations the way dendrite does as of oxidecomputer/dendrite#396. Only the SSM null addresses (232.0.0.0, ff3x::4000:0, RFC 4607 §4.3) and IPv6 scopes 0-2 get rejected (dendrite loosened its earlier, stricter rules, for more flexibility and a closer RFC read). `--multicast-source` has to be an address that can actually send forwarded traffic (e.g., no broadcast, link-local, 0/8, 240/4, or v4-mapped/compatible v6). Otherwise, the join would just sit there receiving nothing, and that looks like a network regression.

The SMF manifest's start method had pointed at `mgd_method_script.sh`, a copy-paste path nothing installs to, so the service flapped between incomplete and running. It now points at `method_script.sh` (matching `package-manifest.toml`), and the script is reworked to spawn one joiner per configured multicast group (SSM encoded as `group@src1,src2`). Multicast joins are pinned through a single `multicast_iface` property. The binary accepts an interface name or bound IP address and resolves it to the IPv4 address or IPv6 interface index required for each join. Interface- and link-local IPv6 groups also use this index as their bind scope. `--scope` is now solely for unicast, and the `ipv6_scope` property has been removed.

The service idles when no groups are configured. If any joiner exits, the service restarts through the `ctrun` contract and every membership is re-joined.
@zeeshanlakhani
zeeshanlakhani removed this pull request from stack #397 October 13, 2026 13:58
@zeeshanlakhani
zeeshanlakhani changed the base branch from zl/mcast-p4-counters to zl/mcast-inner-ttl-guard October 13, 2026 13:58
@zeeshanlakhani
zeeshanlakhani added this pull request to stack #401 October 13, 2026 13:59
@zeeshanlakhani
zeeshanlakhani force-pushed the zl/mcast-type-validation branch from 9af13a9 to dff8591 Compare October 19, 2026 17:32
@zeeshanlakhani
[types, err] typed embedded rendezvous-point errors
6366b61 
This also includes `PartialEq` and `Eq` derives on multicast
error enums, allowing tests to use `assert_eq!` on the full error.
@zeeshanlakhani
zeeshanlakhani force-pushed the zl/mcast-type-validation branch from dff8591 to 6366b61 Compare October 19, 2026 17:41
@zeeshanlakhani zeeshanlakhani mentioned this pull request Oct 20, 2026
Open
zeeshanlakhani added a commit to oxidecomputer/maghemite that referenced this pull request Oct 20, 2026
@zeeshanlakhani
[mcast, pt1] multicast address, source, and route key types
beaa7a3 
This is the first of a series of recut PRs split from
#675/#696. Validated address types now live in client-common (`Vni`,
`OverlayMulticast`, `UnderlayMulticastIpv6`). We'll eventually get to
the great big shared oxnet typed world :)!

MRIB route types in mg-api-types-versions point toward the upcoming v13.
While the API version itself is not declared here, we'll be
working with this version throughout the upcoming split set.

Group validation is essentially ported from dendrite
oxidecomputer/dendrite#396: RFC-based (and flexible) ruleset(s) plus the
internal, Oxide-specific `ff04::/64` underlay reservation.

Proptests can be found in `impls/mrib/` and import `crate::latest::mrib`.
The proptest suite is differential: `spec_*` fns encode the RFC
rules and the Oxide underlay reservation, and the properties check that
the constructors agree, error variant(s) included.

Includes:

- Fixed: in test-proptest.sh, the `rdb`/`bgp` blocks copied regressions to
  paths that the output rule never could match, and copies were skipped
  on failures as well; the output rule is now `**/*` over per-crate subdirs.
- Running client-common unit tests in CI.
zeeshanlakhani added a commit to oxidecomputer/maghemite that referenced this pull request Oct 20, 2026
@zeeshanlakhani
[mcast, pt1] multicast address, source, and route key types
2005d61 
This is the first of a series of recut PRs split from
#675/#696. Validated address types now live in client-common (`Vni`,
`OverlayMulticast`, `UnderlayMulticastIpv6`). We'll eventually get to
the great big shared oxnet typed world :)!

MRIB route types in mg-api-types-versions point toward the upcoming v13.
While the API version itself is not declared here, we'll be
working with this version throughout the upcoming split set.

Group validation is essentially ported from dendrite
oxidecomputer/dendrite#396: RFC-based (and flexible) ruleset(s) plus the
internal, Oxide-specific `ff04::/64` underlay reservation.

Proptests can be found in `impls/mrib/` and import `crate::latest::mrib`.
The proptest suite is differential: `spec_*` fns encode the RFC
rules and the Oxide underlay reservation, and the properties check that
the constructors agree, error variant(s) included.

Includes:

- Fixed: in test-proptest.sh, the `rdb`/`bgp` blocks copied regressions to
  paths that the output rule never could match, and copies were skipped
  on failures as well; the output rule is now `**/*` over per-crate subdirs.
- Running client-common unit tests in CI.
@zeeshanlakhani
zeeshanlakhani removed this pull request from stack #401 October 21, 2026 01:55
@zeeshanlakhani
zeeshanlakhani added this pull request to stack #410 October 21, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cfzimmerman cfzimmerman Awaiting requested review from cfzimmerman
@FelixMcFelix FelixMcFelix Awaiting requested review from FelixMcFelix
@taspelund taspelund Awaiting requested review from taspelund

Assignees

@zeeshanlakhani zeeshanlakhani

Labels

multicast Multicast-related feature/impl

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@zeeshanlakhani