[t3hk0d3]

Spike for #84, not for merge. Exploration of TREL-like satellite routers: Ziggurat nodes linked over an IP backbone ("ZLOE", Zigbee Link Over Ethernet), configured out-of-band from the coordinator. Opened as a draft to share the code and findings.

What it does

• ziggurat-phy-trel: a RadioPhy wrapper that adds the backbone as a second link. 802.15.4 frames travel in UDP datagrams; unicasts to backbone peers go over UDP with ack/retry (falling back to the air), broadcasts go over the air and the backbone. No NWK changes: backbone links show up as ordinary neighbours with low link cost.
• Discovery over mDNS/DNS-SD (_zloe._udp, TXT xa/xp as in TREL, plus nwk, role, mc/mc4), scoped by extended PAN ID. Dual-stack socket, IPv6 link-local preferred, IPv4 supported.
• --zloe-satellite: a node finds the coordinator on the (trusted) backbone, fetches the network settings over ZLOE and starts in router role. No joining, no persistence.
• Broadcasts by IPv6/IPv4 multicast with a liveness probe; peers multicast does not reach (or that opt out with --zloe-mcast-port 0, e.g. Wi-Fi) get unicast copies.
• Bench tooling: --test-network (coordinator without ZHA), zloe-ctl (permit joins, On/Off, Mgmt_Lqi), --deny (fake out-of-range), multicast/IPv4-only test hooks.

Findings (ESP32-C6 OpenThread RCPs, two esp-zigbee-lib routers)

• With the coordinator unable to hear a router over the air, traffic reaches it through the satellite: 10/10 On/Off, 42–74 ms to APS ack (direct air path: 40–68 ms).
• Third-party (ZBOSS) routers list the coordinator and the satellite as ordinary sibling routers; nothing unusual in their Link Status handling.
• Multicast fallback and recovery work per peer and per address family.
• Details: crates/ziggurat-phy-trel/SPIKE.md and crates/ziggurat-phy-trel/BROADCAST.md.

Not upstream material as-is

• Contains a workaround for Router devices take 20-60 s to join, or fail to join #88 (direct delivery to router children); the real fix is in Directly contact new router children for two link status periods #89.
• --test-network answers Node_Desc_req itself (zigpy's job normally).
• Satellites cannot accept joins and have no children; backbone is unauthenticated by design.

ZLOE link degradation (TREL-like)

NWK does not see two links. ZLOE folds the backbone's health into the per-frame LQI, so a degrading backbone shows up as rising link cost, and Zigbee routes away from it on its own.

How a backbone peer's link degrades and recovers:

1. Per-frame ack. Every unicast datagram asks for a ZLOE ack: 50 ms timeout, 3 attempts. If none comes, the frame goes out over the air instead, so NWK sees the radio's result, not a lost frame.
2. Link health. Each attempt updates the peer's health: an exponential average of ack success, newest sample weighted 1/8. LQI = health × 255.
3. Health to NWK cost. Frames received from the peer over the backbone carry that LQI. NWK keeps the median of the last 3 samples and maps it to link cost: 193–255 → 1, 129–192 → 2, …, ≤ 16 → 7. That cost goes into our Link Status, so route discovery starts preferring other paths as the backbone degrades.
4. Address rotation. When all 3 attempts to an address fail, the peer moves to its next mDNS-advertised address. This covers wrong link-local scopes and interfaces that went away.
5. Fallback to radio. A peer counts as live on the backbone while it has sent us a datagram in the last 35 s.
   • While it's live, copies of its frames heard over the air are dropped, because the same frames arrive over the backbone. Otherwise its radio LQI would mix into its backbone LQI.
   • Once it goes quiet, its air frames are delivered with their real radio LQI, and the neighbour entry moves to the radio link.
6. Ageing. If the peer is heard on neither link, NWK ages it out after 3 missed Link Status rounds (45 s), as for any router.
7. Broadcasts. Each peer's multicast reachability is probed every 5 s on its preferred group, with the other group added while that one is unconfirmed. A peer with no group confirmed in the last 12 s gets unicast copies, which go through steps 1–4.

Compared with TREL (OpenThread multi-radio):

	TREL	ZLOE (spike)
Per-link state	Per neighbour and per radio: supported links plus a preference, adjusted on TX success, TX failure and RX	Per peer: one backbone health value plus liveness (last datagram)
Link choice	Radio selector picks the preferred link per frame and may probe the other	Backbone first whenever the peer is known; radio only after the backbone fails for that frame
Duplicates across links	MAC sequence-number check across radios	Air copies dropped while the backbone is live; backbone retransmits dropped by packet number
What the network layer sees	Thread link quality per neighbour	Zigbee LQI/link cost per neighbour, derived from backbone health
Acks	TREL ack packets	ZLOE ack datagrams (same idea)

Gaps:

• Health only reflects our own sends. A peer we rarely unicast to keeps its last value, and without traffic health never recovers. Probe acks could feed it.
• One direction stands in for both. Our TX success to the peer sets the LQI of frames we receive from it.
• No per-frame switching or hysteresis. A peer whose backbone is alive but lossy keeps getting backbone-first sends, each paying up to 150 ms of retries before falling back to the air. TREL's selector would demote the link instead. The fix: prefer the radio while health is below a threshold.
• Air copies are dropped even when the backbone is degraded. The radio may be the better link at that moment.
• Radio LQI is invisible while the backbone is live. Routing can't compare the two links of a neighbour, so it can't choose radio over a degraded backbone.

[t3hk0d3]

Benchmark (bench run 2026-11-10)

Bench: one Linux host, backbone on a wired NIC (--zloe-interface, IPv6 link-local). Three ESP32-C6 boards on stock OpenThread RCP (UART, 460800, no flow control): coordinator plus two satellites. One more ESP32-C6 runs an esp-zigbee-lib 2.1 (ZBOSS) on/off-light router as the third-party device. Test network on channel 25, driven without ZHA (--test-network + zloe-ctl).

1. Backbone datagrams per broadcast (3 backbone nodes, 120 s window)

Transport	NWK broadcasts	Data datagrams	Per broadcast	Probes
Multicast (default)	38	38	1.00	144
Unicast only (--zloe-mcast-port 0)	38	76	2.00 (N−1)	0

The backbone mesh stayed at link cost 1/1 in both modes.

Caveat: at this traffic level (mostly Link Status and many-to-one route requests, about 0.1 broadcasts/s per node), the probes cost more than multicast saves. That's 144 probes plus about 288 unicast acks, against 38 data datagrams saved. Each node probes each announced group (IPv6 and IPv4) every 5 s. Options before this is worth it on small networks:

• Probe only the preferred group.
• Lengthen the probe interval.
• Set the ack-request flag on an occasional real broadcast instead of sending separate probes.

2. Coordinator outage (two satellites, 60 s)

• During the outage, both satellites kept each other at cost 1/1; only the coordinator aged out of their neighbour tables.
• Within 50 s of the coordinator restarting, all backbone links were back at cost 1/1.

3. Out-of-range device through the backbone

The coordinator is told to ignore one ZBOSS router over the air (--deny), so On/Off to that router has to go through the backbone. 10 toggles each, latency to the APS ack:

Setup	Delivered	Latency (min–max, median)
Direct over the air (no deny), 2 backbone nodes	10/10	40–68 ms, ~41 ms
Via satellite, 2 backbone nodes, unicast broadcasts	10/10	42–74 ms, ~55 ms
Via satellite, IPv6 link-local backbone	4/4	41–64 ms
Via satellite, 3 backbone nodes, multicast	10/10	40–126 ms, ~60 ms

• The coordinator source-routes through the satellite, and the satellite relays over the air.
• Mgmt_Lqi from the ZBOSS routers lists the coordinator and the satellites as ordinary sibling routers.

4. Multicast failover

Case	Result
Multicast to one peer blocked	Peer switched to unicast copies after 15 s; back on multicast 10 s after unblocking
Only IPv4 blocked (IPv4 preferred)	Peer stayed on multicast through the IPv6 group, no unicast copies
IPv4-only satellite	Bootstraps over IPv4, reached through the IPv4 group, cost 1/1

Detection takes 12–17 s, inside the 45 s neighbour ageing.

Issues found on the way

• Router devices take 20-60 s to join, or fail to join #88: worked around for the bench.
• No Node_Desc_req answer without zigpy: ZBOSS joiners leave. --test-network answers it.
• Radius 0 from the host: it's clamped to 1, so the frame is dropped at the first relay.
• Rejoining under a new NWK address: a board that rejoined as a satellite kept its old address in the other nodes' neighbour tables, which are keyed by EUI64. Satellites now send Device_annce on start, as routers do after joining.

[puddly]

Nice! Mapping 802.15.4 straight to UDP datagrams I think is the most performant option, it reuses the existing retry machinery in the upper layers and completely sidesteps the issues caused by TCP (and mostly resolved by QUIC). This is really awesome!

I'll try this out with my home network some time this week to see if the unicast/multicast split becomes more noticeable with 100+ devices and group commands.

---

The part that worries me about encapsulation alone is that it strips away the one layer keeping Zigbee's weak security from being more of a problem: isolation. If Zigbee and IP are bridged via a satellite that allows unauthenticated traffic, you basically move the exploitation boundary. Before, it was physically present attackers with an 802.15.4 radio or a misconfigured HA server on the local network. Now, anything on the local network can inject packets straight into the Zigbee network. There are many insecure installations out there (Z2M hardcoded a single network key up until a few years ago, deCONZ used insecure PRNGs and made their network key derivable from the PAN ID for some installations, etc.).

I wonder what could be done for encryption and authentication? Using the network key as a shared secret and rejecting weak keys might be a reasonable compromise, as the alternative would be some sort of key exchange, sessions, etc.

[t3hk0d3]

@puddly

The part that worries me about encapsulation alone is that it strips away the one layer keeping Zigbee's weak security from being more of a problem: isolation. If Zigbee and IP are bridged via a satellite that allows unauthenticated traffic, you basically move the exploitation boundary. Before, it was physically present attackers with an 802.15.4 radio or a misconfigured HA server on the local network. Now, anything on the local network can inject packets straight into the Zigbee network. There are many insecure installations out there (Z2M hardcoded a single network key up until a few years ago, deCONZ used insecure PRNGs and made their network key derivable from the PAN ID for some installations, etc.).

I wonder what could be done for encryption and authentication? Using the network key as a shared secret and rejecting weak keys might be a reasonable compromise, as the alternative would be some sort of key exchange, sessions, etc.

Yeah, we can definitively add it. I've just skipped it for PoC.
Just encrypted NWK frames are not enough.
Maybe we can use QUIC-like protocol with certificate-based authentication, with coordinator as certificate authority.
Only problem there is multicast-based broadcasts (there is only IETF draft for QUIC-multicasts).

[t3hk0d3]

Real ZHA + Home Assistant: topology scan

Bench:

• Home Assistant 2026.11.1 in Docker, with ZHA connected to the coordinator over the Ziggurat WebSocket API. ZHA formed a fresh network on channel 20.
• Both satellites were started with --zloe-satellite only, with no other configuration. They found the coordinator over mDNS and fetched ZHA's network settings over ZLOE.
• An ESP32-C6 running esp-zigbee-lib (ZBOSS) paired through ZHA's permit-join and was interviewed as ESPRESSIF otbr_test_dev.

ZHA topology scan, after the fixes below:

Device	Neighbours reported
Coordinator 0x0000	satellite 0xEF2F LQI 255, satellite 0x4965 LQI 255
Satellite 0xEF2F	coordinator (Coordinator, depth 0) LQI 255, satellite 0x4965 LQI 255, ZBOSS router 0x93E4 LQI 11; route to 0x0000 active
Satellite 0x4965	coordinator LQI 255, satellite 0xEF2F LQI 255, ZBOSS router LQI 11; route to 0x0000 active
ZBOSS router 0x93E4	coordinator LQI 88, satellites LQI 65 and 49, all as siblings

LQI 255 is a backbone link. The low values are radio links: ESP RCPs report low LQI even for boards a few centimetres apart.

ZHA's network visualization shows all four nodes. It draws only each node's link to the coordinator, not the satellite-to-satellite or satellite-to-router links, although they are in the scanned neighbour data.

Problems found and fixed on the spike branch:

1. Satellites never appeared in ZHA. zigpy starts interviewing a satellite after its Device_annce, but a satellite has no client to answer ZDP, so the interview never finished. Satellites now answer Node_Desc_req (router descriptor) and Active_EP_req (no endpoints) themselves. They show up as unk_manufacturer unk_model routers.
2. The coordinator's own tables couldn't be scanned. zigpy scans them with ZDP Mgmt_Lqi_req and Mgmt_Rtg_req addressed to 0x0000. Ziggurat route-discovered its own address and failed with ROUTE_DISCOVERY_TIMEOUT. Frames addressed to our own NWK address are now delivered locally: the stack answers its ZDP requests, and the answers loop back to zigpy. This affects any Ziggurat coordinator under ZHA, not just ZLOE.
3. Routers reported the coordinator as a Router at depth 1. Neighbour entries learned from Link Status default to Router, and the depth was hardcoded to 1. 0x0000 is now reported as Coordinator at depth 0. This affects any Ziggurat router.

Not fixed: when the ZBOSS router joined, the coordinator logged Verify key hash mismatch several times during the trust-center link key exchange. The router stays joined and works.

Soak: On/Off to the ZBOSS router every 30 s through the live ZHA network, running until stopped. First results: 5 of 5 acknowledged, 40–42 ms to APS ack.

Next: a design for secure ZLOE over QUIC, in crates/ziggurat-phy-trel/BROADCAST.md:

• Keys: all derived from the Zigbee network key, so nodes stay stateless.
• Peer links: mutual TLS between peers, frames carried as QUIC datagrams.
• Enrollment: a one-time code, bound to the TLS session.
• Multicast: sealed with a key derived from the network key, plus a per-start epoch against replays.

[t3hk0d3]

Soak (ZHA, 3 backbone nodes, multicast)

Setup:

• Home Assistant 2026.11.1 with ZHA, coordinator plus two satellites on a wired backbone (IPv6 link-local), multicast on.
• One On/Off every 30 s to the ESP32-C6 ZBOSS router 0x93E4, sent through the coordinator's API alongside ZHA.

Window: 17:31–18:23, 52.5 minutes.

Metric	Result
On/Off sent	106
Acknowledged (APS ack)	106 (100 %)
Latency to APS ack	min 40 ms, median 41 ms, p95 50 ms, p99 66 ms, max 66 ms
Backbone broadcasts (3 nodes, 45 min)	542, all sent as one multicast datagram, 0 unicast copies
Multicast probes (3 nodes, 45 min)	1626, preferred group only (one every 5 s per node)
Backbone links	coordinator and satellites at cost 1/1 in every 5-minute snapshot
Radio links to the ZBOSS router	stable at in-cost 7, out-cost 4–5 (ESP RCP LQI ~10)

Observations:

• No failed send, no neighbour lost, and no fallback to unicast copies during the whole run.
• The router is a direct radio neighbour of the coordinator, so the On/Off traffic stays on the radio. The backbone carries the mesh's own traffic: Link Status, many-to-one route requests, ZHA's topology scans.
• Probe traffic is now about 3× the broadcast traffic on this quiet network (1626 against 542). Piggybacking the ack request on real broadcasts would remove most of it.

[t3hk0d3]

I didn't notice Claude started spamming comments here. Sorry.

My next test: segmentation.