fix: update ENR with actual port when bound to ephemeral port 0#198
Merged
lucassaldanha merged 7 commits intoConsensys:masterfrom Feb 24, 2026
Merged
Conversation
When DiscoverySystem is started with port 0, the OS assigns an ephemeral port at bind time. Previously the local NodeRecord (ENR) was never updated with the actual bound port, so advertised addresses were unreachable. - NettyDiscoveryServerImpl.getListenAddress() now returns the configured IP with the actual OS-assigned port after bind when port was 0; non-zero port behaviour is unchanged. - DiscoveryManagerImpl.start() calls LocalNodeRecordStore.onSocketAddressChanged() with the real port after bind, but only when the ENR was initialised with port 0. The advertised IP is preserved (not replaced with the bind address). For fixed-port nodes the callback is never triggered, so existing behaviour is identical.
|
All contributors have signed the CLA ✍️ ✅ |
Contributor
Author
|
I have read the CLA Document and I hereby sign the CLA |
Contributor
Author
|
recheck |
Replace the onSocketAddressChanged approach in DiscoveryManagerImpl with onBoundPortResolved, a new LocalNodeRecordStore method that directly updates the UDP/UDP6 port without going through NewAddressHandler. The previous approach was broken for callers (e.g. Besu) that configure a NewAddressHandler which rejects all external address changes. Because onSocketAddressChanged routes through the handler, the ENR port was silently left at 0 even after the OS assigned an actual port. onBoundPortResolved is intentionally exempt from NewAddressHandler because ephemeral port resolution is an internal OS operation, not an externally-reported address change. The advertised IP from the ENR is preserved; only the port field is updated. - Add LocalNodeRecordStore.onBoundPortResolved(InetSocketAddress) - Simplify DiscoveryManagerImpl.thenAccept to call onBoundPortResolved - Add unit tests: ephemeral port is updated, non-ephemeral is no-op Signed-off-by: Usman Saleem <usman@usmans.info>
…ual-stack race In dual-stack configurations with both IPv4 and IPv6 on port 0, the two thenAccept callbacks in DiscoveryManagerImpl.start() run concurrently on separate Netty threads. Both called onBoundPortResolved simultaneously, read the same latestRecord snapshot, each built a new record updating only their own IP family's port, and the second write overwrote the first — leaving one family's ENR port stuck at 0. Replace volatile NodeRecord with AtomicReference<NodeRecord> and use a CAS retry loop in onBoundPortResolved. If Thread 2's compareAndSet fails (Thread 1 already wrote), it retries against the fresh record which already has Thread 1's port update, so the second build correctly preserves it. Other methods (onSocketAddressChanged, onCustomFieldValueChanged) are updated to use AtomicReference.get/set for consistency but do not use CAS loops as they are not called concurrently in the same way. Signed-off-by: Usman Saleem <usman@usmans.info>
Add onBoundPortResolvedHandlesConcurrentDualStackUpdates to verify that simultaneous IPv4 and IPv6 port resolution in dual-stack mode does not lose either update. Uses CyclicBarrier to force both threads to race, then asserts both udp and udp6 ports are non-zero in the final ENR. Demonstrates correctness of the AtomicReference CAS loop.
Signed-off-by: Usman Saleem <usman@usmans.info>
6 tasks
usmansaleem
added a commit
to usmansaleem/besu
that referenced
this pull request
Feb 23, 2026
When --p2p-port=0 the OS assigns the UDP port only after bind. Previously updateNodeRecord() was called before the library resolved the port, leaving udp/udp6=0 in the persisted ENR indefinitely. Requires Consensys/discovery#198 which adds onBoundPortResolved() to LocalNodeRecordStore. The localNodeRecordListener in PeerDiscoveryAgentV5 detects port-0 → real-port transitions and calls NodeRecordManager.onDiscoveryPortResolved() to sync them before writing. In dual-stack mode both UDP servers bind concurrently; the listener defers the ENR write until both ports are resolved so the seq counter increments exactly once. A ReentrantLock guards onDiscoveryPortResolved and updateNodeRecord against concurrent RocksDB commits. Signed-off-by: Usman Saleem <usman@usmans.info>
usmansaleem
added a commit
to usmansaleem/besu
that referenced
this pull request
Feb 23, 2026
Strips the ephemeral port handling so this branch contains only Fix 1 (RLPx dual-stack TCP binding) and Fix 2 (ENR tcp/tcp6 port fields). Removed: - NodeRecordManager.onDiscoveryPortResolved() and ReentrantLock - Full localNodeRecordListener in PeerDiscoveryAgentV5 - hasEphemeralPort helper Ephemeral port resolution (Fix 3) depends on Consensys/discovery#198 and will be tracked separately. Signed-off-by: Usman Saleem <usman@usmans.info>
5 tasks
lucassaldanha
approved these changes
Feb 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When `DiscoverySystem` is started with port `0` (e.g. Besu `--p2p-port=0`), the OS assigns an ephemeral port at bind time. Previously the local NodeRecord (ENR) was never updated with the actual bound port, so the advertised address was unreachable by peers.
Changes
`NettyDiscoveryServerImpl.getListenAddress()`
Returns the configured IP with the actual OS-assigned port after bind, but only when the configured port was `0`. Non-zero port behaviour is completely unchanged.
`LocalNodeRecordStore.onBoundPortResolved(InetSocketAddress)`
New method that updates the UDP or UDP6 port in the ENR after the OS assigns an actual port for an ephemeral bind. Unlike `onSocketAddressChanged`, this method bypasses `NewAddressHandler` because ephemeral port resolution is an internal OS operation, not an externally-reported address change. The advertised IP is preserved from the existing ENR; only the port is updated. No-op if the current ENR port is not `0`.
`DiscoveryManagerImpl.start()`
Calls `localNodeRecordStore.onBoundPortResolved(boundAddress)` in the per-server `thenAccept` callback after each server binds. The `NewAddressHandler` is intentionally bypassed here — callers such as Besu configure their handler to block all external address changes (to prevent peers from altering the advertised address), which would otherwise silently suppress the ephemeral port update.
Motivation
Fixes the ephemeral UDP port issue in Besu where `--p2p-port=0` (or `--p2p-discovery-port=0`) caused the ENR to advertise `udp=0` / `udp6=0`, making the node unreachable by peers.
The initial fix attempt used `onSocketAddressChanged` but that routes through `NewAddressHandler`. Besu's handler returns the old record unchanged (to prevent peers from overriding the advertised address), so the ENR update was silently dropped. `onBoundPortResolved` is the correct path for trusted internal updates.
After this fix, Besu's `warnIfEphemeralPortsWithDiscV5()` warning can be removed since ephemeral UDP ports now work correctly.
Test plan
Note
Medium Risk
Touches discovery startup and ENR mutation logic; incorrect port/IP-family handling or concurrency bugs could make nodes advertise unreachable addresses, though changes are scoped and covered by new unit/integration tests.
Overview
Ensures nodes started with an ephemeral UDP port (
port=0) advertise the actual OS-assigned port in their local ENR after binding.NettyDiscoveryServerImpl.getListenAddress()now reports the bound port once started (while preserving the configured IP), andDiscoveryManagerImpl.start()uses that bound address to both select the channel’s IP family and to call the newLocalNodeRecordStore.onBoundPortResolved().LocalNodeRecordStoreis made thread-safe viaAtomicReferenceand addsonBoundPortResolved()to update only the UDP/UDP6 port (bypassingNewAddressHandler) with a CAS loop to avoid losing concurrent dual-stack updates; new tests cover server listen address behavior, ENR port resolution in an integration run, and the new store logic including dual-stack concurrency.Written by Cursor Bugbot for commit 8e5a224. This will update automatically on new commits. Configure here.