Skip to content

feat(tools): force_ger_update — force periodic L1 GER updates#1730

Merged
arnaubennassar merged 19 commits into
developfrom
feat/force-ger-update-tool
Jul 21, 2026
Merged

feat(tools): force_ger_update — force periodic L1 GER updates#1730
arnaubennassar merged 19 commits into
developfrom
feat/force-ger-update-tool

Conversation

@arnaubennassar

Copy link
Copy Markdown
Collaborator

🔄 Changes Summary

  • Add force_ger_update: a standalone, long-running CLI tool that guarantees the L1 Global Exit
    Root (GER) is updated at least every configurable X
    . It watches L1 for UpdateL1InfoTree
    events and, if none happens organically within the window, sends a bridgeMessage with
    forceUpdateGlobalExitRoot = true to force a new L1 info root.
  • Motivation (OP-FEP / DA provability): for aggchains running OP-FEP, every GER update on L1
    appends a leaf to the L1 info tree that embeds an L1 block hash, and the aggchain proof uses
    that block hash to attest to what happened on L1 — including data availability (DA). Anything
    posted on L1 after the last L1 info root update is covered by no block hash inside any L1 info
    root and therefore cannot be proven until a new update lands. If DA is posted after the last
    update, certificate progress stalls until an organic GER update happens — with no bound on the
    wait. This tool bounds that wait to X.
  • Design: no syncer / reorg detector / event DB — boot scan via chunked FilterLogs, then WS
    WatchUpdateL1InfoTree (if L1WSURL set) or FilterLogs polling; sends via the standard
    zkevm-ethtx-manager; keys via go_signer (local keystore or GCP/AWS KMS).
  • Wired into Makefile (build-force_ger_update, added to build-tools) and shipped in the
    Docker image at /usr/local/bin/force_ger_update.
  • CI: the Tier-2 real-network e2e (TestForceGERUpdateE2E) now runs in CI on its own dedicated,
    isolated job/runner (test-go-e2e-force-ger-update in .github/workflows/test-go-e2e.yml)
    instead of being permanently skipped, so it is exercised automatically without risking the shared
    test-go-e2e env/job.

⚠️ Breaking Changes

  • 🛠️ Config: none (new standalone tool config; no existing config changed).
  • 🔌 API/CLI: none (additive — new tool binary only).
  • 🗑️ Deprecated Features: none.

📋 Config Updates

  • 🧾 New standalone TOML with a [ForceGERUpdate] root section (+ nested […EthTxManager] /
    […EthTxManager.Etherman]). See tools/force_ger_update/example-config.toml and the README's
    configuration reference. Mandatory & validated: L1URL, GlobalExitRootManagerAddr, BridgeAddr,
    DestinationNetwork (≠ 0), MaxTimeWithoutGERUpdate (> 0), CheckInterval (> 0),
    FilterLogsChunkSize (> 0), and EventPollInterval (> 0, poll mode only).

✅ Testing

  • 🤖 Automatic:
    • Tier-1 (unit + integration on the simulated backend, run by make test-unit): config
      load/validate, monitor (boot scan + watch/poll), sender (calldata/selector, statuses, dry-run),
      main loop (stale-on-boot, event-reset, in-flight guard, clean shutdown) — go test -race ./tools/force_ger_update/...; plus a simulated-backend integration test
      (integration_test.go) exercising both watch and poll modes and asserting real on-chain
      UpdateL1InfoTree events and the 0x240ff378 selector.
    • Tier-2 (isolated real-network e2e CI job): TestForceGERUpdateE2E (test/e2e/) execs the
      built binary against the docker-compose op-pp env and proves causality — the new GER update's
      tx From == tool sender and input starts with 0x240ff378. It is env-gated
      (RUN_FORCE_GER_UPDATE_E2E=true) so it stays skipped in the shared make test-e2e / main
      test-go-e2e CI job (avoiding interference with that job's shared post-test bridge health
      check), and instead runs automatically in CI on its own dedicated runner —
      test-go-e2e-force-ger-update, a new job in .github/workflows/test-go-e2e.yml that reuses the
      existing docker-image build/pull pipeline and calls make test-e2e-force_ger_update. That
      target sets both RUN_FORCE_GER_UPDATE_E2E=true and E2E_SKIP_POSTTEST_BRIDGE_CHECK=true (the
      latter opts the isolated job out of TestMain's shared post-test L1↔L2 bridge health check,
      since forcing GER updates legitimately leaves that check unhealthy — same root cause as the
      existing remove_ger e2e skips). Locally: make test-e2e-force_ger_update (docker-compose env
      must be up).
  • 🖱️ Manual:
    • DryRun = true smoke test against a live env logs the boot-derived last-GER-update age without
      sending a transaction.

🐞 Issues

  • N/A

🔗 Related PRs

  • [Optional]

📝 Notes

  • Timer semantics: lastGERUpdate is an L1 block timestamp; elapsed is measured against the tool's
    wall clock. Clock skew is immaterial next to a minutes-to-hours threshold and is safe in both
    directions (a block timestamp slightly ahead of wall-clock yields a negative elapsed → the tool
    simply waits longer; it never over-fires and never panics).
  • The timer is reset only by an observed UpdateL1InfoTree event, never by the send completing;
    in poll mode this permits at most one redundant (harmless) send per window if the observation lags
    the mine. Sends are serialized by an atomic in-flight guard, so they are never concurrent or
    unbounded.
  • gasOffset is hardcoded to 0; gas is controlled via the ethtxmanager config
    (ForcedGas/GasPriceMarginFactor/MaxGasPriceLimit).
  • The Tier-2 e2e's isolation contract (RUN_FORCE_GER_UPDATE_E2E, E2E_SKIP_POSTTEST_BRIDGE_CHECK)
    is documented in the README's "How to test" section and in code comments at both env-var
    check-sites.

arnaubennassar and others added 16 commits July 20, 2026 10:13
…plumbing

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…tdown

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…ionale

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…backend

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…op-pp env)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…ll green)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
- config validation: reject non-positive MaxTimeWithoutGERUpdate/CheckInterval/
  FilterLogsChunkSize and (poll mode) EventPollInterval — the latter two feed
  time.NewTicker, which panics on <=0, crashing the tool on boot when omitted
- README: fix docker command to override the aggkit ENTRYPOINT; mark newly
  required config fields
- e2e: skip TestForceGERUpdateE2E by default (remove_ger precedent — perturbs the
  shared post-test bridge health check); runnable manually
- run.go: document timer/clock-skew semantics (block ts vs wall clock)
- PLAN.md: append PR draft + final summary (section 7) and S8 review log

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…plete)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…1 (open PR)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
… runner

TestForceGERUpdateE2E now runs gated on RUN_FORCE_GER_UPDATE_E2E=true instead
of an unconditional t.Skip, so a dedicated CI job can opt in while the main
test-e2e suite (no env vars) still skips it with zero interference.

testmain_test.go's post-test L1<->L2 bridge health check gains an opt-in
E2E_SKIP_POSTTEST_BRIDGE_CHECK=true escape hatch (default unset = unchanged
behavior), since forcing a GER update legitimately leaves that shared check
unhealthy (same reason removeger_test.go skips).

Adds a `test-e2e-force_ger_update` Makefile target wiring both env vars, and
a new isolated `test-go-e2e-force-ger-update` job in test-go-e2e.yml that
reuses the existing docker-image build/pull artifacts rather than duplicating
that pipeline; the existing test-go-e2e job is untouched.

Also fixes one pre-existing thelper lint finding in forcegerupdate_test.go
(missing t.Helper()), unrelated to this change but required to keep
`make lint` at 0 issues.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
Update the section-7 PR draft to reflect S10's isolated Tier-2 e2e CI
job (test-go-e2e-force-ger-update, RUN_FORCE_GER_UPDATE_E2E /
E2E_SKIP_POSTTEST_BRIDGE_CHECK) under Changes Summary, Testing, and
Notes; keep the DA-provability motivation and template section order.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
@arnaubennassar arnaubennassar self-assigned this Jul 20, 2026
arnaubennassar and others added 2 commits July 20, 2026 18:02
TestRunLoop_StaleOnBoot_SendsExactlyOnce asserted the send count after the loop
returned, where a tick racing with ctx cancellation could legitimately start a
second (harmless) send during shutdown -> flaked in CI (got 2, want 1). Now the
send is held in flight and the count is asserted on the stable in-flight window,
proving the guard without depending on teardown ordering.

TestRunLoop_EventBeforeThreshold_NoSend delivered the reset event from a goroutine
after a 5ms sleep, which on a loaded runner could arrive after the threshold tick.
Now the event is pre-buffered so the loop's first select consumes it before the
first tick, and the ctx window is shorter than the threshold -> deterministic.

Verified: go test -race -count=30 -run TestRunLoop passes; package -race + lint green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d
…iming

TestForceGERUpdate/PollMode flaked in CI: scenario 2 (require.Never a 2nd send)
tripped because integrationSenderPoll == integrationEventPollInterval (10ms) gave
no ordering margin — the in-flight guard could release (send confirmed) before the
monitor's poll observed the reset event, letting the next tick fire the documented
"redundant harmless" send. Make senderPoll (300ms) ~30x the monitor poll (10ms),
encoding the production invariant (mining latency >> event-detection latency), so
the guard stays armed until after the reset is observed. Robust under proportional
scheduler starvation (ratio-based, not absolute).

Also run the WatchMode/PollMode subtests sequentially instead of in parallel: each
drives its own simulated backend + monitor/sender loop, and running both at once
starves the timing-sensitive poll/commit goroutines on constrained CI runners.

Verified: go test -race -count=10 passes; also -count=5 under GOMAXPROCS=1 (heavy
starvation) passes; package -race + lint green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRmUbKmtYbSjhhRaT3Ea5d

@joanestebanr joanestebanr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@arnaubennassar
arnaubennassar merged commit 44de0a3 into develop Jul 21, 2026
39 of 41 checks passed
@arnaubennassar
arnaubennassar deleted the feat/force-ger-update-tool branch July 21, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants