Add a local spectator panel to the agent client#13
Merged
Conversation
The agent plays headless, so watching it meant logging a second character in next to it. The client already holds everything a spectator needs — nearby entities, houses, the height sampler, chat/combat events, and every LLM turn — so serve it: an axum server on 127.0.0.1 (watch_port, default 8808) renders a live top-down map from real height tiles plus a categorized event feed including the prompts and responses the LLM exchanges, with per-NPC tabs when several run on one instance. Event capture hooks into SharedState::push_event before it mutates, while player names still resolve, reusing the prompt formatter so the panel and the LLM read the same lines. LLM turns are recorded in the scheduler, so every backend is covered at one choke point. Ring buffers live in the per-NPC hub and survive reconnects; the panel starts before Google sign-in so the page is reachable while the device flow waits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
XpGained carries new_level, max_hp and current_hp, but nothing landed in self_player, so the level shown by the spectator panel — and by the world state fed to the LLM — stayed at its join-time value forever. Apply it to the earning player, whether that is us or a nearby player. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Julian-adv
added a commit
that referenced
this pull request
Jul 23, 2026
…wn map Review follow-ups to PR #13. The panel worked; what it cost, and what it drew, did not hold up. Drawing the world twice. The map sampled `/api/height` and coloured it with its own elevation ramp, while `data/terrain/minimap/` already holds 1024 baked region PNGs — 1 px per metre, coloured from `shared/palette.json`, and already drawn by the web client's WorldMapDialog. Two colour schemes for one world: re-bake the terrain and only the spectator map drifts. The panel now serves those PNGs (`/api/minimap/{rx}/{rz}`) using the client's own region math, from a local terrain tree or proxied from the tile API when the agent runs elsewhere — the game server sends no CORS headers, so the browser cannot fetch them itself. That deletes the endpoint, the ramp, and `sample_grid`; zoomed-out resolution improves from ~10.7 m/sample to 1 m/px. The panel handle had reached too deep. `LlmScheduler::submit` took a spectator handle, so a fleet-wide queue knew about a debug UI, and the handle had to be threaded through `DriverConfig`, `spawn_llm_task`, and a `RollAdvisor` struct that existed only to dodge an argument-count lint. `build_llm_backend` is already the one funnel where every backend is constructed; wrapping there (`WatchedBackend`) covers claude / codex / openrouter and removes all five threading points. The scheduler no longer knows the panel exists. This does give up the queue-wait figure, but the bug it was added for — counting queue time as inference — stays fixed: the decorator times the backend call only. The feed rebuilt itself every second. `feed_snapshot` returned the last 200 of a 300-item ring, and the page detected rotation by comparing the first item's timestamp, which changes on every push once the ring turns. Past 200 events the list was torn down and rebuilt each poll, snapping shut any LLM prompt you had opened to read. Items now carry a monotonic seq and `/api/state?since=` returns only what the page has not drawn — roughly 340 KB/s of re-sent feed at a typical cadence, and the ring's last 100 items become reachable instead of being trimmed away unseen. The response is also serialized after the agent's state mutex is released, not while holding it. `watch_port = 0` turned off the server but not the instrumentation: the hub was built unconditionally, so every NPC still formatted feed lines and copied every prompt for a feed nobody could read. The hub is built only when the panel is on, and the default is off — it serves full LLM prompts, so it should be asked for. A `Host` check closes DNS rebinding against those prompts. Smaller, from the same pass: - Feeds were keyed by label, but `NpcConfig::label()` falls back to "agent", so two Google-auth NPCs without a `character_name` shared one feed and clobbered each other's state handle. Sessions key by config index. - `XpGained`'s `nearby_players` arm was unreachable — the server only ever sends it direct to the player it describes. - `sample_height` kept neighbour-tile loads that cannot fire: `VERTS_PER_SIDE` is `TILE_DIM + 1`, so every bilinear corner is in the covering tile. The arithmetic feeding them duplicated `sample_cached`. - `draw()` ran full-canvas at 60 fps forever, re-parsing `ctx.font` and measuring text per entity per frame, though data arrives at 1 Hz. - An unknown `?npc=` returned 200 with an error body the page rendered as a live NPC; now 404. The merge needed one fix of its own: master replaced `RECONNECT_DELAY` with the `ws::retry_delay` backoff while the PR was open, and the two auto-merged into code referencing a constant that no longer exists. The reconnect notice moved to `run_npc_loop`, which knows the real delay. Verified against the dev server with both NPCs connected: the served PNG for Karl's region is byte-identical to the baked file and 1024x1024, an unbaked region 404s, `since` returns an empty feed while still sending state, loopback Hosts serve while evil.com and 127.0.0.1.evil.com get 403, and `watch_port = 0` opens no listener while the agents still enter the game. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Owner
|
Thanks for this PR 🙏 Watching a headless agent has been a real gap, and you nailed the spots that matter — capturing every LLM turn at one choke point (
Shipped as agent-client v0.3.0. Thanks again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The agent plays headless — the only way to watch it was logging a second character in next to it. This adds a read-only web panel served by the agent-client itself on
127.0.0.1:8808(watch_portin config,0disables):LlmScheduler::submitso every backend (claude / codex / openrouter) is covered at one choke pointImplementation notes:
SharedState::push_eventbefore the message mutates state, while player names still resolve, reusingdriver::prompt::format_eventso the panel shows exactly the lines the LLM reads. Movement/time-sync spam stays off the feed (the map shows it live instead)./api/heightsamples the sharedHeightSamplerserver-side, so the browser needs no tile protocol and the tile cache is reused.127.0.0.1, strictly read-only (no command path from the panel to the agent).Tested:
cargo test -p agent-client20/20 passing, clippy clean for the new code (the 3 pre-existingneedless_borrowwarnings in orchestrator.rs are untouched), plus a live smoke test — panel served real prod height tiles around (5000, −4000) with a stubbed session snapshot to verify map, feed, filters, and LLM-turn rendering.🤖 Generated with Claude Code