Skip to content

[codex] fix(input): buffer Windows console decoder bytes#29

Merged
tonyfettes merged 12 commits into
mainfrom
codex/win32-input-record-enum
Jun 10, 2026
Merged

[codex] fix(input): buffer Windows console decoder bytes#29
tonyfettes merged 12 commits into
mainfrom
codex/win32-input-record-enum

Conversation

@tonyfettes

Copy link
Copy Markdown
Contributor

What changed

  • Add internal/io.ByteQueue, an internal byte source that implements @async/io.Reader and supports synchronous byte enqueueing.
  • Replace the Windows console input byte pipe with ByteQueue so key-record bytes are buffered before the shared input decoder consumes them.
  • Drain available Windows console records synchronously and queue direct native events with try_put.
  • Add regression coverage for queued direct events followed by a complete SGR mouse byte sequence.
  • Update examples/raw to show printable ASCII characters next to raw byte values.

Why

Windows ReadConsoleInputW key records can represent VT mouse sequences as individual UTF-16 characters. The previous unbuffered pipe write could suspend inside the short-lived record producer; if a direct event won the read race, the producer could be cancelled before the tail of the VT sequence reached the decoder. That made examples/input display fragments such as [3;20M as key events.

Buffering bytes in an internal queue makes producer-side enqueueing synchronous, while preserving the existing shared input decoder and direct native event queue.

Validation

  • moon fmt
  • moon test internal/io --target-dir .moon-test-internal-io-bytequeue-build
  • moon test . --filter "win32*" --target-dir .moon-test-win32-bytequeue-shape-build
  • moon test internal/input --filter "decode delayed *mouse*" --target-dir .moon-test-input-delayed-mouse-bytequeue-shape-build
  • moon check --target-dir .moon-check-bytequeue-shape-build
  • moon info --target-dir .moon-info-bytequeue-shape-build
  • git diff --check

@tonyfettes tonyfettes marked this pull request as ready for review June 10, 2026 03:05

@tonyfettes tonyfettes left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Don't eats up error without testing for @async.is_being_cancelled.
  2. Use if ... is, don't use match on Option
  3. Use guard to de-indent code.

Comment thread examples/raw/main.mbt Outdated
Comment thread internal/io/byte_queue.mbt Outdated
Comment thread internal/io/byte_queue.mbt Outdated
Comment thread internal/io/byte_queue.mbt
Comment thread internal/io/byte_queue.mbt Outdated
Comment thread win32_input.mbt Outdated
Comment thread win32_input.mbt Outdated
Comment thread win32_input.mbt Outdated
Comment thread win32_input.mbt Outdated
Comment thread win32_input_wbtest.mbt

tonyfettes commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the latest review feedback in 15a7926, corrected the ByteQueue visibility in 74b6916, and removed the unnecessary cancellation shield in a267bf8.

Notes on the non-literal bits:

  • ByteQueue is now a plain struct, which generates type ByteQueue in internal/io/pkg.generated.mbti. That keeps it usable by the root package through its public methods while hiding its representation.
  • ByteQueue._direct_read relies on Queue::get() for the wake/cancel race, then performs the buffer write and drain synchronously before returning len.
  • Removed the cancellable producer task from the Win32 decoder tests by queueing records synchronously before reading.
  • Replaced the unsafe char conversions with a checked Int::to_char() helper.

Comment thread internal/io/byte_queue.mbt Outdated
} noraise {
byte => byte
}
@async.protect_from_cancel() <| () => {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we protected from cancel here?

@tonyfettes tonyfettes merged commit c7d3bcf into main Jun 10, 2026
3 checks passed
@tonyfettes tonyfettes deleted the codex/win32-input-record-enum branch June 10, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant