Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ This board tracks implementation direction for `moonbit-community/tty`. Use
| TTY-22 | done | Windows input record enum | `docs/plans/2026-06-09-windows-input-record-enum.md`, root package | Windows `INPUT_RECORD` bytes parse into private typed variants, including native mouse records | `moon fmt`, targeted Windows wbtests, `moon test .`, `moon test`, `moon check`, `moon info`, `git diff --check` |
| MVP-2 | done | queued input and shell commands in agent demo | `docs/plans/2026-05-21-agent-queued-shell.md`, `examples/agent` | `Tab` queues input for delayed submission and `!cmd` runs through `moonbitlang/async/process` without background tasks writing directly to tty | `moon fmt`, `moon check examples/agent`, `moon test examples/agent`, `moon check`, `moon test`, `moon info`, `git diff --check` |
| TTY-23 | done | serialize root `Tty` output writes | `docs/plans/2026-06-15-output-write-lock.md`, `internal/lock/`, root package | a single `Tty::write`/`Tty::write_string` call lands atomically under concurrent writers via an internal `Lock`, with no public API change | `moon fmt`, `moon test internal/lock`, `moon test .`, `moon test`, `moon check`, `moon info`, `git diff --check` |
| DEP-1 | in progress | upgrade `moonbitlang/async` 0.19.1 -> 0.19.4, working around the Windows console-open regression by opening `CONIN$`/`CONOUT$` directly and wrapping both platforms' terminal fd in `RawFdStream` | `docs/plans/2026-06-15-async-0.19.4-upgrade.md`, all workspace `moon.mod` files, `tty_open.c`, `tty_win32.mbt`, `tty_unix.mbt`, `io.mbt`, `isatty.mbt` | every module pins `0.19.4`, console devices bypass `@async/fs.open`, and CI stays green on all platforms; `.mbti` gains only the intended `RawFdStream` impls | `moon fmt --check`, `moon check --deny-warn`, `moon test --deny-warn`, `moon test` (`tests/`), `moon info`, `git diff --check` |

## Current Rules

Expand Down
135 changes: 135 additions & 0 deletions docs/plans/2026-06-15-async-0.19.4-upgrade.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Upgrade moonbitlang/async 0.19.1 -> 0.19.4

## Goal

Move every workspace module from `moonbitlang/async@0.19.1` to
`moonbitlang/async@0.19.4`, working around the Windows console-open regression
that the bump exposes, and confirm the upgrade is safe on all CI platforms
(notably windows-latest).

## Status

In progress. Root cause of the windows-latest regression is identified and a
workaround is implemented (open the console devices directly instead of through
`@async/fs.open`). Pending windows-latest CI confirmation.

## Context And Decisions

- The output write lock (TTY-23,
`docs/plans/2026-06-15-output-write-lock.md`) only needs `Semaphore`, which
exists in `0.19.1`, so it intentionally stayed on `0.19.1`. This task does the
dependency bump on its own so the two concerns stay reviewable in isolation.
- Bump all six manifests that pin async so the workspace resolves to a single
version and declared deps match what is actually used:
root, `tests`, `tests/isatty`, `tests/resize`, `tests/open`, `examples`.

### Root cause of the windows-latest regression

- `Tty::open` on Windows opened `CONIN$` / `CONOUT$` through
`@async/fs.open(..., mode=ReadWrite)`.
- In async `0.19.2` (carried into `0.19.4`), the Windows file-open worker changed
how it detects the file kind. `0.19.1` used `GetFileType()`
(`moonbitlang_async_kind_of_fd`), which classifies a console as `CharDevice`
and always succeeds. `0.19.2+` instead calls `GetFileInformationByHandle()`
(then `kind_from_attr`) to also obtain `dev_id` / `file_id` for the new
`FileIdentity` / `fs.watch` feature.
- `GetFileInformationByHandle` is a disk-file API; it fails on console
(character) devices. On failure the open worker sets `job->err` and closes the
handle, so `@async/fs.open("CONIN$"/"CONOUT$")` now fails. That broke the
entire Windows `Tty::open` path and showed up as the `tests/open` pty test
failing on windows-latest.
- The regression is upstream and was introduced in `0.19.2`, not `0.19.4`.

### Workaround

- `CONIN$` / `CONOUT$` are console devices, not filesystem files, so they should
not go through `@async/fs.open`. Open them directly with `CreateFileW`
(synchronous, non-overlapped) in `tty_open.c`, then wrap each handle with
`@async/raw_fd.RawFdStream`. `RawFdStream` detects the kind with `GetFileType`
(`CharDevice`) and never registers the handle for overlapped IO
(`is_async=false`), which exactly reproduces the working `0.19.1` console
read/write path.
- Unify the controlling-terminal handling: the unix path previously hand-rolled
a `ControllingTerminal` struct around `@async/raw_fd.RawFd` with six manual
trait impls. `RawFdStream` provides the same buffered `@io.Reader`/`@io.Writer`
behavior, so `ControllingTerminal` is removed and both platforms now wrap their
terminal fd in `RawFdStream`. The unix change is exercised by the pty-backed
`Tty::open` test on macOS/Linux.

## Target Files

- `moon.mod`, `tests/moon.mod`, `tests/isatty/moon.mod`,
`tests/resize/moon.mod`, `tests/open/moon.mod`, `examples/moon.mod`
- `tty_open.c` (Windows `CONIN$` / `CONOUT$` open + handle-valid helper)
- `tty_win32.mbt` (new: Windows `Tty::open` via `RawFdStream`)
- `tty_unix.mbt` (drop `ControllingTerminal`; open `/dev/tty` and wrap in
`RawFdStream`)
- `io.mbt` (`Reader` / `Writer` impls for `RawFdStream`)
- `isatty.mbt` (`Fd` impl for `RawFdStream`)
- `docs/plan.md`
- generated `.mbti` files from `moon info`

## Public API Changes

- New public impls on all platforms: `Fd`, `Reader`, and `Writer` for
`@async/raw_fd.RawFdStream`. This lets raw fd streams be used with `Tty::new`
the same way `@async/fs.File`, pipes, and stdio already can.
- `Tty` public shape, `Tty::open`, `Tty::write`, and `Tty::write_string`
signatures are unchanged.

## Invariants

- `Tty::open` produces a coordinated handle whose output write path is the
non-overlapped worker path on Windows (`CharDevice`, `is_async=false`),
identical to the `0.19.1` behavior.
- Console devices are never opened through `@async/fs.open`.
- Workspace resolves to exactly one `moonbitlang/async` version (`0.19.4`).
- Unix and Windows share one terminal-stream wrapper (`RawFdStream`).

## Acceptance Criteria

- All six manifests pin `0.19.4`.
- `moon check`, `moon check --deny-warn`, and the full test suite pass locally,
including the pty-backed `Tty::open` test.
- CI passes on ubuntu-latest, macos-latest, and windows-latest (the windows
pty `Tty::open` test in particular).
- Generated `.mbti` reflects only the intended `RawFdStream` impl additions.

## Validation Commands

- `moon fmt --check`
- `moon check --deny-warn`
- `moon test --deny-warn` (root module)
- `moon test` in `tests/` (pty-backed module)
- `moon info`
- review generated `.mbti` diff
- `git diff --check`

## Public API Audit

- Added `pub impl Fd`, `pub impl Reader`, and `pub impl Writer` for
`@async/raw_fd.RawFdStream`, plus the `moonbitlang/async/raw_fd` import in the
generated `.mbti`. These are intentional, consistent with the existing
`File`/`Pipe`/`stdio` impls, and let callers wrap raw fd streams.
- No new public type, mutable field, parser state, input event, platform handle,
backend selection, or capability query.
- The dependency change is the async version (`0.19.1` -> `0.19.4`).

## Validation Results

- `moon fmt --check` passed.
- `moon check --deny-warn` passed.
- `moon test --deny-warn` (root) passed: 165 tests.
- `moon test` (`tests/`) passed: 133 tests, including the pty-backed `Tty::open`
test that drives `Tty::open` -> `RawFdStream` -> a real terminal write.
- `moon info` produced only the intended `RawFdStream` impl additions.
- `git diff --check` passed.
- windows-latest CI: pending.

## Open Questions

- Report the upstream `GetFileInformationByHandle`-on-console regression to
`moonbitlang/async` (the open worker should fall back to `GetFileType` /
`kind_of_fd` when `GetFileInformationByHandle` fails, instead of erroring).
- The Windows-only `Tty::open` path cannot be compiled or run locally on macOS;
it relies on windows-latest CI for verification.
2 changes: 1 addition & 1 deletion examples/moon.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ version = "0.1.0"

import {
"moonbit-community/tty@0.1.0",
"moonbitlang/async@0.19.1",
"moonbitlang/async@0.19.4",
"kawaz/grapheme@0.10.2",
"rami3l/unicodewidth@0.2.0",
}
Expand Down
14 changes: 14 additions & 0 deletions io.mbt
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,17 @@ pub impl Reader for @async/stdio.Input with fn close(_) {
pub impl Writer for @async/stdio.Output with fn close(_) {
()
}

///|
/// Allow raw file-descriptor streams to serve as terminal input handles.
/// Used for the controlling terminal (`/dev/tty`) and Windows console devices.
pub impl Reader for @async/raw_fd.RawFdStream with fn close(self) {
@async/raw_fd.RawFdStream::close(self)
}

///|
/// Allow raw file-descriptor streams to serve as terminal output handles.
/// Used for the controlling terminal (`/dev/tty`) and Windows console devices.
pub impl Writer for @async/raw_fd.RawFdStream with fn close(self) {
@async/raw_fd.RawFdStream::close(self)
}
7 changes: 7 additions & 0 deletions isatty.mbt
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,10 @@ pub impl Fd for @async/stdio.Input with fn fd(self) {
pub impl Fd for @async/stdio.Output with fn fd(self) {
@async/stdio.Output::fd(self)
}

///|
/// Allow raw file-descriptor streams (used for the controlling terminal and
/// Windows console devices) to be used with file-descriptor based terminal APIs.
pub impl Fd for @async/raw_fd.RawFdStream with fn fd(self) {
self.fd
}
2 changes: 1 addition & 1 deletion moon.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "moonbit-community/tty"
version = "0.2.5"

import {
"moonbitlang/async@0.19.1",
"moonbitlang/async@0.19.4",
Comment thread
tonyfettes marked this conversation as resolved.
}

readme = "README.md"
Expand Down
4 changes: 4 additions & 0 deletions pkg.generated.mbti
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import {
"moonbitlang/async/io",
"moonbitlang/async/os_error",
"moonbitlang/async/pipe",
"moonbitlang/async/raw_fd",
"moonbitlang/async/stdio",
}

Expand Down Expand Up @@ -128,6 +129,7 @@ pub(open) trait Fd {
pub impl Fd for @fs.File
pub impl Fd for @pipe.PipeRead
pub impl Fd for @pipe.PipeWrite
pub impl Fd for @raw_fd.RawFdStream
pub impl Fd for @stdio.Input
pub impl Fd for @stdio.Output

Expand All @@ -136,12 +138,14 @@ pub(open) trait Reader : @io.Reader + Fd {
}
pub impl Reader for @fs.File
pub impl Reader for @pipe.PipeRead
pub impl Reader for @raw_fd.RawFdStream
pub impl Reader for @stdio.Input

pub(open) trait Writer : @io.Writer + Fd {
fn close(Self) -> Unit
}
pub impl Writer for @fs.File
pub impl Writer for @pipe.PipeWrite
pub impl Writer for @raw_fd.RawFdStream
pub impl Writer for @stdio.Output

2 changes: 1 addition & 1 deletion tests/isatty/moon.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ version = "0.1.0"

import {
"moonbit-community/tty@0.1.0",
"moonbitlang/async@0.19.1",
"moonbitlang/async@0.19.4",
}

preferred_target = "native"
2 changes: 1 addition & 1 deletion tests/moon.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ version = "0.1.0"
import {
"moonbit-community/tty@0.1.0",
"moonbit-community/pty@0.2.1",
"moonbitlang/async@0.19.1",
"moonbitlang/async@0.19.4",
}

preferred_target = "native"
2 changes: 1 addition & 1 deletion tests/open/moon.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ version = "0.1.0"

import {
"moonbit-community/tty@0.1.0",
"moonbitlang/async@0.19.1",
"moonbitlang/async@0.19.4",
}

preferred_target = "native"
2 changes: 1 addition & 1 deletion tests/resize/moon.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ version = "0.1.0"

import {
"moonbit-community/tty@0.1.0",
"moonbitlang/async@0.19.1",
"moonbitlang/async@0.19.4",
}

preferred_target = "native"
40 changes: 40 additions & 0 deletions tty_open.c
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,43 @@ moonbit_tty_open_controlling_terminal(void) {
#endif
}
#endif

#ifdef _WIN32
#include <windows.h>

// `CONIN$` / `CONOUT$` are console (character) devices, not filesystem files.
// They are opened directly here as synchronous (non-overlapped) handles so they
// can be wrapped with `@async/raw_fd.RawFdStream`, which classifies them by
// `GetFileType` (CharDevice). Going through `@async/fs.open` instead would probe
// the handle with `GetFileInformationByHandle`, which fails on console devices.
static HANDLE
moonbit_tty_open_console(const wchar_t *name) {
return CreateFileW(
name,
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
0, // no FILE_FLAG_OVERLAPPED: RawFdStream requires a non-overlapped handle
NULL
);
}

MOONBIT_FFI_EXPORT
HANDLE
moonbit_tty_open_console_input(void) {
return moonbit_tty_open_console(L"CONIN$");
}

MOONBIT_FFI_EXPORT
HANDLE
moonbit_tty_open_console_output(void) {
return moonbit_tty_open_console(L"CONOUT$");
}

MOONBIT_FFI_EXPORT
int32_t
moonbit_tty_handle_is_valid(HANDLE fd) {
return (fd != INVALID_HANDLE_VALUE && fd != NULL) ? 1 : 0;
}
#endif
Loading
Loading