Gateway session executor crashes mid-task (Executor exited unexpectedly, code 1 / heartbeat_lost) — failure never surfaced to Slack

[reedbousquet1-commits]

Summary

A gateway-triggered session's executor process died mid-task twice in a row, with both tasks ending in error_message: "Executor exited unexpectedly with code 1" / sdk_failure.reason: "heartbeat_lost". Neither failure was surfaced to the user in Slack — the thread just went silent, as if the bot ignored them.

Where

• Branch private-customer-escalations-bot, session 019fa479
• Triggered from Slack #C02FW38QPGU

Repro / timeline (real incident, 2026-07-27)

• 21:07:46 UTC — user asks the bot to move a card to Shipped. Task crashes: error_message: "Executor exited unexpectedly with code 1", sdk_failure.reason: "heartbeat_lost".
• 21:16:07 UTC — user follows up ("you still there? I think this session died"). Second task starts as a fresh process, makes zero tool calls, and crashes with the identical error/reason.
• Both Slack messages went unanswered — no error surfaced, no retry, just silence.

Root cause evidence

From ~/.claude/debug/<sdk_session_id>.txt, both crashes share an identical fingerprint:

1. Process runs normally for ~5 seconds.
2. The agor and zapier MCP connections simultaneously report "HTTP connection closed after 5s (cleanly)", immediately followed by "dropped after 5s uptime" / AbortError: The operation was aborted.
3. The log stops cold right there — no exception, no stack trace, no graceful-shutdown message.

Two independent MCP transports dying at the exact same instant, with a clean-looking teardown and total silence afterward (not a caught exception), points to the executor process itself being killed by something outside the app (host/container OOM-kill, resource limit, or scheduler eviction) rather than an application-level bug in either MCP client.

Impact

• User-facing: the assistant appears to silently ignore Slack messages, with no error, no retry, and no indication anything went wrong. Same "silent failure" experience as Gateway session can wedge in 'running' with no watchdog, silently blocking the assistant's Slack queue #1844, but the underlying cause here is a genuine process crash, not a wedge.
• Two consecutive crashes in the same session/branch suggests this isn't a one-off blip — worth checking whether the host/container running this executor has a tight memory/resource limit that's getting tripped repeatedly.

Suggested directions

1. Platform/infra check: look for OOM-kills or resource-limit evictions on the executor process/container for this branch's environment around 21:07–21:16 UTC on 2026-07-27.
2. Surface the failure to the user: when a gateway task ends in heartbeat_lost/crashed, the gateway should post something to the originating Slack thread (even a generic "this task crashed, retrying..." message) instead of going silent — related to the Slack-visibility gap in Gateway session can wedge in 'running' with no watchdog, silently blocking the assistant's Slack queue #1844.
3. Capture exit signal: if the executor wrapper can distinguish a real process exit code/signal (e.g. SIGKILL from OOM) from a generic "code 1", surfacing that in error_message would make root-causing this much faster next time.

Related

• Gateway session can wedge in 'running' with no watchdog, silently blocking the assistant's Slack queue #1844 — Gateway session can wedge with no watchdog, silently blocking the Slack queue (same "user gets silence" symptom, different mechanism — wedge vs. crash)
• Executor heartbeat supervisor false-positive: busy-but-alive sessions marked failed #1809 — Executor heartbeat supervisor false-positive (different mechanism — here the process genuinely appears to have died, not just gone heartbeat-stale)
• Codex sessions hang forever after an early executor-socket transport close (heartbeat-alive / agent-dead zombie) #1820 — Codex sessions hang after early transport close (different agent type / hang vs. crash, but same family of executor-transport-liveness issues)

Blockers / Dependencies

None identified — this can be investigated independently, though the platform/infra check (host resource limits) may need input from whoever owns the executor hosting environment.

[mistercrunch]

This might be fixed in main right now if it's related to the 1MB socket overflow we just fixed. Let me push a new minor version so you can install from npm, see if that fixes your issue.