Skip to content
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
6cebb50
Add operation timeouts to sandbox provisioning and teardown
paddybyers May 3, 2026
6730b98
Increase integration test suite timeouts from 60s to 120s
paddybyers May 3, 2026
2953ebe
Fix RSL2b3 history time range test to use server timestamps
paddybyers May 3, 2026
89ad6a2
Restructure UTS tests to match spec repo unit/integration layout
paddybyers May 3, 2026
e0c1345
Add proxy integration test infrastructure and tests
paddybyers May 3, 2026
774ed58
Add tier 1+2 proxy integration tests
paddybyers May 3, 2026
f9f6166
Add ably-js tests for missing UTS spec coverage
paddybyers May 3, 2026
b61e070
Auto-launch Go test proxy from Node.js
paddybyers May 3, 2026
0302d6b
Add RSC15f test: expired fallback not resurrected by late in-flight s…
paddybyers May 3, 2026
743e8a3
Add RTN15a test variant for TCP close without WebSocket close frame
paddybyers May 4, 2026
910c107
Download uts-proxy binary from GitHub releases instead of building fr…
paddybyers May 4, 2026
7444c39
Remove deviations.md from repo
paddybyers May 4, 2026
0e3b974
Fix RSA4c3 tests: auth failure while CONNECTED should not set errorRe…
paddybyers May 5, 2026
3f87a4b
Fix RTN24 test: connectionId is not inside connectionDetails
paddybyers May 5, 2026
09a3ac9
Fix TM4 test: spec requires constructors, not toJSON
paddybyers May 5, 2026
73f6568
Fix batch operation test mocks to match server response format (RSC24…
paddybyers May 5, 2026
ac8a387
Fix RSC15l mock, add proxy integration tests for REST fallback (RSC15…
paddybyers May 5, 2026
184c10f
Add comprehensive REST proxy integration tests for HTTP error handling
paddybyers May 6, 2026
71a0297
Add UTS test IDs to all tests and implement 50 missing test scenarios
paddybyers May 6, 2026
3eaa563
Align unit test endpoints with UTS specs
paddybyers May 7, 2026
b94191d
Run data-path integration tests with both JSON and msgpack (G1)
paddybyers May 8, 2026
eacbc38
Add RSL6a3 msgpack interoperability tests
paddybyers May 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@
"test:react": "vitest run",
"test:package": "grunt test:package",
"test:uts": "npm run build:node && mocha --no-config --require tsx/cjs 'test/uts/**/*.test.ts'",
"test:uts:unit": "npm run build:node && mocha --no-config --require tsx/cjs --ignore 'test/uts/**/proxy/**' --ignore 'test/uts/**/integration/**' 'test/uts/**/*.test.ts'",
"concat": "grunt concat",
"build": "grunt build:all && npm run build:react",
"build:node": "grunt build:node",
Expand Down
283 changes: 167 additions & 116 deletions test/uts/deviations.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion test/uts/realtime/integration/auth/token_renewal.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import {
} from '../sandbox';

describe('uts/realtime/integration/auth/token_renewal', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
2 changes: 1 addition & 1 deletion test/uts/realtime/integration/delta_decoding.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ function makeCountingDecoder() {
}

describe('uts/realtime/integration/delta_decoding', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
255 changes: 255 additions & 0 deletions test/uts/realtime/integration/helpers/proxy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
/**
* TypeScript helper for the Go test proxy.
*
* Wraps the proxy's REST control API to create sessions, add rules,
* trigger imperative actions, retrieve event logs, and clean up.
*
* The proxy binary is built and spawned automatically on first use
* via ensureProxy(). It is killed when the Node.js process exits.
*/

import { execSync, spawn, ChildProcess } from 'child_process';
import * as path from 'path';
import * as fs from 'fs';

const CONTROL_PORT = process.env.PROXY_CONTROL_PORT || '9100';
const PROXY_CONTROL_HOST = process.env.PROXY_CONTROL_HOST || `http://localhost:${CONTROL_PORT}`;
const PROXY_SRC = path.resolve(__dirname, '../../../../../../specification/uts/proxy');
const PROXY_BIN = path.join(PROXY_SRC, 'test-proxy');

let _proxyProcess: ChildProcess | null = null;
let _proxyEnsured = false;

const SANDBOX_REALTIME_HOST = 'sandbox-realtime.ably.io';
const SANDBOX_REST_HOST = 'sandbox-rest.ably.io';

let nextPort = 19000 + Math.floor(Math.random() * 1000);

function allocatePort(): number {
return nextPort++;
}
Comment on lines +31 to +35
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Stop hand-allocating session ports.

nextPort++ never checks or reserves the socket, so parallel workers or anything already bound in the 19000-19999 range can make createProxySession() fail nondeterministically. This needs OS-backed allocation, or proxy-side ephemeral port assignment, instead of a process-local counter.

Also applies to: 146-176

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 31 - 35, The
current allocatePort() uses a process-local counter (nextPort++) which can
collide across processes; replace it with an OS-backed ephemeral allocation by
creating a temporary socket bound to port 0 to let the OS pick a free port, read
the assigned port from the socket (e.g., via server.address().port), close the
socket and return that port, or change createProxySession() to ask the proxy to
bind an ephemeral port and return the chosen port; update the allocatePort()
caller(s) and any logic in createProxySession() to use this OS-assigned port
rather than relying on nextPort++.


interface ProxyRule {
match: {
type: string;
count?: number;
action?: string;
channel?: string;
method?: string;
pathContains?: string;
queryContains?: Record<string, string>;
delayMs?: number;
};
action: {
type: string;
closeCode?: number;
delayMs?: number;
message?: Record<string, any>;
status?: number;
body?: Record<string, any>;
headers?: Record<string, string>;
};
times?: number;
comment?: string;
}

interface ProxyEvent {
timestamp: string;
type: string;
direction?: string;
url?: string;
queryParams?: Record<string, string>;
message?: any;
method?: string;
path?: string;
status?: number;
initiator?: string;
closeCode?: number;
ruleMatched?: string | null;
headers?: Record<string, string>;
}

interface ImperativeAction {
type: string;
message?: Record<string, any>;
closeCode?: number;
}

class ProxySession {
readonly sessionId: string;
readonly proxyHost: string;
readonly proxyPort: number;
private controlUrl: string;

constructor(sessionId: string, proxyHost: string, proxyPort: number, controlUrl: string) {
this.sessionId = sessionId;
this.proxyHost = proxyHost;
this.proxyPort = proxyPort;
this.controlUrl = controlUrl;
}

async addRules(rules: ProxyRule[], position: 'append' | 'prepend' = 'append'): Promise<void> {
const resp = await fetch(`${this.controlUrl}/sessions/${this.sessionId}/rules`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ rules, position }),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`addRules failed (${resp.status}): ${body}`);
}
}

async triggerAction(action: ImperativeAction): Promise<void> {
const resp = await fetch(`${this.controlUrl}/sessions/${this.sessionId}/actions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(action),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`triggerAction failed (${resp.status}): ${body}`);
}
}

async getLog(): Promise<ProxyEvent[]> {
const resp = await fetch(`${this.controlUrl}/sessions/${this.sessionId}/log`);
if (!resp.ok) {
const body = await resp.text();
throw new Error(`getLog failed (${resp.status}): ${body}`);
}
const data = await resp.json();
return data.events || [];
Comment on lines +97 to +127
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Does Node.js global fetchhave a default request timeout, and how isAbortSignal.timeout() used to cap a request?

💡 Result:

No, the Node.js global fetch does not have a single default request timeout parameter in its API. Instead, the underlying Undici implementation enforces specific timeouts: headersTimeout (300 seconds / 5 minutes, time to receive headers), bodyTimeout (300 seconds, time between body chunks), and connectTimeout (around 10 seconds in some cases). These can cause requests to fail with errors like HeadersTimeoutError or ConnectTimeoutError if exceeded, even without an explicit user-set timeout. AbortSignal.timeout(ms) creates an AbortSignal that automatically aborts after the specified milliseconds with a TimeoutError. Pass it via the signal option to cap the entire request duration, overriding or complementing Undici's internal timeouts: try { const response = await fetch('https://example.com', { signal: AbortSignal.timeout(5000) // 5 second cap }); // handle response } catch (err) { if (err.name === 'TimeoutError') { console.log('Request timed out after 5s'); } } This is supported in Node.js since v17.3.0 (AbortSignal.timeout added v17.3.0/v16.14.0; stable fetch since v18). For finer control over Undici timeouts (e.g., increasing headersTimeout), use a custom Agent dispatcher: import { Agent } from 'undici'; await fetch(url, { dispatcher: new Agent({ headersTimeout: 600e3 }) });

Citations:


🏁 Script executed:

# Find the proxy.ts file and examine the ensureProxy method around line 263
find . -type f -name "proxy.ts" | head -5

Repository: ably/ably-js

Length of output: 105


🏁 Script executed:

# Check the file to understand the ensureProxy method and timeoutMs usage
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '250,270p'

Repository: ably/ably-js

Length of output: 671


🏁 Script executed:

# Also check the full context of ensureProxy method
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '250,300p'

Repository: ably/ably-js

Length of output: 1462


🏁 Script executed:

# Check other fetch calls mentioned (lines 132, 164-168, 206-208, 251-267)
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '1,50p'

Repository: ably/ably-js

Length of output: 1910


🏁 Script executed:

# Check the other fetch calls mentioned in the review
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '95,135p'

Repository: ably/ably-js

Length of output: 1731


🏁 Script executed:

# Check lines around 164-168 and 206-208
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '160,210p'

Repository: ably/ably-js

Length of output: 2223


Add per-request timeouts to all control-plane fetch calls.

These fetch() calls are unbounded. Each can hang for up to 5 minutes (Undici's headersTimeout), which defeats the startup timeout budget at line 282 (ensureProxy(timeoutMs)). While the loop measures wall-clock time, individual health probes ignore it—if localhost accepts a connection and stops responding, the suite hangs beyond the intended timeout.

Use AbortSignal.timeout(ms) to cap each request. Allocate a per-request timeout (e.g., 5–10 seconds for startup probes; similar for session operations) and let the startup loop handle retries within its budget.

Affects: addRules() (line 97), triggerAction() (line 109), getLog() (line 121), close() (line 132), createProxySession() (lines 164–168), downloadProxy() (line 206), and ensureProxy() health polling (line 266).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 97 - 127, All
control-plane fetch calls are unbounded and must be given per-request timeouts:
for each function named addRules, triggerAction, getLog, close,
createProxySession, downloadProxy and the health poll inside ensureProxy, create
an AbortSignal with AbortSignal.timeout(ms) and pass it as the signal option to
fetch; choose short sensible timeouts (e.g. 5–10s for startup/health probes,
similar or slightly longer for session operations/downloads) so each fetch can
abort quickly and let the outer retry/timeout logic (ensureProxy timeoutMs)
handle retries. Ensure you construct the signal per-request and cleanly let
fetch throw on abort so existing error handling (resp.ok checks and throws)
remains effective.

}

async close(): Promise<void> {
try {
await fetch(`${this.controlUrl}/sessions/${this.sessionId}`, { method: 'DELETE' });
} catch {
// Ignore errors during cleanup
}
}
}

interface CreateProxySessionOpts {
endpoint?: 'sandbox';
port?: number;
rules?: ProxyRule[];
timeoutMs?: number;
}

async function createProxySession(opts: CreateProxySessionOpts = {}): Promise<ProxySession> {
const port = opts.port || allocatePort();
const controlUrl = PROXY_CONTROL_HOST;

const target = {
realtimeHost: SANDBOX_REALTIME_HOST,
restHost: SANDBOX_REST_HOST,
};

const body: Record<string, any> = {
target,
port,
rules: opts.rules || [],
};
if (opts.timeoutMs) {
body.timeoutMs = opts.timeoutMs;
}

const resp = await fetch(`${controlUrl}/sessions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});

if (!resp.ok) {
const text = await resp.text();
throw new Error(`createProxySession failed (${resp.status}): ${text}`);
}

const data = await resp.json();
return new ProxySession(data.sessionId, 'localhost', port, controlUrl);
}

function buildProxy(): void {
if (!fs.existsSync(PROXY_SRC)) {
throw new Error(`Proxy source not found at ${PROXY_SRC}`);
}

const needsBuild = !fs.existsSync(PROXY_BIN) || fs.readdirSync(PROXY_SRC)
.filter((f) => f.endsWith('.go'))
.some((f) => fs.statSync(path.join(PROXY_SRC, f)).mtimeMs > fs.statSync(PROXY_BIN).mtimeMs);

if (needsBuild) {
execSync('go build -o test-proxy .', { cwd: PROXY_SRC, stdio: 'inherit' });
}
}

function spawnProxy(): ChildProcess {
const child = spawn(PROXY_BIN, ['--port', CONTROL_PORT], {
stdio: ['ignore', 'inherit', 'inherit'],
detached: false,
});

child.on('error', (err) => {
console.error(`Proxy process error: ${err.message}`);
});

process.on('exit', () => {
if (child.exitCode === null) {
child.kill();
}
});

return child;
}

async function ensureProxy(timeoutMs = 15000): Promise<void> {
if (_proxyEnsured) return;

// Check if proxy is already running (e.g. started externally)
try {
const resp = await fetch(`${PROXY_CONTROL_HOST}/health`);
if (resp.ok) {
_proxyEnsured = true;
return;
}
} catch {
// Not running — we'll start it
}

buildProxy();
_proxyProcess = spawnProxy();

const start = Date.now();
while (Date.now() - start < timeoutMs) {
try {
const resp = await fetch(`${PROXY_CONTROL_HOST}/health`);
if (resp.ok) {
_proxyEnsured = true;
return;
}
} catch {
// Not ready yet
}
await new Promise((r) => setTimeout(r, 200));
}

_proxyProcess.kill();
_proxyProcess = null;
throw new Error(`Proxy failed to start within ${timeoutMs}ms`);
Comment on lines +246 to +279
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Memoize proxy startup instead of racing it.

Two callers can pass Line 247 before _proxyEnsured flips, then both try to spawn on the same control port. The later call also overwrites _proxyProcess, which can leave the real proxy orphaned from stopProxy(). Please serialize this with a module-scoped in-flight promise.

Possible shape of the fix
 let _proxyProcess: ChildProcess | null = null;
 let _proxyEnsured = false;
+let _proxyEnsurePromise: Promise<void> | null = null;

 async function ensureProxy(timeoutMs = 15000): Promise<void> {
   if (_proxyEnsured) return;
+  if (_proxyEnsurePromise) return _proxyEnsurePromise;

-  // existing startup logic
+  _proxyEnsurePromise = (async () => {
+    // existing startup logic
+  })();
+
+  try {
+    await _proxyEnsurePromise;
+  } finally {
+    _proxyEnsurePromise = null;
+  }
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 246 - 279, The
ensureProxy function risks racing: two callers can pass the _proxyEnsured check
and both call spawnProxy, overwriting _proxyProcess and orphaning the real
process; fix by adding a module-scoped in-flight Promise (e.g.,
_ensureProxyPromise) that is set at the start of ensureProxy and returned if
present so concurrent callers wait on the same startup flow, ensure only the
first caller runs downloadProxy() and spawnProxy(), and clear/reset
_ensureProxyPromise and set/clear _proxyProcess/_proxyEnsured appropriately on
success or failure (killing the process and rejecting) so stopProxy() can
reliably reference the correct _proxyProcess.

}

async function waitForProxy(timeoutMs = 15000): Promise<void> {
await ensureProxy(timeoutMs);
}

function stopProxy(): void {
if (_proxyProcess && _proxyProcess.exitCode === null) {
_proxyProcess.kill();
_proxyProcess = null;
}
_proxyEnsured = false;
}

export { ProxySession, ProxyRule, ProxyEvent, ImperativeAction, createProxySession, waitForProxy, ensureProxy, stopProxy, allocatePort };
62 changes: 62 additions & 0 deletions test/uts/realtime/integration/helpers/run-proxy-tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
set -euo pipefail

# Runs proxy integration tests:
# 1. Builds the Go test proxy (if needed)
# 2. Starts it on the control port
# 3. Runs the mocha tests matching the proxy pattern
# 4. Kills the proxy on exit

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROXY_SRC="${SCRIPT_DIR}/../../../../../../specification/uts/proxy"
PROXY_BIN="${PROXY_SRC}/test-proxy"
CONTROL_PORT="${PROXY_CONTROL_PORT:-9100}"
MOCHA_ARGS="${@}"

# Build proxy if source is newer than binary
if [ ! -f "$PROXY_BIN" ] || [ "$(find "$PROXY_SRC" -name '*.go' -newer "$PROXY_BIN" 2>/dev/null | head -1)" ]; then
echo "Building test proxy..."
(cd "$PROXY_SRC" && go build -o test-proxy .)
fi

cleanup() {
if [ -n "${PROXY_PID:-}" ]; then
kill "$PROXY_PID" 2>/dev/null || true
wait "$PROXY_PID" 2>/dev/null || true
fi
}
trap cleanup EXIT

# Start proxy
echo "Starting test proxy on control port $CONTROL_PORT..."
"$PROXY_BIN" --port "$CONTROL_PORT" &
PROXY_PID=$!

# Wait for proxy to be ready
for i in $(seq 1 30); do
if curl -sf "http://localhost:${CONTROL_PORT}/health" > /dev/null 2>&1; then
echo "Proxy ready (PID $PROXY_PID)"
break
fi
if ! kill -0 "$PROXY_PID" 2>/dev/null; then
echo "Proxy process died unexpectedly"
exit 1
fi
sleep 0.2
done

if ! curl -sf "http://localhost:${CONTROL_PORT}/health" > /dev/null 2>&1; then
echo "Proxy failed to start within 6 seconds"
exit 1
fi

# Run proxy tests
export PROXY_CONTROL_HOST="http://localhost:${CONTROL_PORT}"
cd "$(dirname "$SCRIPT_DIR")/../../../.."

npx mocha --no-config --require tsx/cjs \
'test/uts/realtime/integration/proxy/**/*.test.ts' \
--timeout 60000 \
$MOCHA_ARGS

echo "Proxy tests complete."
2 changes: 1 addition & 1 deletion test/uts/realtime/integration/mutable_messages.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
} from './sandbox';

describe('uts/realtime/integration/mutable_messages', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
} from '../sandbox';

describe('uts/realtime/integration/presence/presence_lifecycle', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import {
} from '../sandbox';

describe('uts/realtime/integration/presence/presence_sync', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
Loading
Loading