Skip to content
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
6cebb50
Add operation timeouts to sandbox provisioning and teardown
paddybyers May 3, 2026
6730b98
Increase integration test suite timeouts from 60s to 120s
paddybyers May 3, 2026
2953ebe
Fix RSL2b3 history time range test to use server timestamps
paddybyers May 3, 2026
89ad6a2
Restructure UTS tests to match spec repo unit/integration layout
paddybyers May 3, 2026
e0c1345
Add proxy integration test infrastructure and tests
paddybyers May 3, 2026
774ed58
Add tier 1+2 proxy integration tests
paddybyers May 3, 2026
f9f6166
Add ably-js tests for missing UTS spec coverage
paddybyers May 3, 2026
b61e070
Auto-launch Go test proxy from Node.js
paddybyers May 3, 2026
0302d6b
Add RSC15f test: expired fallback not resurrected by late in-flight s…
paddybyers May 3, 2026
743e8a3
Add RTN15a test variant for TCP close without WebSocket close frame
paddybyers May 4, 2026
910c107
Download uts-proxy binary from GitHub releases instead of building fr…
paddybyers May 4, 2026
7444c39
Remove deviations.md from repo
paddybyers May 4, 2026
0e3b974
Fix RSA4c3 tests: auth failure while CONNECTED should not set errorRe…
paddybyers May 5, 2026
3f87a4b
Fix RTN24 test: connectionId is not inside connectionDetails
paddybyers May 5, 2026
09a3ac9
Fix TM4 test: spec requires constructors, not toJSON
paddybyers May 5, 2026
73f6568
Fix batch operation test mocks to match server response format (RSC24…
paddybyers May 5, 2026
ac8a387
Fix RSC15l mock, add proxy integration tests for REST fallback (RSC15…
paddybyers May 5, 2026
184c10f
Add comprehensive REST proxy integration tests for HTTP error handling
paddybyers May 6, 2026
71a0297
Add UTS test IDs to all tests and implement 50 missing test scenarios
paddybyers May 6, 2026
3eaa563
Align unit test endpoints with UTS specs
paddybyers May 7, 2026
b94191d
Run data-path integration tests with both JSON and msgpack (G1)
paddybyers May 8, 2026
eacbc38
Add RSL6a3 msgpack interoperability tests
paddybyers May 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@
"test:react": "vitest run",
"test:package": "grunt test:package",
"test:uts": "npm run build:node && mocha --no-config --require tsx/cjs 'test/uts/**/*.test.ts'",
"test:uts:unit": "npm run build:node && mocha --no-config --require tsx/cjs --ignore 'test/uts/**/proxy/**' --ignore 'test/uts/**/integration/**' 'test/uts/**/*.test.ts'",
"concat": "grunt concat",
"build": "grunt build:all && npm run build:react",
"build:node": "grunt build:node",
Expand Down
273 changes: 157 additions & 116 deletions test/uts/deviations.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion test/uts/realtime/integration/auth/token_renewal.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import {
} from '../sandbox';

describe('uts/realtime/integration/auth/token_renewal', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
2 changes: 1 addition & 1 deletion test/uts/realtime/integration/delta_decoding.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ function makeCountingDecoder() {
}

describe('uts/realtime/integration/delta_decoding', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
294 changes: 294 additions & 0 deletions test/uts/realtime/integration/helpers/proxy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
/**
* TypeScript helper for the Go test proxy.
*
* Wraps the proxy's REST control API to create sessions, add rules,
* trigger imperative actions, retrieve event logs, and clean up.
*
* The proxy binary is downloaded from GitHub releases on first use
* via ensureProxy(). It is killed when the Node.js process exits.
*/

import { execSync, spawn, ChildProcess } from 'child_process';
import * as crypto from 'crypto';
import * as path from 'path';
import * as fs from 'fs';
import { pipeline } from 'stream/promises';

const PROXY_VERSION = 'v0.1.0';
const PROXY_REPO = 'ably/uts-proxy';

const CONTROL_PORT = process.env.PROXY_CONTROL_PORT || '9100';
const PROXY_CONTROL_HOST = process.env.PROXY_CONTROL_HOST || `http://localhost:${CONTROL_PORT}`;
const CACHE_DIR = path.resolve(__dirname, '../../../../../node_modules/.cache/uts-proxy', PROXY_VERSION);
const PROXY_BIN = path.join(CACHE_DIR, 'uts-proxy');

let _proxyProcess: ChildProcess | null = null;
let _proxyEnsured = false;

const SANDBOX_REALTIME_HOST = 'sandbox-realtime.ably.io';
const SANDBOX_REST_HOST = 'sandbox-rest.ably.io';

let nextPort = 19000 + Math.floor(Math.random() * 1000);

function allocatePort(): number {
return nextPort++;
}
Comment on lines +31 to +35
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Stop hand-allocating session ports.

nextPort++ never checks or reserves the socket, so parallel workers or anything already bound in the 19000-19999 range can make createProxySession() fail nondeterministically. This needs OS-backed allocation, or proxy-side ephemeral port assignment, instead of a process-local counter.

Also applies to: 146-176

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 31 - 35, The
current allocatePort() uses a process-local counter (nextPort++) which can
collide across processes; replace it with an OS-backed ephemeral allocation by
creating a temporary socket bound to port 0 to let the OS pick a free port, read
the assigned port from the socket (e.g., via server.address().port), close the
socket and return that port, or change createProxySession() to ask the proxy to
bind an ephemeral port and return the chosen port; update the allocatePort()
caller(s) and any logic in createProxySession() to use this OS-assigned port
rather than relying on nextPort++.


interface ProxyRule {
match: {
type: string;
count?: number;
action?: string;
channel?: string;
method?: string;
pathContains?: string;
queryContains?: Record<string, string>;
delayMs?: number;
};
action: {
type: string;
closeCode?: number;
delayMs?: number;
message?: Record<string, any>;
status?: number;
body?: Record<string, any>;
headers?: Record<string, string>;
};
times?: number;
comment?: string;
}

interface ProxyEvent {
timestamp: string;
type: string;
direction?: string;
url?: string;
queryParams?: Record<string, string>;
message?: any;
method?: string;
path?: string;
status?: number;
initiator?: string;
closeCode?: number;
ruleMatched?: string | null;
headers?: Record<string, string>;
}

interface ImperativeAction {
type: string;
message?: Record<string, any>;
closeCode?: number;
}

class ProxySession {
readonly sessionId: string;
readonly proxyHost: string;
readonly proxyPort: number;
private controlUrl: string;

constructor(sessionId: string, proxyHost: string, proxyPort: number, controlUrl: string) {
this.sessionId = sessionId;
this.proxyHost = proxyHost;
this.proxyPort = proxyPort;
this.controlUrl = controlUrl;
}

async addRules(rules: ProxyRule[], position: 'append' | 'prepend' = 'append'): Promise<void> {
const resp = await fetch(`${this.controlUrl}/sessions/${this.sessionId}/rules`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ rules, position }),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`addRules failed (${resp.status}): ${body}`);
}
}

async triggerAction(action: ImperativeAction): Promise<void> {
const resp = await fetch(`${this.controlUrl}/sessions/${this.sessionId}/actions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(action),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`triggerAction failed (${resp.status}): ${body}`);
}
}

async getLog(): Promise<ProxyEvent[]> {
const resp = await fetch(`${this.controlUrl}/sessions/${this.sessionId}/log`);
if (!resp.ok) {
const body = await resp.text();
throw new Error(`getLog failed (${resp.status}): ${body}`);
}
const data = await resp.json();
return data.events || [];
Comment on lines +97 to +127
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Does Node.js global fetchhave a default request timeout, and how isAbortSignal.timeout() used to cap a request?

💡 Result:

No, the Node.js global fetch does not have a single default request timeout parameter in its API. Instead, the underlying Undici implementation enforces specific timeouts: headersTimeout (300 seconds / 5 minutes, time to receive headers), bodyTimeout (300 seconds, time between body chunks), and connectTimeout (around 10 seconds in some cases). These can cause requests to fail with errors like HeadersTimeoutError or ConnectTimeoutError if exceeded, even without an explicit user-set timeout. AbortSignal.timeout(ms) creates an AbortSignal that automatically aborts after the specified milliseconds with a TimeoutError. Pass it via the signal option to cap the entire request duration, overriding or complementing Undici's internal timeouts: try { const response = await fetch('https://example.com', { signal: AbortSignal.timeout(5000) // 5 second cap }); // handle response } catch (err) { if (err.name === 'TimeoutError') { console.log('Request timed out after 5s'); } } This is supported in Node.js since v17.3.0 (AbortSignal.timeout added v17.3.0/v16.14.0; stable fetch since v18). For finer control over Undici timeouts (e.g., increasing headersTimeout), use a custom Agent dispatcher: import { Agent } from 'undici'; await fetch(url, { dispatcher: new Agent({ headersTimeout: 600e3 }) });

Citations:


🏁 Script executed:

# Find the proxy.ts file and examine the ensureProxy method around line 263
find . -type f -name "proxy.ts" | head -5

Repository: ably/ably-js

Length of output: 105


🏁 Script executed:

# Check the file to understand the ensureProxy method and timeoutMs usage
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '250,270p'

Repository: ably/ably-js

Length of output: 671


🏁 Script executed:

# Also check the full context of ensureProxy method
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '250,300p'

Repository: ably/ably-js

Length of output: 1462


🏁 Script executed:

# Check other fetch calls mentioned (lines 132, 164-168, 206-208, 251-267)
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '1,50p'

Repository: ably/ably-js

Length of output: 1910


🏁 Script executed:

# Check the other fetch calls mentioned in the review
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '95,135p'

Repository: ably/ably-js

Length of output: 1731


🏁 Script executed:

# Check lines around 164-168 and 206-208
cat -n test/uts/realtime/integration/helpers/proxy.ts | sed -n '160,210p'

Repository: ably/ably-js

Length of output: 2223


Add per-request timeouts to all control-plane fetch calls.

These fetch() calls are unbounded. Each can hang for up to 5 minutes (Undici's headersTimeout), which defeats the startup timeout budget at line 282 (ensureProxy(timeoutMs)). While the loop measures wall-clock time, individual health probes ignore it—if localhost accepts a connection and stops responding, the suite hangs beyond the intended timeout.

Use AbortSignal.timeout(ms) to cap each request. Allocate a per-request timeout (e.g., 5–10 seconds for startup probes; similar for session operations) and let the startup loop handle retries within its budget.

Affects: addRules() (line 97), triggerAction() (line 109), getLog() (line 121), close() (line 132), createProxySession() (lines 164–168), downloadProxy() (line 206), and ensureProxy() health polling (line 266).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 97 - 127, All
control-plane fetch calls are unbounded and must be given per-request timeouts:
for each function named addRules, triggerAction, getLog, close,
createProxySession, downloadProxy and the health poll inside ensureProxy, create
an AbortSignal with AbortSignal.timeout(ms) and pass it as the signal option to
fetch; choose short sensible timeouts (e.g. 5–10s for startup/health probes,
similar or slightly longer for session operations/downloads) so each fetch can
abort quickly and let the outer retry/timeout logic (ensureProxy timeoutMs)
handle retries. Ensure you construct the signal per-request and cleanly let
fetch throw on abort so existing error handling (resp.ok checks and throws)
remains effective.

}

async close(): Promise<void> {
try {
await fetch(`${this.controlUrl}/sessions/${this.sessionId}`, { method: 'DELETE' });
} catch {
// Ignore errors during cleanup
}
}
}

interface CreateProxySessionOpts {
endpoint?: 'sandbox';
port?: number;
rules?: ProxyRule[];
timeoutMs?: number;
}

async function createProxySession(opts: CreateProxySessionOpts = {}): Promise<ProxySession> {
const port = opts.port || allocatePort();
const controlUrl = PROXY_CONTROL_HOST;

const target = {
realtimeHost: SANDBOX_REALTIME_HOST,
restHost: SANDBOX_REST_HOST,
};

const body: Record<string, any> = {
target,
port,
rules: opts.rules || [],
};
if (opts.timeoutMs) {
body.timeoutMs = opts.timeoutMs;
}

const resp = await fetch(`${controlUrl}/sessions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});

if (!resp.ok) {
const text = await resp.text();
throw new Error(`createProxySession failed (${resp.status}): ${text}`);
}

const data = await resp.json();
return new ProxySession(data.sessionId, 'localhost', port, controlUrl);
}

const CHECKSUMS: Record<string, string> = {
'uts-proxy_darwin_amd64.tar.gz': 'eb8abf5eec7f7137cf9e7cb6ab6f45fd162303c242b4567ab9e354c4b9a4a4ff',
'uts-proxy_darwin_arm64.tar.gz': '845da80af7d5b1daacbdf30b34aff6ca1b2bb88c708065bdc5d9a636baf32a1f',
'uts-proxy_linux_amd64.tar.gz': '79f444c23362cc277d163deb243dc16063c74665ff63b8bd3e56789b9d9610c7',
'uts-proxy_linux_arm64.tar.gz': '7357e4605f19451d83bb419ee959537d6e95ca74b766721eae006d4171371030',
};

function assetName(): string {
const platform = process.platform === 'darwin' ? 'darwin' : 'linux';
const arch = process.arch === 'arm64' ? 'arm64' : 'amd64';
return `uts-proxy_${platform}_${arch}.tar.gz`;
}
Comment on lines +179 to +190
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

What values can Node.js process.platform return, and how is Windows identified?

💡 Result:

Node.js process.platform returns one of the following string values: 'aix', 'darwin', 'freebsd', 'linux', 'openbsd', 'sunos', or 'win32'. The value 'android' may also be returned if Node.js is built on Android (experimental support). Windows is identified by the value 'win32' (regardless of 32-bit or 64-bit architecture).

Citations:


Handle unsupported platforms explicitly.

The assetName() function at line 187 maps every non-darwin platform to linux. Since Windows is identified as win32 in Node.js, this will return a Linux artifact name on Windows and fail during spawn() execution. Add explicit handling for the linux platform and throw a clear error for unsupported systems (including win32) until matching artifacts are available.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 179 - 190, The
assetName() helper currently treats any non-darwin platform as linux which
returns wrong artifacts on Windows; update assetName() to explicitly handle
process.platform === 'linux' and process.platform === 'darwin' and throw a
descriptive Error for unsupported platforms (e.g., 'win32') so callers fail
fast; reference the assetName() function and the CHECKSUMS map when adding the
error message so it’s clear which platforms are supported.


async function downloadProxy(): Promise<void> {
if (fs.existsSync(PROXY_BIN)) return;

const asset = assetName();
const expectedHash = CHECKSUMS[asset];
if (!expectedHash) {
throw new Error(`No checksum for ${asset} — unsupported platform/arch`);
}

fs.mkdirSync(CACHE_DIR, { recursive: true });

const url = `https://github.com/${PROXY_REPO}/releases/download/${PROXY_VERSION}/${asset}`;
console.log(`Downloading uts-proxy ${PROXY_VERSION} (${asset})...`);

const resp = await fetch(url, { redirect: 'follow' });
if (!resp.ok || !resp.body) {
throw new Error(`Failed to download ${url}: ${resp.status} ${resp.statusText}`);
}

const tarball = path.join(CACHE_DIR, asset);
const fileStream = fs.createWriteStream(tarball);
// @ts-ignore — Node fetch body is a web ReadableStream; pipeline handles it in Node 18+
await pipeline(resp.body, fileStream);

const hash = crypto.createHash('sha256').update(fs.readFileSync(tarball)).digest('hex');
if (hash !== expectedHash) {
fs.unlinkSync(tarball);
throw new Error(`Checksum mismatch for ${asset}: expected ${expectedHash}, got ${hash}`);
}

execSync(`tar xzf ${JSON.stringify(asset)}`, { cwd: CACHE_DIR });
fs.chmodSync(PROXY_BIN, 0o755);
fs.unlinkSync(tarball);
}

function spawnProxy(): ChildProcess {
const child = spawn(PROXY_BIN, ['--port', CONTROL_PORT], {
stdio: ['ignore', 'inherit', 'inherit'],
detached: false,
});

child.on('error', (err) => {
console.error(`Proxy process error: ${err.message}`);
});

process.on('exit', () => {
if (child.exitCode === null) {
child.kill();
}
});

return child;
}

async function ensureProxy(timeoutMs = 15000): Promise<void> {
if (_proxyEnsured) return;

// Check if proxy is already running (e.g. started externally)
try {
const resp = await fetch(`${PROXY_CONTROL_HOST}/health`);
if (resp.ok) {
_proxyEnsured = true;
return;
}
} catch {
// Not running — we'll start it
}

await downloadProxy();
_proxyProcess = spawnProxy();

const start = Date.now();
while (Date.now() - start < timeoutMs) {
try {
const resp = await fetch(`${PROXY_CONTROL_HOST}/health`);
if (resp.ok) {
_proxyEnsured = true;
return;
}
} catch {
// Not ready yet
}
await new Promise((r) => setTimeout(r, 200));
}

_proxyProcess.kill();
_proxyProcess = null;
throw new Error(`Proxy failed to start within ${timeoutMs}ms`);
Comment on lines +246 to +279
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Memoize proxy startup instead of racing it.

Two callers can pass Line 247 before _proxyEnsured flips, then both try to spawn on the same control port. The later call also overwrites _proxyProcess, which can leave the real proxy orphaned from stopProxy(). Please serialize this with a module-scoped in-flight promise.

Possible shape of the fix
 let _proxyProcess: ChildProcess | null = null;
 let _proxyEnsured = false;
+let _proxyEnsurePromise: Promise<void> | null = null;

 async function ensureProxy(timeoutMs = 15000): Promise<void> {
   if (_proxyEnsured) return;
+  if (_proxyEnsurePromise) return _proxyEnsurePromise;

-  // existing startup logic
+  _proxyEnsurePromise = (async () => {
+    // existing startup logic
+  })();
+
+  try {
+    await _proxyEnsurePromise;
+  } finally {
+    _proxyEnsurePromise = null;
+  }
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/uts/realtime/integration/helpers/proxy.ts` around lines 246 - 279, The
ensureProxy function risks racing: two callers can pass the _proxyEnsured check
and both call spawnProxy, overwriting _proxyProcess and orphaning the real
process; fix by adding a module-scoped in-flight Promise (e.g.,
_ensureProxyPromise) that is set at the start of ensureProxy and returned if
present so concurrent callers wait on the same startup flow, ensure only the
first caller runs downloadProxy() and spawnProxy(), and clear/reset
_ensureProxyPromise and set/clear _proxyProcess/_proxyEnsured appropriately on
success or failure (killing the process and rejecting) so stopProxy() can
reliably reference the correct _proxyProcess.

}

async function waitForProxy(timeoutMs = 15000): Promise<void> {
await ensureProxy(timeoutMs);
}

function stopProxy(): void {
if (_proxyProcess && _proxyProcess.exitCode === null) {
_proxyProcess.kill();
_proxyProcess = null;
}
_proxyEnsured = false;
}

export { ProxySession, ProxyRule, ProxyEvent, ImperativeAction, createProxySession, waitForProxy, ensureProxy, stopProxy, allocatePort };
2 changes: 1 addition & 1 deletion test/uts/realtime/integration/mutable_messages.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
} from './sandbox';

describe('uts/realtime/integration/mutable_messages', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
} from '../sandbox';

describe('uts/realtime/integration/presence/presence_lifecycle', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import {
} from '../sandbox';

describe('uts/realtime/integration/presence/presence_sync', function () {
this.timeout(60000);
this.timeout(120000);

before(async function () {
await setupSandbox();
Expand Down
Loading
Loading