MILAB-6145: !!!EXPERIMENTAL DON'T MERGE!!! add :pframes.build-query-wasm (opt-in wasm-backed buildQuery)#1685
Draft
blackcat wants to merge 4 commits into
Draft
MILAB-6145: !!!EXPERIMENTAL DON'T MERGE!!! add :pframes.build-query-wasm (opt-in wasm-backed buildQuery)#1685blackcat wants to merge 4 commits into
blackcat wants to merge 4 commits into
Conversation
🦋 Changeset detectedLatest commit: 750480e The changes in this PR will be included in the next version bump. This PR includes changesets to release 10 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request migrates the buildQuery function from a pure-Tengo implementation to a WASM-based implementation calling into the @milaboratories/pframes-rs-wasip2 component, and removes the Tengo-side unit tests. A critical issue was identified in the new implementation where the multiple return values of json.encode and json.decode are not handled correctly, which will cause runtime or compilation errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1685 +/- ##
==========================================
- Coverage 52.31% 52.22% -0.09%
==========================================
Files 273 273
Lines 16201 16218 +17
Branches 3523 3524 +1
==========================================
- Hits 8475 8470 -5
- Misses 6588 6610 +22
Partials 1138 1138 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…uery
Introduces a sibling library to `:pframes.build-query` that routes
through the `@milaboratories/pframes-rs-wasip2` wasm component, so the
SpecQueryJoinEntry tree is built by the rust source-of-truth
(pframes-rs/packages/spec/src/requests/build_query/{logic,request}.rs).
Public API mirrors the pure-Tengo lib exactly: `buildQuery(input) -> map`.
Why a separate lib rather than swapping the impl behind the same path:
`:pframes.build-query` is transitively imported by `:pframes.build-table`,
which is reachable from `workflow/index.lib.tengo`. Adding the wasm
import there bundles the pframes-rs-wasip2 wasm bytes into every block's
`.plj.gz`, which exposes a divergence between pl-middle-layer's cached
and legacy template upload paths (different content hashes for the
same wasm-bundled pack — confirmed by template-cache-v3.test.ts). Keep
the migration scoped: pure-Tengo entry point remains the default;
callers can opt in to the wasm-backed lib explicitly. The pl-middle-layer
template cache can be addressed separately.
Impl notes:
- Lazy-init the wasm api so renders that import the lib but never call
buildQuery pay no Store-allocation cost.
- Wasm-side serde requires `spec` on every linker/filter step's
ColumnIdAndSpec, but BuildQuery::execute() reads only columnId. Stamp
a synthetic placeholder spec on steps that omit it; matches the
tolerance of the pure-Tengo lib.
- Added build-query-wasm.tpl.tengo + a parallel "buildQuery (wasm lib)"
loop in build-query.test.ts so every fixture is cross-checked against
the SpecDriver byte-for-byte for both libs.
blackcat
force-pushed
the
MILAB-6145-buildQuery-wasm
branch
from
3211b9d to
4cb4fc0
Compare
blackcat
changed the title
Adds wasm.pframes-rs-inner — an outer template that delegates to wasm.pframes-rs-inner-body via render.create. The inner template's body makes the same assets.importWasm + frame.buildQuery call that wasm.pframes-rs.tpl.tengo makes directly. Tengo-builder bundles the same wasm bytes into the same pack. The hypothesis: per-render Wasm map the workflow controller builds at compileTemplate time is scoped to the outermost rendered template only; nested renders launched via render.create see an empty Wasm map even though tengo-builder declares the dep at the pack level. The direct test passes because it makes the wasm call in the outer template's own body. The nested test should fail with "alias … is not a wasm dependency of this template". Observed symptom in rarefaction (blocks/rarefaction adapted to call :pframes.build-query-wasm in wf.body): same error message, same call shape. If this test fails as predicted, the dep propagation through nested renders is the right level to fix.
…tion repro)" This reverts commit 785b3db.
blackcat
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@platforma-sdk/workflow-tengo:pframes.build-query-wasm— a sibling library to:pframes.build-querythat routes through the@milaboratories/pframes-rs-wasip2wasm component, so theSpecQueryJoinEntrytree is built by the rust source-of-truth (pframes-rs/packages/spec/src/requests/build_query/{logic,request}.rs).:pframes.build-queryexactly:buildQuery(input) -> map.:pframes.build-queryis unchanged and remains the entry point used by:pframes.build-table. Callers opt in to the wasm version by importing the new lib directly.Why a sibling library, not an in-place swap
Earlier attempts inlined the wasm call directly in
:pframes.build-query. That made every block transitively bundle the pframes-rs-wasip2 wasm bytes (viaworkflow/index.lib.tengo→:pframes.table-builder→:pframes.build-table.tpl→:pframes.build-query). The bundled.plj.gzthen exposed a divergence betweenpl-middle-layer's cached and legacy template upload paths —template-cache-v3.test.ts > cached and legacy templates deduplicate to same original (V3)failed deterministically because the two paths produce different content hashes for wasm-bundled packs.Scoping the change to an opt-in sibling library avoids that. The pl-middle-layer template cache can be addressed separately.
Impl details
buildQuerypay no Store-allocation cost. Within a render, repeated calls share one Store + Instance.specon every linker/filter step'sColumnIdAndSpec, butBuildQuery::execute()reads onlycolumnId. The new lib stamps a synthetic placeholder spec on steps that omit it; matches the tolerance of the pure-Tengo lib.Test plan
pnpm build --filter='@platforma-sdk/workflow-tengo' --filter='@platforma-sdk/workflow-tengo-tests'— green.build-query-wasm.tpl.tengo+ a parallelbuildQuery (wasm lib)loop intests/workflow-tengo/src/pframes/build-query.test.ts. Every existing fixture (direct column, qualified, single filter, linker chain, mixed paths, multi-filter right-fold, filter-then-linker, three-step chain) is now cross-checked against the wasmSpecDriverbyte-for-byte for both libs.