Skip to content

Add language SDKs (Python, Ruby, Go, Node, Haskell) over a JSON-over-FFI core#105

Open
domenkozar wants to merge 50 commits into
mainfrom
feat/language-sdks
Open

Add language SDKs (Python, Ruby, Go, Node, Haskell) over a JSON-over-FFI core#105
domenkozar wants to merge 50 commits into
mainfrom
feat/language-sdks

Conversation

@domenkozar

@domenkozar domenkozar commented Jun 11, 2026

Copy link
Copy Markdown
Member

What

Adds first-class SDKs for Python, Ruby, Go, Node.js, and Haskell so non-Rust apps inherit every secretspec provider, profile, chain, generation, and as_path behavior with no language-side resolution logic. All resolution stays in the Rust core; each SDK is a thin client over a single JSON-in / JSON-out boundary.

How it fits together

  • secretspec-ffi — a narrow C ABI cdylib exposing secretspec_resolve / secretspec_free / secretspec_abi_version, funneling through one resolve_json(&str) -> String boundary that catches panics and returns a uniform {"ok": …} envelope.
  • Core additions — a value-carrying resolve payload (secretspec resolve, prints JSON) and a value-free resolution report (secretspec check --json / --explain), both versioned with a canonical JSON Schema under schema/.
  • Shared codegen IR (secretspec::codegen) — the derive macro and every SDK's typed accessors are generated from one JSON Schema via quicktype, instead of per-language emitters.
  • The SDKs — Python (cffi), Ruby (Fiddle), Go (purego), Haskell (FFI) load the cdylib at runtime; Node uses a napi-rs addon that wraps the core directly and resolves off the event loop.
  • Cross-language conformance suite (conformance/) — shared fixtures every SDK must reduce to an identical canonical result, run inside each SDK's own test runner.
  • CI + packaging — per-ecosystem build/publish workflows (PyPI wheels, RubyGems, npm addon, Go libs, Hackage) and an aggregate SDK suite.

Review-driven hardening (final commits)

The branch closes the highest-impact findings from a multi-agent review of itself:

  • fix(core): the value-free surfaces (report(), no_values, check --json) are now side-effect-free. They no longer mint and store a brand-new secret in the provider for an unset generate secret, and no longer write as_path secrets to disk. A provider-chain primary outage now surfaces the provider error instead of silently reporting the secret as missing.
  • fix(go): TOCTOU-safe extraction into a per-user owner-only cache dir, loader panic recovery, zero-value Builder safety, and a switch to the system-library distribution model (git-LFS + the module proxy cannot ship a working library).
  • fix(haskell): the native, secret-bearing response is freed under mask/finally, closing an async-exception leak.

Test plan

  • Rust core: 300 unit/integration tests pass (incl. new regressions for the value-free and chain-error fixes).
  • SDK suites: Python 13, Ruby 12, Node 13, Go (build/vet/test), Haskell 9/9 all green, plus the cross-language conformance vectors.
  • cargo fmt / cargo clippy clean (no new lints); gofmt clean.

Known follow-ups (not blocking)

Lower-severity items remain tracked: Go loader retry on transient miss, Node stale-addon staleness check, schema honoring SECRETSPEC_PROFILE, a resolve --json doc-flag sweep, a stale Node README, a __proto__ key edge case, and Windows support (URI parsing, CI leg, purego.Dlopen compile).

🤖 Generated with Claude Code

domenkozar and others added 30 commits June 8, 2026 17:48
@domenkozar @claude
feat: add value-free resolution report via check --json/--explain
402b5a1 
Surface the resolution waterfall the resolver already computes as a stable,
versioned, machine-readable contract that never carries secret values. This is
phase 1 of polyglot language support: the per-secret provenance type every
later phase (FFI, codegen, SDKs) depends on, shipped standalone as the
check --explain/--json quick win.

- New public ResolutionReport / SecretResolution / ResolutionStatus types
  (schema_version 1), serde-serializable, with to_explain_string() and
  all_required_present() helpers.
- validate_audited now records per-secret provenance (status, the serving
  provider's credential-free URI, generated, default_applied, as_path) instead
  of discarding it; entries sorted by name for deterministic output.
- ValidatedSecrets and ValidationErrors carry the resolution and expose
  report(), so the report is available on both success and missing-required.
- check --json (versioned JSON) and check --explain (human trace) skip the
  prompt-for-missing flow and exit non-zero when a required secret is missing,
  so CI can gate on them. No secret values are ever printed.
- Canonical JSON Schema committed at schema/resolution-report.schema.json.
- Golden wire-format test plus an end-to-end provenance test through a real
  dotenv backend; CLI reference and CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add value-carrying resolve payload and secretspec resolve --json
91a8579 
Phase 2, slice 1: the authoritative value-carrying resolution output the C ABI
and other-language SDKs consume. The FFI crate (next slice) is a thin wrapper
over this, keeping resolution logic in one place.

- New Secrets::resolve() -> ResolveResponse, building on phase 1 provenance:
  per-secret value (or persisted temp-file path for as_path), source
  (provider/generated/default), and the serving provider's credential-free
  URI. On a missing required secret it returns an empty secrets map plus
  missing_required, mirroring the derive crate's load().
- New public ResolveResponse / ResolvedSecret / ResolvedSource types
  (schema_version 1, BTreeMap for deterministic key order) with is_ok() and
  without_values().
- secretspec resolve --json prints the payload (values to stdout, meant to be
  piped); --no-values emits the same structure value-free. Exits non-zero when a
  required secret is missing.
- Canonical JSON Schema at schema/resolve-response.schema.json; CLI reference
  and CHANGELOG updated; tests cover values, provenance, missing-required, and
  as_path path persistence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add secretspec-ffi crate exposing the narrow C ABI
f927037 
Phase 2, slice 2: the in-process boundary other-language SDKs bind to. A
deliberately narrow, JSON-in/JSON-out C ABI keeps every binding thin and keeps
resolution logic in the secretspec crate alone.

- Three-function surface: secretspec_resolve(request_json) -> response_json,
  secretspec_free(ptr), secretspec_abi_version(). Request and response are the
  versioned JSON contract; the response envelope separates transport failure
  ({"ok":false,"error":{kind,message}}) from a successful resolution
  ({"ok":true,"response":ResolveResponse}) that still reports missing_required.
- Panics are caught at the boundary (never unwind across FFI); returned strings
  are caller-owned and freed via secretspec_free; null and bad input are handled.
- Hand-written C header at secretspec-ffi/include/secretspec.h; crate builds as
  cdylib + staticlib + rlib.
- SecretSpecError::kind() promoted to pub for typed SDK error handling.
- Tests drive the real extern \"C\" entry points (values, no_values,
  missing-required, invalid input, missing manifest); a committed smoke.c plus a
  drafted per-platform ffi-build workflow build and smoke-test the cdylib on
  linux/macos/windows (native per-runner; portable packaging is follow-up).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add shared codegen IR (secretspec::codegen)
5f80f53 
Phase 3, slice 1: the single brain for typed-accessor generation. Every
generator (the Rust derive macro and the future TS/Python/Go/Ruby emitters)
needs the same manifest decisions; computing them in one place stops drift.

- build_ir(&Config) -> CodegenIr reduces a manifest to a language-neutral IR:
  project name, sorted profile list, the union field set (optional if optional-in
  or missing-from any profile, a path if as_path in any profile), and the
  per-profile raw (non-merged) field sets.
- Faithfully reproduces derive macro semantics, including the long-standing quirk
  that an unspecified `required` is treated as optional (differs from the runtime
  resolver) so generated output stays stable.
- IR types are serde-serializable for emitters and tooling. Unit tests cover
  union optionality, missing-in-profile, as_path-in-any, per-profile exactness,
  descriptions, and the empty-profiles default case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
refactor: drive the derive macro through the shared codegen IR
d8ed866 
Phase 3, slice 2: validate the IR against the known-good consumer and remove the
duplicated typing logic, so the derive macro and the future TS/Python/Go/Ruby
emitters share one brain.

- declare_secrets now calls secretspec::codegen::build_ir(&config) once and
  sources every typing decision from it: the union struct fields, per-profile
  enum variants, the Profile list, and the load_profile arms. The empty-profiles
  special case disappears because the IR already models it.
- Removed the derive's own is_secret_optional / is_field_optional_across_profiles
  / is_field_as_path / analyze_field_types / get_profile_variants; the token
  emitters now read optional/as_path straight off the IR (only the Rust type
  mapping stays local). Dropped the unit tests that covered those moved helpers
  (now tested in secretspec::codegen).
- Generated API is unchanged: 15 derive unit + 3 trybuild UI + 12 integration
  tests pass, and the example crate resolves through the generated builder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add secretspec-py, the Python reference SDK over the C ABI
07c9f03 
Phase 4: the first non-Rust consumer, proving the whole stack from a generic
dlopen caller (the same C ABI path Go/purego and Ruby/ffi will reuse).

- secretspec-py binds secretspec-ffi via cffi: marshals a JSON request to
  secretspec_resolve, parses the envelope, frees the buffer. No resolution logic
  in Python; every provider is inherited from the Rust core.
- Mirrors the derive crate's vocabulary: SecretSpec.builder().with_provider()
  .with_profile().with_reason().load() -> Resolved(.secrets/.provider/.profile),
  plus set_as_env(). MissingRequiredError vs SecretSpecError(.kind) separate a
  missing required secret from a transport failure. as_path yields a file path.
- Library discovery via SECRETSPEC_FFI_LIB, a wheel-bundled copy, or a Cargo
  target dir. pyproject packages it; README documents it.
- pytest suite (6 tests) drives the real cdylib end to end: values, default
  source, missing-optional, set_as_env, missing-required, as_path, invalid input.
  conftest builds the crate and locates the library automatically.
- devenv.nix now provides Python + cffi + pytest + maturin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add secretspec codegen --lang python (IR-driven typed accessors)
7dffc98 
Completes phase 4: the codegen half of the Python reference SDK, so typed
accessors and the runtime SDK ship together. The emitter is a thin template over
the shared codegen IR, so it cannot drift from the derive macro or future
language emitters.

- codegen::python::emit(&CodegenIr) -> String generates a module that mirrors
  the derive crate's shape over the runtime SDK: a SecretSpec union dataclass
  plus one <Profile>Secrets dataclass per profile, each with a builder-style
  load(). Idiomatic Python: snake_case attributes typed str / Optional[str] /
  Path, required pulled directly, optional guarded, as_path wrapped in Path.
- New `secretspec codegen --lang python [-o FILE]` CLI command (value-free;
  reads only the manifest via the now-non-test-gated Secrets::config()).
- Rust test asserts the emitted types/assignments; two Python e2e tests generate
  a module via the CLI, import it, and resolve through the generated accessors
  (union + profile-pinned, including as_path). conftest builds the CLI too.
- Generated code avoids `from __future__ import annotations` so it is robust when
  imported/exec'd in any context. CLI reference and CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add secretspec-go, the Go SDK via purego
2a3cbd1 
Phase 5: the Go binding over the same C ABI, reusing the generic dlopen path the
Python SDK validated (here in a static language, for the devops/k8s audience).

- secretspec-go binds secretspec-ffi via purego (dlopen, no cgo): marshals a
  JSON request to secretspec_resolve, reads the C string, frees it. No
  resolution logic in Go; every provider comes from the Rust core.
- Mirrors the derive vocabulary with idiomatic Go (PascalCase): New()
  .WithProvider().WithProfile().WithReason().Load() -> *Resolved with
  Provider/Profile/Secrets and SetAsEnv(). *MissingRequiredError vs *Error{Kind}
  separate a missing required secret from a transport failure. as_path yields a
  readable file path.
- Library discovery via SECRETSPEC_FFI_LIB or a Cargo target dir. Tests (gofmt
  clean) drive the real cdylib end to end via a TestMain that builds and locates
  it: abi version, values+provenance, missing-required, as_path, invalid input.
- devenv.nix now provides Go.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add secretspec-rb, the Ruby SDK via stdlib Fiddle
73c2eeb 
Phase 5: the Ruby binding over the same C ABI. Uses stdlib Fiddle (dlopen)
rather than the ffi gem, so there is no native gem build; same generic C ABI
path as Python/Go.

- secretspec-rb binds secretspec-ffi via Fiddle: marshals a JSON request to
  secretspec_resolve, reads the C string, frees it. No resolution logic in Ruby.
- Mirrors the derive vocabulary idiomatically:
  Secretspec::SecretSpec.builder.with_provider.with_profile.with_reason.load ->
  Resolved(#provider/#profile/#secrets) plus set_as_env!. MissingRequiredError
  vs Error(#kind) separate a missing required secret from a transport failure.
  as_path yields a readable file path.
- Library discovery via SECRETSPEC_FFI_LIB or a Cargo target dir. minitest suite
  (6 tests) drives the real cdylib end to end, building/locating it first.
- gemspec + README; devenv.nix now provides Ruby.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat: add secretspec Node.js/TypeScript SDK via koffi
0e8ad5f 
Phase 5: the Node binding over the same C ABI, via koffi (dlopen), keeping Node
on the identical generic C ABI path as Python/Go/Ruby. (napi-rs remains the
future production-distribution option; koffi keeps the reference uniform.)

- secretspec-node binds secretspec-ffi via koffi: marshals a JSON request to
  secretspec_resolve, decodes the C string, frees it. No resolution logic in JS.
- Mirrors the derive vocabulary idiomatically (camelCase):
  SecretSpec.builder().withProvider().withProfile().withReason().load() ->
  Resolved(provider/profile/secrets) plus setAsEnv(). MissingRequiredError vs
  SecretSpecError(.kind) separate a missing required secret from a transport
  failure. as_path yields a readable file path. TypeScript types in index.d.ts.
- Library discovery via SECRETSPEC_FFI_LIB or a Cargo target dir. node:test
  suite (6 tests) drives the real cdylib end to end, building/locating it first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
test: add cross-language conformance suite
5cc327d 
The capstone of phase 5: prove the Python, Go, Ruby, and Node SDKs agree. They
are all thin clients over one C ABI, so the risk is in each SDK's parsing and
exposure, not the resolver. This suite locks that down.

- conformance/fixtures/* are self-contained cases (manifest + .env +
  expected.json). Each SDK resolves them and projects its result to a canonical
  shape (profile, per-secret {value, source, as_path}, missing lists), then
  asserts equality with expected.json. For as_path secrets the canonical value
  is the materialized file's contents, so it is deterministic across languages.
- Each SDK runs the fixtures inside its own native runner (pytest, go test,
  minitest, node:test), reading conformance/ relative to the repo root. All four
  pass the same two fixtures (basic: provider + default + optional-missing;
  as_path).
- Fixtures cover successful resolutions; error behavior stays in per-SDK suites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
test: add aggregate cross-language conformance runner
e046a06 
conformance/run.sh builds the secretspec-ffi cdylib once, points every SDK at it
via SECRETSPEC_FFI_LIB (so they don't each rebuild), runs all four conformance
suites in their native runners, and prints a combined PASS/FAIL/SKIP summary.
Exits non-zero if any language fails; a missing toolchain is SKIP, not FAIL.
README documents it as the one-command entry point.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
refactor: generate typed shapes via JSON Schema + quicktype, not per-…
10b46a7 
…language emitters

Replaces the hand-written per-language codegen emitter approach (the
`codegen --lang python` command and the WIP Go/Ruby/TypeScript emitters) with a
single JSON Schema emitter, so we no longer maintain a typed-accessor generator
per language. quicktype turns the schema into idiomatic types AND deserializers
for any language; we maintain only a tiny generic `fields()` helper per SDK.

- `secretspec codegen --lang ...` becomes `secretspec schema`: emits a JSON
  Schema (draft-06) with a `SecretSpec` union type plus one `<Profile>Secrets`
  per profile, property names = secret names, optionals nullable. Driven by the
  same shared IR (so it can't drift from the derive macro).
- Each runtime SDK gains a generic `fields()` returning a flat
  `{SECRET_NAME: value}` map (the file path for as_path): Python/Ruby return the
  map, Go/Node also expose a JSON variant (FieldsJSON / fieldsJson) for
  quicktype's bytes/string deserializers.
- The loader the user writes is one line, e.g.
  `SecretSpec.from_dict(resolved.fields())`. quicktype owns naming, optionality,
  and converters for all current and future languages.
- Deleted codegen::{python,go,ruby,typescript} and their emitter tests; added a
  schema emitter test. Python e2e now drives the real pipeline:
  `secretspec schema | quicktype --lang python` then
  `SecretSpec.from_dict(resolved.fields())`. CLI reference + CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
ci: run all language SDK suites in CI
8e1b31d 
Adds a SDKs workflow that builds the cdylib + CLI once and runs every SDK's full
test suite (unit + cross-language conformance + the schema/quicktype codegen
pipeline) via scripts/ci-sdks.sh, so the Python/Go/Ruby/Node bindings cannot
silently rot. The prior CI (`devenv test` -> cargo test --all) covered only the
Rust crates, including secretspec-ffi, but none of the language SDKs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
docs: add Python/Go/Ruby/Node.js SDK pages and README section
4cdbd65 
The polyglot SDKs were undocumented (the docs site had only a Rust SDK page and
the README never mentioned them). Adds a docs page per SDK (quick start, error
model, the schema/quicktype typed-access pattern, library discovery), wires them
into the sidebar, and adds a "Language SDKs" section to the README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
build(python): package a platform wheel bundling the cdylib
e3788c4 
Phase 6 / pillar A, Python: make the Python SDK installable without a separate
native build by bundling the secretspec-ffi cdylib into the wheel. The SDK
already prefers a bundled secretspec/_lib/ over SECRETSPEC_FFI_LIB / a Cargo
target dir, so this wires up the build.

- scripts/stage-cdylib.sh builds the cdylib (release) and stages it into
  secretspec/_lib/ (gitignored) per OS, including the Windows .dll case.
- setup.py forces a platform (non-pure) wheel tagged py3-none-<platform> so pip
  installs the right native library per OS/arch; metadata stays in pyproject.
- Verified locally: the produced wheel is py3-none-linux_x86_64, contains
  secretspec/_lib/libsecretspec_ffi.so, and a clean install (no env var, outside
  the repo) loads the bundled lib and resolves a secret.
- Drafted python-wheels.yml: a per-platform matrix (linux x86_64/aarch64, macOS
  x86_64/aarch64, windows) that stages the cdylib, builds the wheel, and smoke
  tests it. NOTE: Linux wheels are tagged linux_*, not manylinux_*; PyPI
  publishing still needs an auditwheel-repair step to vendor the cdylib's system
  deps (libdbus from keyring) and make glibc portable. That is the remaining
  follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
build(ruby): package a platform gem bundling the cdylib
67b0471 
Phase 6 / pillar A, Ruby: make the Ruby SDK installable without a separate
native build by bundling the secretspec-ffi cdylib into a platform gem.

- The SDK now prefers a vendored vendor/<lib> (staged into a platform gem) over
  SECRETSPEC_FFI_LIB / a Cargo target dir.
- scripts/stage-cdylib.sh builds the cdylib (release) and stages it into
  vendor/ (gitignored) per OS, incl. the Windows .dll case.
- The gemspec includes vendor/* and sets Gem::Platform::CURRENT when the lib is
  staged, so `gem build` produces a platform gem (else a pure-Ruby gem).
- Verified locally: built secretspec-0.12.0-x86_64-linux.gem, and a clean
  install (no env var, outside the repo) loaded the bundled lib and resolved a
  secret. Existing Ruby suite still green.
- Drafted ruby-gems.yml: per-platform matrix that stages, builds, and smoke
  tests the gem. NOTE: like the wheels, a portable Linux gem needs a baseline
  build (e.g. rake-compiler-dock) to vendor the cdylib's system deps (libdbus)
  and glibc; that is the remaining follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
build(go): embed the cdylib via go:embed (behind embed_lib tag)
cf9d34e 
Phase 6 / pillar A, Go: let `go get` work with no native build by embedding the
secretspec-ffi cdylib per platform (go:embed) and extracting it to a temp file
at first use for purego to dlopen.

- Per-platform embedded_<os>_<arch>.go files (linux/darwin/windows x amd64/arm64)
  embed lib/secretspec_ffi_<os>_<arch>.<ext>; embedded.go extracts it to a
  content-addressed temp path. findLibrary prefers SECRETSPEC_FFI_LIB, then the
  embedded lib, then a Cargo target dir.
- Gated behind the `embed_lib` build tag: the default build (CI, `go test`, a
  plain checkout) compiles a nil-stub and needs no staged binary, so nothing
  breaks; only release/distribution builds pass `-tags embed_lib` with the
  libraries staged.
- scripts/stage-cdylib.sh builds + stages the lib under the build-tagged name.
  Verified locally: default `go test` green, and a tagged build outside the repo
  with no SECRETSPEC_FFI_LIB embeds the lib and resolves a secret.
- Drafted go-embed.yml (per-platform build + embedded smoke test). NOTE: the
  embedded libs are ~34 MB each; a release commits them via git-LFS (they are
  gitignored here) and flips embedding on by default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat(node): switch the Node SDK to a napi-rs addon
eb8bbeb 
Phase 6 / pillar A, Node: replace the koffi (dlopen) binding with a napi-rs
native addon that embeds the resolver, so `npm install` needs no cdylib or
SECRETSPEC_FFI_LIB. This is the standard way to ship a Rust-backed npm package.

- New `secretspec::resolve_json(&str) -> String` in the core: the shared
  JSON-in/JSON-out boundary (request -> response envelope). secretspec-ffi is
  refactored into a thin wrapper over it (and drops its serde deps), so the C
  ABI and the napi addon define the envelope contract in exactly one place.
- New secretspec-node-native crate (napi-rs) exposing resolve()/abiVersion()
  over resolve_json; a napi cdylib is a valid Node addon, so scripts/build-addon.sh
  is just `cargo build` + rename to secretspec.node.
- index.js now requires ./secretspec.node instead of koffi; the JS API
  (builder, Resolved, fields/fieldsJson, errors) is unchanged. Dropped the koffi
  and unused typescript deps; the package has no runtime npm dependencies.
- Test harness builds the addon instead of the cdylib; all 8 Node tests (incl.
  conformance) pass, and the full cross-language suite stays green.
- Drafted node-addon.yml (per-platform addon build + smoke test). NOTE: full npm
  distribution publishes per-platform addon packages (the pattern @napi-rs/cli
  automates); that publish wiring is the remaining follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
ci: wire per-ecosystem release/publish pipelines
792943f 
Phase 6 / pillar A follow-up: turn the per-platform build workflows into release
pipelines that produce portable artifacts and publish on a version tag, plus a
RELEASE.md runbook.

- Python (python-wheels.yml): build Linux wheels inside a manylinux_2_28
  container and repair with auditwheel (vendors the cdylib's libdbus/glibc),
  native macOS/Windows wheels, and publish to PyPI via Trusted Publishing (OIDC).
- Ruby (ruby-gems.yml): add a publish job that `gem push`es the platform gems
  (RUBYGEMS_API_KEY). Portable-Linux gem build noted as a follow-up.
- Go: add secretspec-go/.gitattributes so the embedded libs are git-LFS tracked
  when a release commits them.
- RELEASE.md documents each ecosystem's build approach, publish mechanism,
  required secrets, and known gaps (Ruby portable build; Go git-LFS + manual
  commit; Node multi-platform npm via @napi-rs/cli optional packages).

UNVALIDATED: these are cross-platform CI + registry-credential pipelines that
have not been run; they need a CI iteration and the documented repo secrets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
test: e2e the schema/quicktype codegen pipeline in all four SDKs
db26b01 
Closes the loose end where only Python had an automated codegen test (Go/Ruby/TS
were verified by hand). Each SDK now runs the full pipeline in its native runner:
secretspec schema -> quicktype -> typed deserializer over the SDK's fields().

- Schema emitter reworked to a single-root object (the union by default, or a
  profile's fields via `schema --profile`). quicktype only emits a converter for
  the ROOT type, so the previous Manifest wrapper / $ref root gave JS/Go no
  usable `toSecretSpec`/`UnmarshalSecretSpec` and mis-named the type. Pair with
  `quicktype --top-level SecretSpec`.
- New e2e tests: Go (temp module, UnmarshalSecretSpec(FieldsJSON)); Ruby
  (dry-struct, from_dynamic!(fields)); Node (quicktype --lang javascript,
  toSecretSpec(fieldsJson())); Python updated to --top-level. All gated on npx.
- ci-sdks.sh runs all Ruby test files; CLI docs + SDK pages + CHANGELOG updated
  for --top-level and --profile.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
docs: add an SDK overview page
0bd9ca8 
The SDK section jumped straight into per-language pages with no explanation of
how the polyglot stack works. Adds an Overview page (one Rust resolver, thin
clients over the C ABI / napi addon, the shared runtime API and error model,
typed access via schema+quicktype, and the bundled-library distribution model)
and wires it as the first item in the SDK sidebar. Docs site builds clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
docs: add a Language SDKs block to the landing page
ee404ed 
The landing page only showcased the Rust SDK. Adds a "Use it from any language"
showcase section after it, with the shared builder API in Python, Node.js, Go,
and Ruby, plus the one-resolver/thin-client framing and links to the SDK
overview and the schema+quicktype typed-access path. Landing page builds clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
docs: merge the Rust and Language SDKs landing sections into one
4156050 
The landing page had a standalone "Compile-time secrets in Rust" section
adjacent to the new "Language SDKs" section, so Rust read as separate from the
SDKs when it is one of them. Folded the Rust main.rs example into the single
Language SDKs section as its compile-time highlight, after the Python/Node/Go/
Ruby snippets. One coherent SDK section; landing page builds clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(core): align per-profile schema, harden resolve_json, dedupe codegen
082a59e 
- Per-profile JSON Schema (`schema --profile <p>`) now allows additional
  properties: `resolve --profile <p>` returns the profile's own secrets plus
  those inherited from `default`, which the per-profile type intentionally does
  not list (matching the derive macro), so a strict quicktype deserializer would
  otherwise reject a valid resolve result. The union schema stays exhaustive.
- `resolve_json` now catches panics itself, so both native boundaries that
  funnel through it (the C ABI and the napi-rs Node addon) return the same
  `{"ok":false,"error":...}` envelope on an internal panic.
- `secretspec::codegen` exposes one shared `capitalize`, used by both the schema
  emitter and the derive macro (was a byte-identical copy in each), so profile
  type-name casing can never drift.
- `build_ir` computes the union field set in a single pass instead of
  re-scanning every profile per field; `validate`/`resolve` resolve each
  secret's merged config once per pass instead of twice.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(go): validate response, skip null env, verify embedded cdylib
a4ed138 
- Load nil-checks the response and validates `schema_version` against the
  version this SDK was built for, so a skewed library is reported rather than
  nil-panicked or silently misparsed.
- SetAsEnv skips secrets with no usable value (e.g. under no_values) instead of
  exporting an empty string, via a new `usable()` helper.
- extractEmbedded uses an owner-only (0o700) temp dir and reuses the cached
  cdylib only when its content hash matches, not just its size, closing a
  predictable-path load and a stale-file reuse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(python): validate response, lock dlopen, builder parity
d7828f1 
- _resolve_response checks the response is present and validates schema_version,
  raising SecretSpecError on mismatch instead of KeyError / silent misparse.
- _load uses double-checked locking so concurrent first callers do not race to
  dlopen.
- Dropped the divergent `source = "provider"` default (other SDKs pass it
  through); added with_no_values to the builder for parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(ruby): skip null env, validate response, lock dlopen, builder parity
b9a994b 
- set_as_env! skips secrets with no usable value instead of `ENV[name] = nil`,
  which would delete the variable.
- load nil-checks the response and validates schema_version, raising
  Secretspec::Error on mismatch.
- ensure_loaded guards the one-time dlopen with a Mutex and re-checks @loaded
  inside the lock.
- Added with_no_values to the builder for parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(node): validate response, skip null env, builder parity
458a6e9 
- load nil-checks the response and validates schema_version, throwing
  SecretSpecError on mismatch.
- setAsEnv skips secrets with no usable value instead of coercing null to the
  string "null".
- Added withNoValues to the builder (and index.d.ts) for parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(conformance): map the Windows cdylib name in the runner
49dd2a6 
uname under git-bash/msys reports MINGW*/MSYS*/CYGWIN*; map those to
secretspec_ffi.dll so the cross-language conformance gate can run on Windows,
where the FFI artifact already ships.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
domenkozar and others added 11 commits June 9, 2026 16:50
@domenkozar @claude
fix(core): redact credentials from the reported provider URI
cbe4633 
validation_report_provider_uri returned the override and per-secret alias
URIs verbatim, and this branch newly serializes that into the provider field
of the resolution report (check --json/--explain) and the resolve response
(resolve --json, every SDK's response.provider). A user-authored alias or
--provider override embedding a credential (vault+token:s3cr3t@host,
vault://host?token=...) therefore leaked into machine-readable output and
across the FFI boundary, even though the sibling source_provider and the warn
path already redact it.

Route both raw returns through redact_uri_strict; add a regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
perf(node): resolve off the event loop via loadAsync
ce5ad93 
The napi resolve binding was synchronous, so resolving from a network-backed
provider (1Password, LastPass) blocked the Node event loop for the whole
round-trip. Add a resolveAsync binding that runs resolve_json on the libuv
threadpool (napi AsyncTask) and a Builder.loadAsync() that awaits it. The
synchronous load() is unchanged; loadAsync() reports a clear error against an
older addon that lacks the binding.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(sdks): load the most recently built cdylib, not always release
e78ef49 
When SECRETSPEC_FFI_LIB is unset, the Go, Python, and Ruby SDKs walk up to a
Cargo target/ directory to find the library. They preferred release over
debug, so a stale release build silently shadowed the debug build a developer
had just produced (surfacing later as a confusing schema-version mismatch).
Within the nearest target/, pick the candidate with the newest mtime instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
docs: correct the Node "no native build" claim; changelog the fixes
54f0a60 
package.json ships no prebuilt addon and has no per-platform publish wiring
(the CI workflow flags this as a follow-up), so the "npm install needs no
native build" claim in the changelog and SDK docs was unbacked. Reword to say
the addon is built from the Rust core via scripts/build-addon.sh and that
prebuilt per-platform npm packages are a follow-up. Also record the credential
redaction, loadAsync, and cdylib-discovery fixes under Unreleased.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(core): resolve no_values without materializing values; add report()
8c6c6c1 
no_values now routes through a new Secrets::resolve_without_values, which never
exposes a secret value or persists an as_path temp file, so no secret byte
crosses the boundary and as_path resolution leaves nothing on disk. Previously
the resolver fully materialized every value (and persisted every as_path temp
file) and only then stripped them.

Adds Secrets::report() and a mode:"report" request on the shared resolve_json
boundary: a value-free ResolutionReport (per-secret status and provenance) that,
unlike resolve, reports a missing required secret as a status rather than failing
the call. This is the inventory/preflight view the CLI exposes as check --json,
now reachable from every language SDK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(sdks): null fields() consistency, as_path cleanup, and report()
f11fbae 
- Go Fields()/FieldsJSON() now emit JSON null for a value-less secret instead of
  the empty string "", matching Python/Ruby/Node; Fields() returns
  map[string]*string and a new Usable() distinguishes absent from empty. Node
  index.d.ts types fields() as Record<string, string | null>.
- Every SDK gains a cleanup affordance for the persisted as_path temp files: Go
  Resolved.Close(), Python close()/context manager, Ruby close()/load block,
  Node dispose()/Symbol.dispose.
- Every SDK gains report() (Node also reportAsync()) over the value-free report,
  which never fails on a missing required secret.
- Conformance gains no_values and report dimensions (the latter asserts
  source_provider presence), locking the cross-language contract so a divergence
  like the Go ""-vs-null one cannot ship again.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
feat(haskell): add the secretspec-hs SDK
79ab0e3 
A thin client over the secretspec-ffi C ABI, linked at build time via the GHC
FFI. Mirrors the other SDKs: a builder (withPath/withProvider/withProfile/
withReason/withNoValues) plus load/report, returning a Resolved
(get/fields/fieldsJson/setAsEnv/close) or a value-free Report. A missing required
secret throws MissingRequiredError; other failures throw SecretSpecError with a
stable errorKind. as_path secrets come back as a readable file path.

Wires GHC into devenv, the cross-language conformance runner (all three
dimensions), and ci-sdks.sh; adds a schema -> quicktype -> typed codegen e2e test
(quicktype's Haskell target); and a Haskell SDK CI workflow that builds/tests on
PR and publishes to Hackage on a version tag. Adds docs (SDK page, sidebar,
overview, landing) and README entries. The cdylib is linked at build time
(--extra-lib-dirs) and must be on the runtime loader path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@domenkozar @claude
fix(core): make value-free resolve/report side-effect-free and surfac…
6af14bb 
…e chain errors

The value-free surfaces (Secrets::report(), resolve_without_values(), the FFI
no_values/report requests, and check --json/--explain) ran the full resolution,
so they minted and stored a brand new secret in the provider for any declared
generate secret that was not yet set, and failed outright on a read-only
provider. Thread a Materialize flag through validate_audited so those entry
points share the identical resolution logic but skip its two side effects: a
generatable-but-absent secret is reported as it would resolve (generated)
without being created, and no as_path secret is written to a temp file.

Also: a per-secret provider chain whose primary provider errors and whose
fallback chain has no value now surfaces that provider error, exactly as a
single-provider failure already did, instead of silently reporting the secret
as missing_required, so machine consumers can tell an outage from an
unprovisioned secret.

Route check --json/--explain through report() (removing a duplicated inline
copy of its construction).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@domenkozar @claude
fix(go): harden the SDK and switch to the system library distribution…
65c0f7f 
… model

Three robustness fixes plus a distribution correction:

* Embedded (-tags embed_lib) cdylibs are extracted into a per-user, owner-only
  cache directory (os.UserCacheDir) whose privacy is verified before use,
  instead of a predictably named directory under the shared system temp dir.
  This closes a local attacker file swap (TOCTOU) that could run attacker code
  in the process on a shared host, and avoids noexec temp mounts. An embedded
  git LFS pointer (from a botched release) is rejected with a clear error rather
  than fed to dlopen.

* A missing symbol in the loaded library no longer panics: purego.RegisterLibFunc
  panics are recovered and returned as a load error, so an incompatible cdylib
  does not escape sync.Once and leave the loader poisoned with nil pointers.

* A zero-value Builder (var b Builder, not via New()) no longer panics with a
  nil-map write in its WithX setters; the request map initializes lazily.

Distribution moves to the system library model: git LFS plus the module proxy
cannot ship a working library (the proxy serves LFS pointer text), so it is no
longer prescribed. Consumers provide the cdylib via SECRETSPEC_FFI_LIB or vendor
it for an embed_lib build. RELEASE.md, .gitattributes, .gitignore, the workflow,
README, and docs updated accordingly.

Tests release as_path temp files so repeated runs leave no secret files behind.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@domenkozar @claude
fix(haskell): free the native response under mask/finally
9fbd7dd 
callNative copied the response out and freed it in straight-line IO, so an
asynchronous exception (e.g. a System.Timeout.timeout around load/report)
arriving between the call returning and the free leaked the native, secret
bearing response buffer. Install the free under mask and run it via finally so
it always executes.

The conformance test now closes its value-carrying Resolved so as_path temp
files do not accumulate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@domenkozar @claude
test(sdks): release as_path temp files in value-carrying tests
ee59238 
The value-carrying as_path and conformance tests in the Python, Ruby, and Node
suites resolved the as_path fixture but never disposed the result, so each run
left another 0400 secret-bearing temp file behind (only the no_values variants
cleaned up). Close/dispose the result, matching the no_values tests: Python via
try/finally close(), Ruby via the block form of load that closes in ensure, and
Node via try/finally dispose().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs		 secretspec 65a2cc2 Commit Preview URL

Branch Preview URL		 Jun 12 2026, 04:49 PM

domenkozar and others added 9 commits June 11, 2026 13:51
@domenkozar @claude
refactor(cli): drop the resolve subcommand, keep the contract FFI-only
0db108b 
The value-carrying ResolveResponse stays the SDK boundary, but only over
the secretspec-ffi C ABI. Not shipping it as a CLI verb keeps the
command surface verb-level auditable: check never prints a value, get
prints exactly one (per-key audited and reason-gated), and bulk value
extraction never becomes a pipeable plaintext artifact. Adding the
subcommand back later is backwards compatible; removing it after people
script against it would not be.

Secrets::resolve()/resolve_without_values()/resolve_json(), the FFI
crate, the SDKs, and the committed JSON Schema are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
ci: pin Rust via rust-toolchain.toml, scope artifact PR triggers, giv…
0d43614 
…e review devenv

Pin the Rust version in a single rust-toolchain.toml consumed by both
devenv (languages.rust.toolchainFile) and the native CI runners via
rustup, so released artifacts build with the same compiler CI tests
against.

Scope the artifact workflows (ffi, node, python, ruby, go) to PR changes
in their own directories; core resolver changes are already verified on
PRs by the devenv-based test.yml and sdks.yml, and the full matrices
still run on tags and manual dispatch.

Install devenv in the Claude review workflow and allowlist devenv
commands so the reviewer can build and run tests instead of reviewing
blind.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
ci(sdks): free runner disk space before the workspace build
d08f967 
The SDKs job fills the ~14GB free on a hosted ubuntu runner (Nix store +
full debug build with the AWS/GCP/Bitwarden provider stacks) and dies
with ENOSPC, so drop the preinstalled toolchains we never use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
ci(haskell): free runner disk space before the workspace build
d069295 
Same ENOSPC as the SDKs job: the cdylib + CLI debug build with the full
provider stacks plus the Nix store overflows the hosted runner's disk,
this time so badly the runner could not even write its own logs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
ci: replace retired macos-13 runners with macos-15-intel
eed82a8 
The macos-13 label is no longer provisioned by GitHub, so every
x86_64-apple-darwin artifact job sat queued forever while the rest of
the matrix passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
fix(windows): support drive-letter provider paths and Windows SDK builds
5e1be3e 
Three Windows breakages surfaced by the first complete artifact CI runs:

- Provider specs like dotenv://C:\path\.env failed with "invalid port
  number" because C: parsed as a URL host:port. Drive-letter paths are
  now carried in the URL path component (forward-slash separators) and
  the dotenv provider strips the URL's leading slash from them.
- The Go SDK never compiled on Windows: purego.Dlopen and RTLD_* exist
  only on Unix. The library open is now split into build-tagged files,
  using syscall.LoadLibrary on Windows.
- The Node codegen test spawned npx, which is npx.cmd on Windows and
  unreachable without a shell.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
ci: drop Intel macOS artifacts, move arm macOS legs to macos-latest
de2713a 
The Go SDK segfaults in the purego call path on x86_64 darwin and Apple
has moved the platform on; stop building Intel macOS artifacts (FFI
cdylib, wheels, gems, Node addon, Go embed lib) rather than debugging a
dying target. Intel mac users can still build from source via the
system-library path.

Pin the remaining macOS runners to macos-latest instead of macos-14 so
the next image retirement does not silently strand jobs in the queue
like macos-13 did; artifact compatibility comes from rustc's deployment
target, not the runner OS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
fix(node): install the addon atomically and prebuild it in the SDKs CI
21c7a3d 
node --test runs the three test files in parallel processes; each one's
ensureAddon() kicked off build-addon.sh when secretspec.node was absent,
and the final `cp` truncated the addon in place while a sibling process
could already have it mapped, killing it with SIGBUS (seen once in the
SDKs workflow). Install via temp file + rename so an existing mapping
keeps its inode, and build once up front in ci-sdks.sh so the test
processes never race to build at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@domenkozar @claude
ci(python): drop the stale UNVALIDATED note, the wheel pipeline has r…
65a2cc2 
…un green

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@domenkozar