Cross Architecture Node/Relationship Reference Uniqueness

[LeighFinegold]

CALM Schema Change Proposal

Target Schema:

• core.json - node definition, relationship definition, and connects-type
• flow.json - transition definition (the relationship-unique-id field)
• Potentially a new reference.json or additions to core.json#/defs to define the cross-document reference format

Description of Change:

Introduce a cross-architecture reference mechanism so that a node or relationship defined in one CALM document can be uniquely addressed from another CALM document, without duplicating its definition.

Today, CALM's unique-id field on both nodes and relationships is document-scoped. Two separate architecture documents can both define a node with unique-id: "web-gui" or a relationship with unique-id: "web-gui-uses-accounts" and CALM has no way to say these are the same element or different elements. When composing architectures (for example, building an ecosystem-level view from multiple system-level documents) there is no formal identity link across document boundaries.

This applies equally to nodes and relationships. A flow transition references a relationship-unique-id, but if that relationship lives in a different document there is no way to address it. An ecosystem document may want to reference a relationship already defined in a child system document rather than redefining it.

This was identified as a gap during the original ecosystem/flows discussion in #650, where the question was asked:

"What happens if I need to reference a relationship between services of nodes across systems? Where is that relationship defined?"

The workaround proposed at the time was to re-declare the external node locally with a detailed-architecture URL pointing back to its origin. This works but introduces duplication and fragility: the local stub must be kept in sync with the source, and there is no schema-level mechanism to enforce or validate the link.

The same gap compounds the flows discussion in #1876, where externalising flows from core.json requires transitions to reference relationships that may live in a different document. The relationship-url concept proposed there (e.g. ./architecture.json#ui-to-service) is a specific instance of the broader cross-document reference problem.

Use Cases:

1. Ecosystem composition - An ecosystem-level CALM document groups multiple system-level architectures. The ecosystem document needs to define relationships between nodes that are owned by different system documents, without re-declaring those nodes in full.

2. Cross-system flows - A business flow (e.g. "Submit Trade") traverses nodes and relationships that span multiple architecture documents. Today, either the flow must live inside one document (which doesn't own all the nodes), or the nodes must be duplicated. A reference mechanism allows flow transitions to point at relationships across documents.

3. CALM Hub traversal - CALM Hub needs to discover and traverse the graph of related architecture documents. A formal reference convention enables tooling to resolve references, build dependency graphs, and generate consolidated documentation sites.

4. Shared infrastructure nodes - An API gateway, message broker, or database cluster may appear in many team-level architectures. A reference mechanism lets each team point at a canonical definition rather than maintaining independent copies.

5. Incremental architecture - Teams can author their own system-level documents independently. A platform team can compose them into an ecosystem view using references, without requiring coordination on unique-id values.

6. Cross-moment references in timelines - A timeline defines moments, each with its own detailed architecture. A reference mechanism would allow a node in one moment to be identified as the same node in another moment, enabling tooling to track how a node evolves across the timeline (e.g. "this service in moment 2 is the same service that existed in moment 1, but with different properties").

Current Limitations:

1. unique-id is document-scoped for both nodes and relationships - There is no namespace or document-qualified form. Two documents can use the same unique-id for unrelated nodes or relationships, or different unique-ids for the same logical element, with no way to disambiguate.

2. detailed-architecture is one-way and unstructured - A node can point to a more detailed document via details.detailed-architecture, but this is a plain string with no defined resolution semantics. There is no reverse link, no way to reference a specific node within the target document, and no validation that the target exists.

3. Relationships can only reference local nodes - connects, composed-of, interacts, and deployed-in all reference nodes by unique-id, which must exist in the same document. To model a cross-system connection, the external node must be re-declared locally.

4. Flow transitions can only reference local relationships - relationship-unique-id in flow.json is a bare string assumed to resolve within the same document. The relationship-url concept from Allow flows array to be a reference to a document referencing a flow #1876 would address this for flows, but needs a general reference convention to build on.

Proposed Schema Changes:

The exact format is open for discussion. All options below use the same scenario to illustrate the problem and solution:

The scenario: Two system-level architecture documents exist independently. The Trading System owns web-gui. The Ref Data System owns accounts-service. We need to express that web-gui connects to accounts-service across these two documents.

trading-system.arch.json (owned by Trading team):

{
  "$schema": "https://calm.finos.org/draft/2026-03/meta/calm.json",
  "nodes": [
    {
      "unique-id": "trading-system",
      "node-type": "system",
      "name": "Trading System",
      "description": "Simple Trading System"
    },
    {
      "unique-id": "web-gui",
      "node-type": "service",
      "name": "Web GUI",
      "description": "Allows employees to manage accounts and book trades"
    }
  ],
  "relationships": [
    {
      "unique-id": "trading-system-composed-of",
      "relationship-type": {
        "composed-of": {
          "container": "trading-system",
          "nodes": ["web-gui"]
        }
      }
    }
  ]
}

refdata-system.arch.json (owned by Ref Data team):

{
  "$schema": "https://calm.finos.org/draft/2026-03/meta/calm.json",
  "nodes": [
    {
      "unique-id": "refdata-system",
      "node-type": "system",
      "name": "Ref Data System",
      "description": "Account and Product Reference Data"
    },
    {
      "unique-id": "accounts-service",
      "node-type": "service",
      "name": "Accounts Service",
      "description": "Provides account lookup"
    }
  ],
  "relationships": [
    {
      "unique-id": "refdata-system-composed-of",
      "relationship-type": {
        "composed-of": {
          "container": "refdata-system",
          "nodes": ["accounts-service"]
        }
      }
    }
  ]
}

Now: how do we express that web-gui (Trading System) connects to accounts-service (Ref Data System)?

---

Option A: Node stub with source reference

The consuming document re-declares the external node as a minimal stub, adding a ref-source field that tells tooling where the canonical definition lives. Relationships then use local unique-ids as normal.

refdata-system.arch.json with cross-reference:

{
  "$schema": "https://calm.finos.org/draft/2026-03/meta/calm.json",
  "nodes": [
    {
      "unique-id": "refdata-system",
      "node-type": "system",
      "name": "Ref Data System",
      "description": "Account and Product Reference Data"
    },
    {
      "unique-id": "accounts-service",
      "node-type": "service",
      "name": "Accounts Service",
      "description": "Provides account lookup"
    },
    {
      "unique-id": "web-gui",
      "node-type": "service",
      "name": "Web GUI",
      "description": "Trading System's Web GUI",
      "ref-source": "./trading-system.arch.json#web-gui"
    }
  ],
  "relationships": [
    {
      "unique-id": "refdata-system-composed-of",
      "relationship-type": {
        "composed-of": {
          "container": "refdata-system",
          "nodes": ["accounts-service"]
        }
      }
    },
    {
      "unique-id": "web-gui-uses-accounts",
      "relationship-type": {
        "connects": {
          "source": { "node": "web-gui" },
          "destination": { "node": "accounts-service" }
        }
      }
    }
  ]
}

The ref-source field (./trading-system.arch.json#web-gui) tells validators and tooling: "this node is not defined here; the canonical definition is the node with unique-id: web-gui in trading-system.arch.json." Tooling can resolve and validate it. The stub still needs name and node-type so the document renders standalone, but the source of truth is elsewhere.

---

Option B: Qualified node references directly in relationship fields

Instead of re-declaring the external node at all, allow the relationship itself to reference nodes in other documents using a <document>#<unique-id> format:

refdata-system.arch.json with cross-reference:

{
  "$schema": "https://calm.finos.org/draft/2026-03/meta/calm.json",
  "nodes": [
    {
      "unique-id": "refdata-system",
      "node-type": "system",
      "name": "Ref Data System",
      "description": "Account and Product Reference Data"
    },
    {
      "unique-id": "accounts-service",
      "node-type": "service",
      "name": "Accounts Service",
      "description": "Provides account lookup"
    }
  ],
  "relationships": [
    {
      "unique-id": "refdata-system-composed-of",
      "relationship-type": {
        "composed-of": {
          "container": "refdata-system",
          "nodes": ["accounts-service"]
        }
      }
    },
    {
      "unique-id": "web-gui-uses-accounts",
      "relationship-type": {
        "connects": {
          "source": { "node": "./trading-system.arch.json#web-gui" },
          "destination": { "node": "accounts-service" }
        }
      }
    }
  ]
}

Here the node field accepts either a bare unique-id (local) or a qualified <document>#<unique-id> (cross-document). No node stub is needed. The tradeoff is that the document cannot render standalone without resolving the reference, and tooling must understand the qualified format everywhere a node reference appears.

---

Option C: Ecosystem-level document owns the cross-system relationship

Rather than either system document owning the cross-boundary connection, a third ecosystem-level document defines it. Both nodes are referenced by qualified ID:

traderx-ecosystem.arch.json:

{
  "$schema": "https://calm.finos.org/draft/2026-03/meta/calm.json",
  "nodes": [
    {
      "unique-id": "traderx-ecosystem",
      "node-type": "ecosystem",
      "name": "TraderX Ecosystem",
      "description": "TraderX platform ecosystem"
    },
    {
      "unique-id": "trading-system",
      "node-type": "system",
      "name": "Trading System",
      "description": "Simple Trading System",
      "details": {
        "detailed-architecture": "./trading-system.arch.json"
      }
    },
    {
      "unique-id": "refdata-system",
      "node-type": "system",
      "name": "Ref Data System",
      "description": "Account and Product Reference Data",
      "details": {
        "detailed-architecture": "./refdata-system.arch.json"
      }
    }
  ],
  "relationships": [
    {
      "unique-id": "ecosystem-composed-of",
      "relationship-type": {
        "composed-of": {
          "container": "traderx-ecosystem",
          "nodes": ["trading-system", "refdata-system"]
        }
      }
    },
    {
      "unique-id": "web-gui-uses-accounts",
      "description": "Web GUI looks up account data",
      "relationship-type": {
        "connects": {
          "source": { "node": "./trading-system.arch.json#web-gui" },
          "destination": { "node": "./refdata-system.arch.json#accounts-service" }
        }
      },
      "protocol": "HTTPS"
    }
  ]
}

This keeps individual system documents clean (they only define what they own) and puts cross-system connections in the ecosystem document, which is the natural place for them. Flows defined at ecosystem level can then reference web-gui-uses-accounts by local unique-id since the relationship lives in the same document as the flow.

---

Underlying schema definition (shared across all options):

To support the qualified reference format, define a calm-ref type in core.json#/defs:

{
  "defs": {
    "calm-ref": {
      "type": "string",
      "pattern": "^.+#.+$",
      "description": "A cross-document reference: <document-url>#<unique-id>"
    }
  }
}

Fields that accept cross-document references would use anyOf to accept either a bare local unique-id or a calm-ref:

{
  "node": {
    "anyOf": [
      { "type": "string", "description": "Local unique-id" },
      { "$ref": "#/defs/calm-ref", "description": "Cross-document reference" }
    ]
  }
}

Backward Compatibility:

• Existing CALM documents remain valid. All changes are additive; current documents that use bare unique-id strings continue to work. Cross-document references are opt-in.
• No breaking changes. Bare unique-id strings are interpreted as document-local references (current behaviour). Only strings containing # are treated as cross-document references.
• Migration path: Existing documents that duplicate external nodes can gradually replace stubs with references. No forced migration required.

Open Questions:

• Which option (or combination) best balances standalone readability vs. minimal duplication?
• Should the reference format be <relative-path>#<unique-id>, an absolute URL, or support both?
• Should referenced nodes be resolvable at validation time, or can they be treated as opaque identifiers initially?
• How does this interact with the pattern/instantiation model - do references point to pattern nodes, instantiated nodes, or both?
• Does this subsume or complement the existing detailed-architecture field?
• Should cross-system relationships live in one of the participating system documents, or only at ecosystem level?

Additional Context:

Prior art and related issues:

• #650 - Original ecosystem/flows discussion. Identified the cross-architecture node reference gap. The workaround of re-declaring external nodes with detailed-architecture was proposed but acknowledged as incomplete. Closed via PR Add ecosystem to node spec and tie flows into core.json #660 which added the ecosystem node type and composed-of relationship, but did not address cross-document identity.
• #1876 - Flows externalisation discussion. Proposed the relationship-url convention (./architecture.json#ui-to-service) as a way for standalone flow documents to reference relationships in architecture documents. This is a specific instance of the general cross-document reference problem.
• #1875 - Flow-first modelling. Proposes abstract transitions that don't require architecture references. Complementary to this proposal: abstract flows don't need references, but architecture-bound flows do.
• #426 - Original flows implementation. Established the relationship-unique-id convention that is now document-scoped.

[markscott-ms]

I've felt this pain of wanting to reference a specific node from one architecture to another, for example to model how a statement generation system might connect to the account reference data system's API to get an account's mailing address. These may not be considered an ecosystem, and you want to indicate exactly what in the account reference data system you connect to. Do you connect to their legacy API, or more modern one?

That said, I don't think any of Options A/B/C as framed is quite right, and I think the a minimal design can be narrower than the "Target Schema" section assumes (core.json node/relationship/connects-type).

Working through it:

Node identity: stubs, not qualified references

Option B/C's fully-qualified inline references ("node": "./trading-system.arch.json#web-gui") are DRY, but they remove all local trace of what the referenced node looked like. If the source architecture team renames or retypes a node, the pointer gives you no way to detect that drift without reaching into the source team's version history, which you may not have access to (see Use Case 5, independent team authorship). A materialized stub, by contrast, is self-describing: a validator can diff "what this document assumed about web-gui" against "what web-gui currently is," using only information already present in the referencing document.

So for nodes, I'd keep Option A's stub approach. BUT I'd propose not introducing a new ref-source field. detailed-architecture already means "go here for more/canonical information about me"; the only change needed is letting it optionally carry a #<unique-id> fragment, non-breaking since it's currently just {"type": "string"}:

{
  "unique-id": "web-gui",
  "node-type": "service",
  "name": "Web GUI",
  "description": "Trading System's Web GUI",
  "details": {
    "detailed-architecture": "./trading-system.arch.json#web-gui"
  }
}

No fragment = today's meaning (whole document elaborates this node). With fragment = a specific node elsewhere is this node's canonical/detailed counterpart.

Relationships and interfaces: no core.json changes needed

Once a stub exists locally, connects, interacts, deployed-in, and composed-of never need to cross a document boundary. They keep referencing the stub's local unique-id. This also covers interfaces for free: connects-type's node-interface (interface.json) already separates node from interfaces, and interface.json's interface-type variant already permits a bare {"unique-id": "..."} stub (only interface-definition requires definition-url/config). So a stub node can carry minimal interface placeholders with zero schema change:

{
  "unique-id": "web-gui",
  "node-type": "service",
  "name": "Web GUI",
  "description": "Trading System's Web GUI",
  "interfaces": [{ "unique-id": "https-interface" }],
  "details": { "detailed-architecture": "./trading-system.arch.json#web-gui" }
}

{
  "unique-id": "web-gui-uses-accounts",
  "relationship-type": {
    "connects": {
      "source": { "node": "web-gui", "interfaces": ["https-interface"] },
      "destination": { "node": "accounts-service" }
    }
  }
}

Where a real cross-document reference type is still needed: flow.json only

Relationships aren't sensible things to stub (there's no stable "shape" to snapshot the way there is for a node), so a flow transition living in a third document that needs to point at a relationship declared inside, say, refdata-system.arch.json, genuinely needs a qualified reference. That's the one place I'd introduce the calm-ref type from the issue's "Underlying schema definition" section, scoped to flow.json's relationship-unique-id field specifically — not broadly across core.json.

For the reference format itself: keep it generic (<base>#<unique-id>) rather than inventing a bespoke scheme — the MultiStrategy document-loader already dispatches across filesystem/direct-URL/CALM-Hub for detailed-architecture resolution, and CALM Hub already has an established URI convention for addressing a specific hub-hosted document (shared/src/hub/document-id-utils.ts's constructDocumentId(), .../namespaces/{ns}/{type}/{id}/versions/{version}). Reuse that rather than defining a second hub-addressing convention just for cross-references. Version-pinning should be optional (default latest) for ergonomics, consistent with the rest of this proposal being permissive-by-default.

Validation: opaque by default, opt-in deep resolution

Same as current detailed-architecture validation by the CLI - optional deep recursion, and it should be able to use the same mechanisms:

Neither detailed-architecture's fragment nor flow.json's calm-ref should be mandatorily resolved at basic schema-validation time — that would make a document's validity depend on network/repo access it may not have (Use Case 5). This isn't a new pattern: NodeDetailsValidationRule already resolves and recursively validates detailed-architecture targets today, but only as an opt-in deep-validation pass, with cycle-safety via the existing CachingTrackingResolver. Reuse that same opt-in model for both.

Two things this does not solve:

• Reverse links / discoverability. A forward pointer (stub → canonical source) doesn't tell you who references you. That needs something to crawl documents and build a reverse index — squarely Use Case 3 (CALM Hub traversal), but a tooling/indexing concern, not a schema-field concern.
• Patterns. I'd scope all of this to architecture documents only. A pattern node is a structural constraint, not a concretely-identified thing, so "referencing a node in a pattern" is a different question than "referencing a real node in a real architecture" and deserves its own issue if it's ever needed.