|
| 1 | +.. |
| 2 | + # ******************************************************************************* |
| 3 | + # Copyright (c) 2026 Contributors to the Eclipse Foundation |
| 4 | + # |
| 5 | + # See the NOTICE file(s) distributed with this work for additional |
| 6 | + # information regarding copyright ownership. |
| 7 | + # |
| 8 | + # This program and the accompanying materials are made available under the |
| 9 | + # terms of the Apache License Version 2.0 which is available at |
| 10 | + # https://www.apache.org/licenses/LICENSE-2.0 |
| 11 | + # |
| 12 | + # SPDX-License-Identifier: Apache-2.0 |
| 13 | + # ******************************************************************************* |
| 14 | +
|
| 15 | +.. _itf_architecture: |
| 16 | + |
| 17 | +Architecture |
| 18 | +============ |
| 19 | + |
| 20 | +This page explains the core design decisions in ITF: the target abstraction |
| 21 | +layer, the capability system, the plugin lifecycle, and how ITF integrates |
| 22 | +bidirectionally with Bazel. |
| 23 | + |
| 24 | +.. plantuml:: itf_architecture.puml |
| 25 | + |
| 26 | +Target abstraction layer |
| 27 | +------------------------ |
| 28 | + |
| 29 | +The central concept in ITF is the ``Target``. A target represents the device |
| 30 | +or environment under test — a Docker container, a QEMU virtual machine, or |
| 31 | +real hardware. All target types expose the same interface, so test code does |
| 32 | +not need to know which environment it runs on. |
| 33 | + |
| 34 | +.. code-block:: python |
| 35 | +
|
| 36 | + class Target: |
| 37 | + def execute(self, command): ... |
| 38 | + def upload(self, local_path, remote_path): ... |
| 39 | + def download(self, remote_path, local_path): ... |
| 40 | + def restart(self): ... |
| 41 | + def get_capabilities(self) -> Set[str]: ... |
| 42 | +
|
| 43 | +A test that calls ``target.execute("uname -a")`` runs unchanged against a |
| 44 | +Docker container or a QEMU VM. The target type is determined at build time by |
| 45 | +the ``plugins`` attribute on ``py_itf_test``, and at run time by the CLI |
| 46 | +args (e.g. ``--docker-image``) that configure the chosen plugin. |
| 47 | + |
| 48 | +Capability system |
| 49 | +----------------- |
| 50 | + |
| 51 | +Different target environments support different operations. A plain Docker |
| 52 | +container supports ``exec`` and file transfer but not SSH or SFTP unless an |
| 53 | +SSH server is installed. A QEMU VM provides SSH, SFTP, and network-level |
| 54 | +operations. |
| 55 | + |
| 56 | +Each ``Target`` subclass declares its capabilities, either by passing them |
| 57 | +to the base constructor or by relying on ``Target.REQUIRED_CAPABILITIES`` |
| 58 | +(``exec``, ``file_transfer``, ``restart``), which is always merged in. |
| 59 | +``DockerTarget`` uses only the required capabilities, so it passes no |
| 60 | +extras: |
| 61 | + |
| 62 | +.. code-block:: python |
| 63 | +
|
| 64 | + class DockerTarget(Target): |
| 65 | + def __init__(self, container): |
| 66 | + super().__init__() # capabilities come from REQUIRED_CAPABILITIES |
| 67 | + self.container = container |
| 68 | +
|
| 69 | +Tests can be guarded against targets that lack required capabilities using the |
| 70 | +``@requires_capabilities`` decorator: |
| 71 | + |
| 72 | +.. code-block:: python |
| 73 | +
|
| 74 | + from score.itf.plugins.core import requires_capabilities |
| 75 | +
|
| 76 | + @requires_capabilities("ssh", "sftp") |
| 77 | + def test_file_roundtrip(target): |
| 78 | + ... |
| 79 | +
|
| 80 | +If the active target does not provide all listed capabilities, pytest skips the |
| 81 | +test with a clear message. This keeps test suites portable: the same file can |
| 82 | +run against Docker for fast feedback and against a QEMU VM for full-system |
| 83 | +integration, skipping tests that do not apply. |
| 84 | + |
| 85 | +Tests can also query capabilities at runtime and branch accordingly: |
| 86 | + |
| 87 | +.. code-block:: python |
| 88 | +
|
| 89 | + def test_adaptive(target): |
| 90 | + if target.has_capability("ssh"): |
| 91 | + with target.ssh() as ssh: |
| 92 | + ssh.execute_command("echo hello") |
| 93 | + else: |
| 94 | + exit_code, _ = target.execute("echo hello") |
| 95 | +
|
| 96 | +Plugin lifecycle |
| 97 | +---------------- |
| 98 | + |
| 99 | +Each plugin contributes a ``target_init`` pytest fixture. ITF's core plugin |
| 100 | +calls this fixture to obtain the target instance, then wraps it in the |
| 101 | +``target`` fixture that test functions receive. |
| 102 | + |
| 103 | +The lifecycle for a single test is: |
| 104 | + |
| 105 | +1. **Setup**: The plugin's ``target_init`` fixture starts the target |
| 106 | + (spins up a container, boots a QEMU VM, connects to hardware). |
| 107 | +2. **Test execution**: The test function receives the ``target`` fixture |
| 108 | + and exercises the system under test. |
| 109 | +3. **Teardown**: ``target_init`` tears down the target (stops the |
| 110 | + container, shuts down the VM). |
| 111 | + |
| 112 | +With ``--keep-target``, steps 1 and 3 run once per session instead of once |
| 113 | +per test function. This is faster but means tests share target state, so it |
| 114 | +should only be used when tests are designed to be order-independent. |
| 115 | + |
| 116 | +**Plugin loading order is deterministic but should not be relied upon.** |
| 117 | +The core plugin is always registered first. The remaining plugins are |
| 118 | +registered with pytest in the exact order they are listed in |
| 119 | +``py_itf_test.plugins``. While this order is stable, plugins are designed |
| 120 | +to be independent of each other — no plugin should depend on another |
| 121 | +plugin's initialisation having completed first. |
| 122 | + |
| 123 | +Why a plugin-based design |
| 124 | +-------------------------- |
| 125 | + |
| 126 | +Plugin-based design was chosen for three reasons: |
| 127 | + |
| 128 | +**Separation of concerns.** Target management logic (starting containers, |
| 129 | +booting VMs) is entirely isolated from test logic. A test that calls |
| 130 | +``target.execute()`` has no dependency on Docker or QEMU APIs. |
| 131 | + |
| 132 | +**Extensibility without forking.** Custom targets (real hardware, emulators, |
| 133 | +cloud VMs) are added by implementing ``Target`` and ``target_init`` in a new |
| 134 | +plugin. No changes to the ITF core are needed. |
| 135 | + |
| 136 | +**Bazel-native composition.** Because plugins are declared as Bazel targets |
| 137 | +with ``py_itf_plugin``, they carry their own Python libraries, data files, |
| 138 | +and CLI args. Combining plugins — for example Docker + DLT — is as simple as |
| 139 | +listing both labels in ``py_itf_test.plugins``. Bazel resolves transitive |
| 140 | +dependencies automatically. |
| 141 | + |
| 142 | +Bidirectional Bazel integration |
| 143 | +--------------------------------- |
| 144 | + |
| 145 | +ITF integrates with Bazel in both directions: |
| 146 | + |
| 147 | +**Build-time** (Bazel → ITF): The ``py_itf_test`` symbolic macro creates a |
| 148 | +``py_test`` binary that bundles the test code and all plugin Python |
| 149 | +libraries. Plugin CLI args (e.g. ``--docker-image``, paths from |
| 150 | +``$(location ...)``) are resolved at analysis time and baked into a launcher |
| 151 | +script. This means test hermetically carry their full dependency graph, |
| 152 | +including container images or QEMU images referenced via Bazel labels. |
| 153 | + |
| 154 | +**Run-time** (ITF → Bazel): ITF uses Bazel's runfiles mechanism to locate |
| 155 | +data files at runtime. The ``$(location ...)`` substitution in ``args`` |
| 156 | +produces runfiles-relative paths that work regardless of where Bazel places |
| 157 | +files in the output tree. Test results are reported via JUnit XML to |
| 158 | +``$XML_OUTPUT_FILE``, integrating with Bazel's native test reporting and |
| 159 | +caching. |
| 160 | + |
| 161 | +This design means ITF tests participate fully in Bazel's incremental build |
| 162 | +and caching: a test is only re-run if its source, its dependencies, or its |
| 163 | +configuration changes. |
0 commit comments