diff --git a/docs/features/communication/index.rst b/docs/features/communication/index.rst index 460c6fd7538..1c1714c16ec 100644 --- a/docs/features/communication/index.rst +++ b/docs/features/communication/index.rst @@ -31,6 +31,7 @@ Communication (v0.5 beta) :titlesonly: :hidden: + dds_gateway/index.rst docs/**/index ipc/index some_ip_gateway/index diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_builder.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_builder.puml new file mode 100644 index 00000000000..a1b25fe473b --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_builder.puml @@ -0,0 +1,25 @@ +@startuml + +title Sequence Diagram: Build KVS Instance + +participant "User" as actor +participant "«component» :kvs" as kvs + +actor -> kvs: Create KVS builder +kvs --> actor: KVS builder instance + +actor -> kvs: Set need_defaults flag +kvs --> actor: KVS builder instance + +actor -> kvs: Set need_kvs flag +kvs --> actor: KVS builder instance + +actor -> kvs: Build KVS instance + +alt kvs-builder-success + kvs --> actor: KVS instance with builder config +else kvs-builder-error + kvs --> actor: KVS ErrorCode +end + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_check_value_default.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_check_value_default.puml new file mode 100644 index 00000000000..01baa2727f0 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_check_value_default.puml @@ -0,0 +1,24 @@ +@startuml + +title Sequence Diagram: Check if Key contains Default Value + +participant "User" as actor +participant "«component» :kvs" as kvs + +actor -> kvs: Does key have default value + +alt default-key-exists + alt key-exists + alt default-and-key-match + kvs --> actor: Key contains default value + else key-doesnt-exist + kvs --> actor: Key doesn't containt default value + end + else key-doesnt-exist + kvs --> actor: Key contains default value + end +else default-doesnt-exist + kvs --> actor: Key doesn't contain default value +end + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_delete_data_key.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_delete_data_key.puml new file mode 100644 index 00000000000..e2b222e3114 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_delete_data_key.puml @@ -0,0 +1,16 @@ +@startuml + +title Sequence Diagram: Delete Key from KVS Instance + +participant "User" as actor +participant "«component» :kvs" as kvs + +actor -> kvs: Remove key + +alt key-exists + kvs --> actor: Successfully deleted key +else key-doesnt-exist + kvs --> actor: Key-Not-Found-Error +end + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_flush_local_repr_to_file.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_flush_local_repr_to_file.puml new file mode 100644 index 00000000000..947cf782ebb --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_flush_local_repr_to_file.puml @@ -0,0 +1,56 @@ +@startuml + +title Sequence Diagram: Flush Local Representation to Data File + +participant "User" as actor +participant "«component» :kvs" as kvs +participant "«component» :json" as json_parser +participant "«component» :fs" as fs + +actor -> kvs: Flush KVS + +kvs -> json_parser: Generate string from local representation + +alt json-string + json_parser --> kvs: JSON data string +else json-string-error + json_parser --> kvs: JSON generator error + kvs --> actor: JSON generator error +end + +kvs -> kvs: Rotate snapshots + +alt snapshot-rotate-success + kvs --> kvs: Snapshots rotated +else snapshot-rotate-error + kvs --> actor: Snapshot-Rotate-Error +end + +kvs -> kvs: Create JSON data hash + +alt hash-created-success + kvs -> kvs: Data hash +else hash-create-error + kvs --> actor: Hash-Calc-Error +end + +kvs -> fs: Write JSON data string to file + +alt file-write + fs --> kvs: File successfully written +else file-write-error + fs --> kvs: File-Write-Error + kvs --> actor: File-Write-Error +end + +kvs -> fs: Write JSON data hash to file + +alt file-hash-write + fs --> kvs: Hash file successfully written + kvs --> actor: Flush successful +else file-hash-write-error + fs --> kvs: File-Hash-Write-Error + kvs --> actor: File-Hash-Write-Error +end + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_read_data_key.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_read_data_key.puml new file mode 100644 index 00000000000..064e2489115 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_read_data_key.puml @@ -0,0 +1,20 @@ +@startuml + +title Sequence Diagram: Read Key from KVS Instance + +participant "User" as actor +participant "«component» :kvs" as kvs + +actor -> kvs: Get value for key + +alt key-exists + kvs --> actor: Return value for key +else key-doesnt-exist + alt default-exists + kvs --> actor: Return default value for key + else default-doesnt-exist + kvs --> actor: Key-Not-Found error + end +end + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_read_file_into_local_repr.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_read_file_into_local_repr.puml new file mode 100644 index 00000000000..4a154d24df9 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_read_file_into_local_repr.puml @@ -0,0 +1,47 @@ +@startuml + +title Sequence Diagram: Read Data into Local Representation (KvsValue) + +participant "User" as actor +participant "«component» :kvs" as kvs +participant "«component» :json" as json_parser +participant "«component» :fs" as fs + +actor -> kvs: Open KVS + +kvs -> fs: Read defaults file + +alt file-exists + fs --> kvs: Defaults file content (JSON) +else file-based-error + fs --> kvs: File-Error + kvs -> actor: File-Error +end + +kvs -> fs: Read defaults file hash + +alt file-exists + fs --> kvs: Defaults file hash +else file-based-error + fs --> kvs: File-Error + kvs -> actor: File-Error +end + +kvs -> kvs: Generate defaults file hash + +alt hash-match-success + kvs -> json_parser: Parse JSON data +else hash-match-error + kvs -> actor: Hash-Error +end + +alt parsing-success + json_parser --> kvs: Parsed JSON object +else parsing-based-error + json_parser -> kvs: Parser-Error + kvs -> actor: Parser-Error +end + +kvs --> actor: KVS instance + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_restore_snapshot.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_restore_snapshot.puml new file mode 100644 index 00000000000..17eb75b3823 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_restore_snapshot.puml @@ -0,0 +1,16 @@ +@startuml + +title Sequence Diagram: Restore Snapshot + +participant "User" as actor +participant "«component» :kvs" as kvs + +actor -> kvs: Restore snapshot + +alt snapshot-restore-success + kvs --> actor: Snapshot restored successfully +else snapshot-restore-error + kvs --> actor: Snapshot-Not-Available-Error +end + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_dyn_write_data_key.puml b/docs/features/persistency/architecture/_assets/kvs_dyn_write_data_key.puml new file mode 100644 index 00000000000..b34c24cc203 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_dyn_write_data_key.puml @@ -0,0 +1,11 @@ +@startuml + +title Sequence Diagram: Write Key to KVS Instance + +participant "User" as actor +participant "«component» :kvs" as kvs + +actor -> kvs: Set value for key +kvs --> actor: Value set for key + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_interface.puml b/docs/features/persistency/architecture/_assets/kvs_interface.puml new file mode 100644 index 00000000000..46f65a27cc0 --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_interface.puml @@ -0,0 +1,21 @@ +@startuml + +interface IKvs { + open + flush_on_exit + reset + get_all_keys + key_exists + get_value + get_default_value + has_default_value + set_default_value + set_value + remove_key + flush + snapshot_count + snapshot_max_count + snapshot_restore +} + +@enduml diff --git a/docs/features/persistency/architecture/_assets/kvs_static_view.puml b/docs/features/persistency/architecture/_assets/kvs_static_view.puml new file mode 100644 index 00000000000..ebb646f285c --- /dev/null +++ b/docs/features/persistency/architecture/_assets/kvs_static_view.puml @@ -0,0 +1,34 @@ +@startuml +allowmixing + +title Static View - kvs + +!include kvs_interface.puml + + +skinparam package { + BackgroundColor #E0E0D0 + BorderColor Black + backgroundColor<> lightgreen +} + +skinparam component { + backgroundColor<> white +} + +' Define Features +package "persistency" <> { + component kvs <> + component fs <> + component json <> +} + + + +kvs --> IKvs : implements + +kvs ..> json : use +kvs ..> fs : use + + +@enduml diff --git a/docs/features/persistency/architecture/chklst_arc_inspection.rst b/docs/features/persistency/architecture/chklst_arc_inspection.rst new file mode 100644 index 00000000000..e52a0e13320 --- /dev/null +++ b/docs/features/persistency/architecture/chklst_arc_inspection.rst @@ -0,0 +1,176 @@ +.. + # ******************************************************************************* + # Copyright (c) 2025 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + + +.. document:: Persistency Architecture Inspection Checklist + :id: doc__persistency_arc_inspection + :status: draft + :safety: ASIL_B + :security: YES + :realizes: wp__sw_arch_verification + + +Architecture Inspection Checklist +================================= + +Purpose +------- + +The purpose of the software architecture checklist is to ensure that the design meets the criteria and quality as +defined per project processes and guidelines for feature and component architectural design elements. +It helps to check the compliance with requirements, identify errors or inconsistencies, and ensure adherence to best +practices. +The checklist guides evaluation of the architecture design, identifies potential problems, and aids in +communication and documentation of architectural decisions to stakeholders. + +Checklist +--------- + +.. list-table:: Feature Architecture Design Review Checklist Persistency + :header-rows: 1 + + * - Review Id + - Acceptance criteria + - Type + - Guidance + - passed + - Remarks + - Issue link + * - ARC_01_01 + - Is the traceability from software architectural elements to requirements, and other level architectural + elements (e.g. component to interface) established according to the defined :need:`Relations between the architectural elements `? + - automated + - Trace should be checked by Sphinx. Will be removed from checklist once requirement is implemented. + - No + - Traceability not complete, template not followed + - `Issue for Findings `_ + * - ARC_01_02 + - If the architectural element is related to any supplier manuals (incl. safety and security) + are the relevant parts covered? + - manual + - If the architecture makes use of supplied elements, their manuals (like safety) have to be considered (i.e. its provided functionality matches the expectation and assumptions are fulfilled). Note that in case of safety component this means that assumed Technical Safety Requirements and AoUs of the safety manual are covered. + - Yes + - Not applicable + - + * - ARC_01_03 + - Is the architectural element traceable to the lower level artifacts as defined by the workproduct traceability? + - automated + - Will be removed from checklist once requirement is implemented by automated tool check. + Details of possible linking can be depicted from `Traceability Concept `_ + - No + - Traceability not complete, template not followed + - `Issue for Findings `_ + * - ARC_02_01 + - Is the software architecture design compliant with the (overall) feature architecture? + - manual + - On component level check against the feature architecture, on feature level check other features with common components used. + - No + - There is no component architecture available for persistency + - `Issue for Findings `_ + * - ARC_02_02 + - Is appropriate and comprehensible operation/interface naming present in the architectural design? + - manual + - Check :need:`gd_guidl__arch_design` + - No + - Are these interfaces names, any operations? Please update template. + - `Issue for Findings `_ + * - ARC_02_03 + - Are correctness of data flow and control flow within the architectural elements considered? + - manual + - E.g. examine definitions, transformations, integrity, and interaction of data; check error handling, data + exchange between elements, correct response to inputs and documented decision making. + Note: consistency is ensured by the process/tooling, by defining each interface only once. + - Yes + - + - + * - ARC_02_04 + - Are the interfaces between the software architectural element and other architectural elements well-defined? + - manual + - Check if the interface reacts on non-defined behavior or errors; can established protocols be used; are the + interfaces for inputs, outputs, error codes documented; is loose coupling considered and only limited exposure; + can unit or integration test be written against the interface; data amount transferred; no sensitive data + exposure; + - Yes + - + - + * - ARC_02_05 + - Does the software architectural element consider the timing constraints (from the parent requirement)? + - manual + - If there are hard requirements on the timing a programming time estimation should be performed and also + deadline supervision considered. + - No + - No information found, but required + - `Issue for Findings `_ + * - ARC_02_06 + - Is the documentation of the software architectural element, including textual and graphical descriptions + (e.g., UML diagrams), comprehensible and complete? + - manual + - Use of semi-formal notation is expected for architectural elements with an allocated ASIL level. + Is the architecture template correctly filled? + - Yes + - + - + * - ARC_03_01 + - Is the architectural element modular and encapsulated? + - manual + - Check e.g. that only minimal interfaces are used. Design should be object oriented. Interfaces and interactions are clearly defined. Usage of access types (private, protected) properly set. Limited global variables. + - No + - Yes + - + * - ARC_03_02 + - Is the suitability of the software architecture for future modifications and maintainability considered? + - manual + - Check for e.g. loose coupling, separation of concerns, high cohesion, versioning strategy for interfaces, + decision records, use of established design patterns. + - Yes + - Not applicable on feature architecture level + - + * - ARC_03_03 + - Are simplicity and avoidance of unnecessary complexity present in the software architecture? + - manual + - Indicators for complexity are: number of use cases (corresponding to dynamic diagrams) + allocated to single design element, number of interfaces and operations in an interface, + function parameters, global variables, complex types, limited comprehensibility. + + Note: If the "number" above exceeds "3" a design rationale is mandatory (for all types) + - Yes + - Not applicable on feature architecture level + - + * - ARC_03_04 + - Is the software architecture design following best practices and design principles? + - manual + - Refer to architectural guidelines and recommendations within the project documentation. + - Yes + - + - + * - ARC_04_01 + - If software partitioning (different operating system processes) is used to implement freedom from interference between the processes with different rating (QM/ASIL), is effectiveness evidence generated during integration and verification tests? + + Note: see ISO 26262-6, 7.4.9 and Annex D for partitioning + - manual + - + a) the usage of shared resources (cpu time, shared memory, ...) are checked in a way that freedom from interference between the processes is ensured, + b) check if the operating system supports freedom from interference between the processes + - + - + - + * - ARC_04_02 + - Is an upper estimation of the required resources (RAM, ROM, non volatile memory, communication) available and documented? + + Note: see ISO 26262-6, 7.4.11 + - manual + - + - + - + - diff --git a/docs/features/persistency/architecture/index.rst b/docs/features/persistency/architecture/index.rst new file mode 100644 index 00000000000..3f749b6bbb9 --- /dev/null +++ b/docs/features/persistency/architecture/index.rst @@ -0,0 +1,174 @@ +.. + # ******************************************************************************* + # Copyright (c) 2025 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +.. _feature_architecture_persistency: + +Architecture +============ + +.. document:: Persistency KVS Feature Architecture + :id: doc__persistency_architecture + :status: valid + :safety: ASIL_B + :security: NO + :realizes: wp__feature_arch + :tags: persistency + + +.. feat:: Persistency + :id: feat__persistency + :security: YES + :safety: ASIL_B + :status: valid + :provides: logic_arc_int__persistency__interface + +.. comp:: persistency::kvs + :id: comp__persistency_kvs + :security: YES + :safety: ASIL_B + :status: valid + :implements: logic_arc_int__persistency__interface + :belongs_to: feat__persistency + + + +-------- + +The Key-Value-Storage (kvs) provides the capability to efficiently store, +retrieve, and manage key-value pairs in a persistent storage system. + +Description +----------- + +- kvs organize data as pairs, where each unique key is associated with a specific value. + The key acts as a unique identifier for getting the value. +- The data is persisted in JSON format to the file system, providing a human-readable, + and widely supported way to store and manage key-value pairs. +- The JSON data persisted is according to RFC-8259. + +Rationale Behind Architecture Decomposition +******************************************* + +- The architecture is decomposed to include a dedicated JSON parser component (json) to facilitate the persistent storage of data in JSON format. +- The architecture is decomposed to include a FileStorage component (fs) to read and write to the file system. + + +Glossary +-------- + +- User: Program code that is written by a person that initiates the given + functionality call or receives a callback. + + +Static Architecture +------------------- + +.. feat_arc_sta:: Static Architecture + :id: feat_arc_sta__persistency__static + :security: YES + :safety: ASIL_B + :includes: logic_arc_int__persistency__interface + :fulfils: feat_req__persistency__default_value_get,feat_req__persistency__default_values,feat_req__persistency__async_completion,feat_req__persistency__integrity_check,feat_req__persistency__store_data,feat_req__persistency__load_data,feat_req__persistency__snapshot_create,feat_req__persistency__support_datatype_keys,feat_req__persistency__support_datatype_value,feat_req__persistency__variant_management,feat_req__persistency__default_value_file,feat_req__persistency__cfg,feat_req__persistency__async_api,feat_req__persistency__access_control,feat_req__persistency__concurrency + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_static_view.puml + +Dynamic Architecture +-------------------- +.. feat_arc_dyn:: Check if key contains default value + :id: feat_arc_dyn__persistency__check_key_default + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__default_values,feat_req__persistency__default_value_get + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_check_value_default.puml + +.. feat_arc_dyn:: Delete key from KVS instance + :id: feat_arc_dyn__persistency__delete_key + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__support_datatype_keys,feat_req__persistency__support_datatype_value + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_delete_data_key.puml + +.. feat_arc_dyn:: Flush to permanent storage + :id: feat_arc_dyn__persistency__flush + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__store_data,feat_req__persistency__snapshot_create,feat_req__persistency__integrity_check,feat_req__persistency__snapshot_restore + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_flush_local_repr_to_file.puml + +.. feat_arc_dyn:: Read key value + :id: feat_arc_dyn__persistency__read_key + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__support_datatype_keys,feat_req__persistency__support_datatype_value,feat_req__persistency__default_values,feat_req__persistency__default_value_get + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_read_data_key.puml + +.. feat_arc_dyn:: Read data from permanent storage + :id: feat_arc_dyn__persistency__read_from_storage + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__load_data,feat_req__persistency__integrity_check,feat_req__persistency__snapshot_restore + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_read_file_into_local_repr.puml + +.. feat_arc_dyn:: Write value to key + :id: feat_arc_dyn__persistency__write_key + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__support_datatype_keys,feat_req__persistency__support_datatype_value + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_write_data_key.puml + +.. feat_arc_dyn:: Restore snapshot + :id: feat_arc_dyn__persistency__snapshot_restore + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__snapshot_restore,feat_req__persistency__store_data + :status: valid + :belongs_to: feat__persistency + + .. uml:: _assets/kvs_dyn_restore_snapshot.puml + + +Logical Interfaces +------------------ + +.. logic_arc_int:: Ikvs + :id: logic_arc_int__persistency__interface + :security: YES + :safety: ASIL_B + :fulfils: feat_req__persistency__async_api + :status: valid + + .. uml:: _assets/kvs_interface.puml + +.. needextend:: is_external == False and "persistency/kvs/architecture" in docname + :+tags: persistency diff --git a/docs/features/persistency/index.rst b/docs/features/persistency/index.rst index 5644928c46d..ef32a0f12ef 100644 --- a/docs/features/persistency/index.rst +++ b/docs/features/persistency/index.rst @@ -12,8 +12,137 @@ # SPDX-License-Identifier: Apache-2.0 # ******************************************************************************* -Persistency -=========== +.. _features_persistency: -Complete module documentation can be found in `Persistency `__. +Persistency (v0.5 beta) +######################## +.. document:: Persistency + :id: doc__persistency + :status: valid + :safety: ASIL_B + :security: NO + :realizes: wp__feat_request + :tags: feature_request, persistency + +.. toctree:: + architecture/index.rst + architecture/chklst_arc_inspection.rst + requirements/index.rst + requirements/chklst_req_inspection.rst + + +Feature flag +============ + +To activate this feature, use the following feature flag: + +``persistency`` + + +Abstract +======== + +Persistency is a critical feature that ensures the long-term storage and +retrieval of data within S-CORE. It provides a reliable mechanism for +preserving information, allowing the application to maintain its state and data +integrity over time. This feature is essential for enabling the system to +resume operations seamlessly, even in the event of unexpected shutdowns or +system failures. By implementing robust persistence mechanisms, the application +can guarantee the persistence of user-generated content, configuration +settings, and other vital data, ensuring a consistent and reliable user +experience. + +This feature request describes the key-value storage (KVS) that is needed by +applications to store either temporary or permanent data in an easy way that +conforms to most programming languages that provide a hash, hashmap, dictionary, +or similar data structure. Access to the KVS is possible from any supported +language through language-specific interfaces. + +In the scope of S-CORE, an application can range from a system service to an +end-user visible UI. The application uses the KVS as an external memory store +to read and persist data as needed. For example, an application that controls +the air conditioning system in a car could use the KVS to store the current +temperature setting. When the car is started again, the application can +retrieve the temperature setting from the persistent KVS storage, providing a +seamless user experience by restoring the previous state. + + +Motivation +========== + +The current solutions available mostly don't meet the specific needs of the +S-CORE project like storing specific datatypes without a BASE64 conversion or +having no rollback/replay feature. Also the integration into analysis tools is +simpler when the solution grows with the needs instead having to adapt existing +data structures through wrappers. Especially in the focus of security it will +be possible to build a system that integrates the layers from scratch and +provide them as API to any language whilst still using Rust as the backend. + +A main USP of the solution will be the integration of a tracing framework that +allows to understand how events also in the context of other events interact. + +A key-value storage is used within many applications to store e.g. +configuration data and is therefore seen crucial for the Eclipse S-CORE +platform. + + +Rationale +========= + +1. | Requirement 1: Multiple key-value storages per application + | Solution: Allow each application to have multiple key-value storages (KVS) to enable data separation and different levels of security. +2. | Requirement 2: Update mechanism for KVS versions + | Solution: Implement an update mechanism to ensure compatibility through updates and rollbacks of different KVS versions. +3. | Requirement 3: Language-agnostic KVS interface + | Solution: Design a flexible interface that allows the KVS to be read and written from multiple programming languages, including C++, Rust, and others. +4. | Requirement 4: Default values for KVS + | Solution: Configure the KVS to store default values for all keys, returning either the default value or an error if the key needs to be written first. +5. | Requirement 5: Simple data representation for KVS + | Solution: Utilize a simple data representation, such as JSON or Cap'n Proto, that supports versioned up- and downgrading and is easily debuggable by developers. +6. | Requirement 6: KVS integrity checking + | Solution: Ensure the KVS maintains a consistent state, providing either the currently stored data or the previous snapshot if data retrieval is not possible. + + +Backwards Compatibility +======================= + +The API for the specific language tries to represent the language specific +implementation like hashmaps or dictionaries to be mostly backwards compatible +to already existing key-value-storage usage cases. Access without a safe error +handling path, like the array-operator in Rust which can panic, must be +avoided. + + +Security Impact +=============== + +Access to the key-value-storage would allow a malicious user to control the +behavior of the device, so it must be secured to prevent unauthorized access. +To achieve this, debug access should only be provided when a debug firmware +image is installed. + + +Safety Impact +============= + +The expected ASIL level is ASIL-B. To reach this goal we will apply the S-CORE +development process. Key elements of it are listed in the process descriptions +of safety management and safety analysis. In the safety analysis we will +analyze the impact of the feature. + +.. :need:`doc__persistency_fmea` + +We use an iterative development process and apply results from the next steps +back to the feature request. + +To ensure the freedom of interference the feature key-value storage should not +be used within different processes. + + +License Impact +============== + +.. note:: + The key-value storage itself uses the Apache-2.0 license. Licenses of + used libraries are need to be checked. diff --git a/docs/features/persistency/requirements/chklst_req_inspection.rst b/docs/features/persistency/requirements/chklst_req_inspection.rst new file mode 100644 index 00000000000..3500eac586e --- /dev/null +++ b/docs/features/persistency/requirements/chklst_req_inspection.rst @@ -0,0 +1,153 @@ +.. + # ******************************************************************************* + # Copyright (c) 2025 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Persistency Requirements Inspection Checklist +############################################# + +.. document:: Persistency Requirements Inspection Checklist + :id: doc__feature_persistency_requirements_chklst + :status: valid + :safety: ASIL_B + :security: NO + :tags: persistency + :realizes: wp__requirements_inspect + +**Purpose** + +The purpose of this requirement inspection checklist is to collect the topics to be checked during requirements inspection. + +**Checklist** + +.. list-table:: Persistency Requirements Inspection Checklist + :header-rows: 1 + :widths: 10,30,50,6,6,8 + + * - Review ID + - Acceptance Criteria + - Guidance + - Passed + - Remarks + - Issue link + * - REQ_01_01 + - Is the requirement sentence template used? + - see :need:`gd_temp__req_formulation`, this includes the use of "shall". + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_02_01 + - Is the requirement description *comprehensible* ? + - If you think the requirement is hard to understand, comment here. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_02_02 + - Is the requirement description *unambiguous* ? + - Especially search for "weak words" like "about", "etc.", "relevant" and others (see the internet documentation on this). This check shall be supported by tooling. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_02_03 + - Is the requirement description *atomic* ? + - A good way to think about this is to consider if the requirement may be tested by one (positive) test case or needs more of these. The sentence template should also avoid being non-atomic already. Note that there are cases where also non-atomic requirements are the better ones, for example if those are better understandable. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_02_04 + - Is the requirement description *feasible* ? + - Expectation is that at the time of the inspection the requirement has already some implementation. This can be checked via traces, but also :need:`gd_req__req_attr_impl` shows this. In case the requirement is not mature enough at the time of inspection (i.e. not implemented at least as "proof-of-concept"), a development expert should be invited to the Pull-Request review to explicitly check this item. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_02_05 + - Is the requirement description *independent from implementation* ? + - This checkpoint should improve requirements definition in the sense that the "what" is described and not the "how" - the latter should be described in architecture/design derived from the requirement. But there can also be a good reason for this, for example we would require using a file format like JSON and even specify the formatting standard already on stakeholder requirement level because we want to be compatible. A finding in this checkpoint does not mean there is a safety problem in the requirement. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_03_01 + - For stakeholder requirements: Is the *rationale* correct? + - Rationales explain why the top level requirements were invented. Do those cover the requirement? + - N/A + - No stakeholder requirements for Persistency KVS needed. + - https://github.com/eclipse-score/score/issues/960 + * - REQ_03_02 + - For other requirements: Is the *linkage to the parent requirement* correct? + - Linkage to correct levels and ASIL attributes is checked automatically, but it needs checking if the child requirement implements (at least) a part of the parent requirement. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_04_01 + - Is the requirement *internally and externally consistent*? + - Does the requirement contradict other requirements within the same or higher levels? One may restrict the search to the feature for component requirements, for features to other features using same components. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_05_01 + - Do the software requirements consider *timing constraints of the parent requirement*? + - This bullet point encourages to think about timing constraints even if those are not explicitly mentioned in the parent requirement. If the reviewer of a requirement already knows or suspects that the implementation will be time consuming, one should think of the expectation of a "user". + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_06_01 + - Does the Requirement consider *external interfaces*? + - The SW platform's external interfaces (to the user) are defined in the Feature Architecture, so the Feature and Component Requirements should determine the data consumed and set on these interfaces. Are output values completely defined? + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_07_01 + - Is the *ASIL Attribute* set correctly? + - Derived requirements are checked automatically, see :need:`gd_req__req_linkage_safety`. But for the top level requirements this needs to be checked for correctness. Also AoU from external components need check for correct ASIL as those are the "origin" of safety requirements towards the SW platform. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_07_02 + - Is the attribute *security* set correctly? + - Stakeholder requirements security attribute should be set based on Threat Analysis and Risk Assessment (TARA) (process is TBD). Checklist item is supported by automated check: "Every requirement which satisfies a requirement with security attribute set to YES inherits this". Expectation is that the feature/component requirements/architecture may also be subject to a Software Security Criticality Analysis (process is TBD). + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_08_01 + - Is the requirement *verifiable*? + - Expectation is that at the time of the inspection already tests are created for the requirement. This can be checked via traces, but also :need:`gd_req__req_attr_test_covered` shows this. In case the requirement is not mature enough at the time of inspection (i.e. missing test cases), a test expert should be invited to the Pull-Request review to explicitly check this item. + - Yes + - No remarks + - https://github.com/eclipse-score/score/issues/960 + * - REQ_09_01 + - Do the feature requirements defining a safety mechanism contain the error reaction leading to a safe state? + - Alternatively to the safe state there could also be "repair" mechanisms. Also do not forget to consider REQ_05_01 for these. + - + - + - + +The following requirements in "valid" state and with "inspected" tag set are in the scope of this inspection: + +.. needtable:: + :filter: "feature_name" in docname and "requirements" in docname and docname is not None and status == "valid" + :style: table + :types: feat_req + :tags: persistency + :columns: id;status;tags + :colwidths: 25,25,25 + :sort: title + +And also the following AoUs in "valid" state and with "inspected" tag set (for these please answer the questions above as if the AoUs are requirements, except questions REQ_03_01 and REQ_03_02): + +.. needtable:: + :filter: "feature_name" in docname and "requirements" in docname and docname is not None and status == "valid" + :style: table + :types: aou_req + :tags: persistency + :columns: id;status;tags + :colwidths: 25,25,25 + :sort: title diff --git a/docs/features/persistency/requirements/index.rst b/docs/features/persistency/requirements/index.rst new file mode 100644 index 00000000000..e99c22abe99 --- /dev/null +++ b/docs/features/persistency/requirements/index.rst @@ -0,0 +1,481 @@ +.. + # ******************************************************************************* + # Copyright (c) 2025 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +.. _feature_persistency_requirements: + +Requirements +############ + +.. document:: Persistency Requirements + :id: doc__feature_persistency_requirements + :status: valid + :safety: ASIL_B + :security: YES + :realizes: wp__feat_request + :tags: persistency + +.. feat_req:: C++ and Rust language support + :id: feat_req__persistency__cpp_rust + :reqtype: Non-Functional + :security: NO + :safety: QM + :satisfies: stkh_req__dev_experience__prog_languages + :status: valid + + The Persistency shall provide native API support for both C++ and Rust programming languages. + +.. feat_req:: Operating system agnostic implementation + :id: feat_req__persistency__os_agnostic + :reqtype: Non-Functional + :security: NO + :safety: QM + :satisfies: stkh_req__functional_req__operating_system + :status: valid + + The Persistency shall be operating system agnostic. + +.. feat_req:: Variant management support + :id: feat_req__persistency__variant_management + :reqtype: Non-Functional + :security: NO + :safety: QM + :satisfies: stkh_req__overall_goals__variant_management + :status: valid + + The Persistency shall ensure compatibility across different SW versions. + +.. feat_req:: Dynamic memory allocation during runtime + :id: feat_req__persistency__dynamic_memory_alloc + :reqtype: Non-Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall not allocate dynamic memory during runtime. All required dynamic memory shall be allocated during initialization. + + .. note:: + + Dynamic memory allocation violates freedom from interference as the `HEAP` is a shared resource on OS process level. + Additionally, fragmentation of the `HEAP` can lead to non-deterministic behavior of the application. + +.. feat_req:: Multiple KVS per application + :id: feat_req__persistency__multiple_kvs + :reqtype: Functional + :security: NO + :safety: QM + :satisfies: stkh_req__functional_req__data_persistency + :status: valid + + The Persistency shall support multiple independent storages per application. + +.. feat_req:: Access from multiple applications + :id: feat_req__persistency__multiple_app + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall prevent access to a single KVS instance from multiple OS processes. + + .. note:: + Access from multiple OS processes violates freedom from interference, as applications can modify the same data concurrently. + +.. feat_req:: Separate data stores + :id: feat_req__persistency__access_control + :reqtype: Functional + :security: YES + :safety: ASIL_B + :satisfies: stkh_req__dependability__security_features + :status: valid + + The Persistency shall ensure that only authorized applications can access individual data stores. + + .. note:: + Access control is essential to prevent unauthorized access and modification of sensitive data. + The Persistency shall implement mechanisms to enforce access control policies based on user roles and permissions. + +.. feat_req:: Configuration + :id: feat_req__persistency__cfg + :reqtype: Functional + :security: YES + :safety: ASIL_B + :satisfies: stkh_req__functional_req__file_based + :status: valid + :tags: config + + The Persistency shall support configuration via a configuration file. + The configuration shall include: + + - Global settings: + - Maximum number of KVS instances + - Maximum size of a key + + - Settings for KVS instance: + - Instance identifier + - Storage URI + - Maximum number of Key-Value pairs + - Maximum number of snapshots + - Maximum consumed storage size (Including all metadata and redundant data) + - Security settings + - Redundancy settings + - Backend specific settings + + Configuration file shall be optional and all configuration attributes shall have sensible default values defined at compile time. + + .. note:: + To improve the user experience during rapid prototyping, the Persistency shall also be able to operate without a configuration file. + +.. feat_req:: Supported datatypes (Keys) + :id: feat_req__persistency__support_datatype_keys + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support UTF-8 encoded strings as valid key types. + +.. feat_req:: Supported datatypes (Values) + :id: feat_req__persistency__support_datatype_value + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + :tags: persistency + + The Persistency shall support storing both primitive and non-primitive (composite) datatypes as values. + +.. feat_req:: Default values + :id: feat_req__persistency__default_values + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support predefined default values for keys. + +.. feat_req:: Provisioning of default values via external file + :id: feat_req__persistency__default_value_file + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__file_based + :status: valid + + The Persistency shall support import of default values using an external file. + + .. note:: + Default values are read-only and cannot be modified at runtime. This requirement addresses the provisioning of default values + during initial deployment. See :need:`feat_req__persistency__tooling`. + +.. feat_req:: Retrieval of default values + :id: feat_req__persistency__default_value_get + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support retrieval of the default value associated with a key. + +.. feat_req:: Reset to default values + :id: feat_req__persistency__reset_to_default + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support reset of individual key or all keys to their default values. + This is only applicable for existing keys that have a predefined default value. + +.. feat_req:: Store persistent data + :id: feat_req__persistency__store_data + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__data_persistency + :status: valid + + The Persistency shall support storing of key-value pairs to persistent storage. + +.. feat_req:: Reset resistant storage + :id: feat_req__persistency__reset_resistant + :reqtype: Functional + :security: YES + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall ensure that write operations are reset resistant to prevent data corruption in case of expected or unexpected interruption. + + .. note:: + As the constant power supply can not be guaranteed in embedded systems, it is essential to ensure that write operations are completed + successfully, or rolled back to the previous state in case of any kind of interruption. + +.. feat_req:: Recovery from reset + :id: feat_req__persistency__recovery_from_reset + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall recover to a consistent state after reset. + + .. note:: + After a reset, the Persistency shall ensure that all key-value pairs are in a consistent state, reflecting either the last successful write operation or the previous consistent state. + +.. feat_req:: Atomic store operation + :id: feat_req__persistency__atomic_store + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support atomic write operation for entire storage to ensure data consistency. + + .. note:: + + Atomic write operation guarantee that either all key-value pairs are written, or no changes are made at all. + This is required to prevent malfunctions when individual key-value pairs are dependent on each other. + +.. feat_req:: Write amplification minimization + :id: feat_req__persistency__write_amplification + :reqtype: Non-Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__app_architectures__support_data + :status: valid + + The Persistency shall minimize the write amplification during data storage operations to enhance performance and prolong the lifespan of the underlying storage medium. + + .. note:: + Write amplification refers to the phenomenon where the amount of data written to the storage medium exceeds the amount of user data intended to be written. + Minimizing write amplification is crucial for optimizing performance and reducing wear on storage devices, especially in embedded systems with limited number of program-erase cycles. + +.. feat_req:: Load persistent data + :id: feat_req__persistency__load_data + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__data_persistency + :status: valid + + The Persistency shall support loading of key-value pairs from persistent storage. + +.. feat_req:: Cached access + :id: feat_req__persistency__cached_access + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support caching mechanisms to improve access times for frequently accessed key-value pairs. + +.. feat_req:: Direct access + :id: feat_req__persistency__direct_access + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__dependability__availability + :status: valid + + The Persistency shall support direct access to key-value pairs without the necessity to load the entire storage to RAM in advance. + + .. note:: + Direct access improves availability of data and reduces memory consumption for large data sets. + +.. feat_req:: Integrity check + :id: feat_req__persistency__integrity_check + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall detect and report data integrity issues. + +.. feat_req:: Confidential storage + :id: feat_req__persistency__confidential_storage + :reqtype: Functional + :security: YES + :safety: QM + :satisfies: stkh_req__functional_req__data_persistency + :status: valid + + The Persistency shall support confidential storage of key-value pairs using encryption mechanisms. + + .. note:: + Confidential storage is essential to protect sensitive data from unauthorized access, especially in scenarios where the storage medium may be exposed to potential threats. + +.. feat_req:: Multiple storage backends + :id: feat_req__persistency__storage_backends + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support multiple storage backends. + More than one storage backend of the same type shall be optionally supported for the sake of redundancy. + The storage backends shall be compile time configurable for each KVS instance. + + .. note:: + Storage backend represents an abstraction for the underlying storage format and mechanism. + Configurable storage backends allow the user to select the most suitable solution for their specific use case (Performance, easy of use, resource consumption, ...). + +.. feat_req:: Asynchronous operation + :id: feat_req__persistency__async_api + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__dependability__availability, stkh_req__app_architectures__support_request + :status: valid + + The Persistency shall provide an asynchronous API for time consuming operations like loading and storing of data. + +.. feat_req:: Signalling completion of asynchronous operation + :id: feat_req__persistency__async_completion + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__app_architectures__support_data + :status: valid + + The Persistency shall provide a mechanism to signal the completion of an asynchronous operations to the application. + +.. feat_req:: Snapshot create + :id: feat_req__persistency__snapshot_create + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support explicit creation of snapshots. Snapshots are identified by unique IDs. + Snapshots shall also include the version of the data layout. See :need:`feat_req__persistency__versioning`. + + .. note:: + Snapshots are point-in-time, read-only view on all key-value pairs at moment of snapshot creation. They are typically used for backup and rollback purposes. + Implicit snapshots (e.g. created during store operation) shall be prevented to reduce storage consumption. + +.. feat_req:: Snapshot restore + :id: feat_req__persistency__snapshot_restore + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support explicit restoration of snapshots. + +.. feat_req:: Snapshot remove + :id: feat_req__persistency__snapshot_remove + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support explicit removal of snapshots. + +.. feat_req:: Intra-Process data access + :id: feat_req__persistency__concurrency + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__communication__intra_process + :status: valid + + The Persistency shall support concurrent access to key-value pairs from multiple threads within the same process. + +.. feat_req:: Versioning + :id: feat_req__persistency__versioning + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall support versioning for different data representation of KVS. + + .. note:: + Versioning is essential to ensure compatibility between different versions of the Persistency and the stored data. + Each version shall be uniquely identifiable and include information of the data layout and structure. + +.. feat_req:: Update Mechanism + :id: feat_req__persistency__update_mechanism + :reqtype: Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__overall_goals__variant_management + :status: valid + + The Persistency shall implement mechanisms to upgrade from one version to another, including multi-version jumps. + +.. feat_req:: Random access time + :id: feat_req__persistency__fast_access + :reqtype: Non-Functional + :security: NO + :safety: QM + :satisfies: stkh_req__execution_model__short_app_cycles + :status: valid + + The Persistency shall ensure that random read access for key-value pair is completed with constant or logarithmic time complexity relative to the number of stored key-value pairs. + +.. feat_req:: Tooling + :id: feat_req__persistency__tooling + :reqtype: Non-Functional + :security: NO + :safety: ASIL_B + :satisfies: stkh_req__functional_req__support_of_store + :status: valid + + The Persistency shall provide tooling support for: + + - viewing and modifying key-value pairs during development, testing and debugging + - provisioning of default values via external file + +.. feat_req:: Support development mode + :id: feat_req__persistency__dev_mode + :reqtype: Functional + :security: YES + :safety: ASIL_B + :satisfies: stkh_req__dependability__safety_features_11 + :status: valid + + The Persistency shall support the development mode. + The development mode shall allow unrestricted data access and bypass security policies. + +.. feat_req:: Support production mode + :id: feat_req__persistency__prod_mode + :reqtype: Functional + :security: YES + :safety: ASIL_B + :satisfies: stkh_req__dependability__safety_features_11 + :status: valid + + The Persistency shall support the production mode. + The production mode should enforce the most restrictive data access controls feasible. + +.. needextend:: is_external == False and "persistency/requirements" in docname + :+tags: persistency +