Skip to content

Commit 844575b

Browse files
FrawlessFrawless
committed
Update docs and improve comment validation
Signed-off-by: Jakub Stejskal <xstejs24@gmail.com> # Conflicts: # .github/docs/README.md # .github/workflows/run-system-tests.yml # .github/workflows/system-tests.yml
1 parent 2c74682 commit 844575b

4 files changed

Lines changed: 140 additions & 60 deletions

File tree

‎.github/actions/systemtests/parse-comment/action.yml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,14 @@ runs:
7777
// Auto-run smoke tests on PR workflow changes
7878
shouldRun = true;
7979
} else if (ev === 'issue_comment' && context.payload?.issue?.pull_request) {
80-
shouldRun = /^\s*\/gha\s+run\b/i.test(body || '');
80+
// For issue comments on PRs, check if PR is closed first
81+
const issueState = context.payload?.issue?.state;
82+
if (issueState === 'closed') {
83+
core.info(`PR #${context.payload.issue.number} is closed, skipping execution`);
84+
shouldRun = false;
85+
} else {
86+
shouldRun = /^\s*\/gha\s+run\b/i.test(body || '');
87+
}
8188
}
8289
8390
core.setOutput('shouldRun', String(shouldRun));

‎.github/docs/README.md‎

Lines changed: 40 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Building Strimzi
1+
# Build & Test Strimzi in GitHub Actions
22
The following lines put together basic information how is Strimzi built and tested via GitHub Actions.
33
Currently, our main build system still lives on Azure Pipelines and running builds and tests on GHA is in experimental phase.
44
All the steps mentioned bellow re-use scripts from our [azure](../../.azure) directory and thus all the steps are more or less the same on both systems.
@@ -69,6 +69,7 @@ System tests execution actions:
6969
- [set-defaults](../actions/utils/set-defaults)
7070
- [log-variables](../actions/utils/log-variables)
7171
- [parse-comment](../actions/systemtests/parse-comment)
72+
- [determine-ref](../actions/determine-ref)
7273

7374
Utils actions:
7475
- [check-permissions](../actions/utils/check-permissions)
@@ -87,7 +88,8 @@ The actions also had to be put together into workflow as we have in Azure:
8788
- `operator-release pipeline` -> `operator-release workflow`
8889

8990
## Running system tests
90-
With GitHub Actions we are now able to propagate a specific parameters to our e2e jobs which allow us to just run a subset of tests for example or run it against different kubernetes version etc.
91+
With GitHub Actions we are now able to propagate a specific parameters to our e2e jobs.
92+
That allow us to run a subset of tests for example or run it against different kubernetes version etc.
9193

9294
### Triggers
9395
There are two options how the workflows could be triggered - manually via GitHub UI, via issue comment.
@@ -97,32 +99,40 @@ Comment for triggering the workflow has to starts with string `/gha run` and the
9799
The whole script that parse the trigger even is part of [parse-comment](../actions/systemtests/parse-comment) action.
98100
Currently, we have these parameters that can be passed through the comment:
99101

100-
| Name | Info | Default |
101-
|-------------------------------|------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
102-
| pipeline | Name of the pipeline from [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) that wil be executed | regression,upgrade,performance |
103-
| profile | Testing profile from pom that will be executed | regression,upgrade,performance |
104-
| agent | Agent that will be used for a specific pipeline (see list of runners in Strimzi org config for more info) | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
105-
| strimzi_feature_gates | Which Strimzi Feature Gates will be used | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
106-
| strimzi_rbac_scope | RBAC scope for Strimzi | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
107-
| cluster_operator_install_type | How Strimzi will be installed during the tests | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
108-
| parallel | Number of tests that will be executed in parallel | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
109-
| architecture | Which architecture will be used (should match with agent arch) | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
110-
| groups | Which Junit5 groups will be executed | all |
111-
| tests | Which Junit5 tests will be executed | all |
112-
| kubeVersion | Used Kubernetes version as part of Kind/Minikube setup | The one set as default in setup scripts |
113-
| kafkaVersion | Which Kafka version will be used in the tests | Default one from STs config |
114-
115-
Note that parameters are passed only to a `custom` pipeline except `kafkaVersion` and `kubeVersion` that are used for all jobs.
116-
117-
For trigger via GitHub UI you can specify `releaseVersion`, `kafkaVersion`, and `profile`.
118-
The first two parameters are working in the same manner as in Azure.
119-
`profile` is used to filter out all pipelines defined in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) that contains one of the profile specified in comma separated list in `profile` parameter.
102+
| Name | Info | Default |
103+
|-------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|
104+
| pipeline | Name of the pipeline from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that wil be executed | regression,upgrade |
105+
| profile | Testing profile from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that will be executed, it has to be defined in pom file | smoke |
106+
| agent | Agent that will be used for a specific pipeline (see list of runners in Strimzi org config for more info) | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
107+
| strimzi_feature_gates | Which Strimzi Feature Gates will be used | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
108+
| strimzi_rbac_scope | RBAC scope for Strimzi | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
109+
| cluster_operator_install_type | How Strimzi will be installed during the tests | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
110+
| parallel | Number of tests that will be executed in parallel | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
111+
| architecture | Which architecture will be used (should match with agent arch) | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
112+
| groups | Which Junit5 groups will be executed | all |
113+
| tests | Which Junit5 tests will be executed | all |
114+
| kubeVersion | Used Kubernetes version as part of Kind/Minikube setup | The one set as default in setup scripts |
115+
| kafkaVersion | Which Kafka version will be used in the tests | Default one from STs config |
116+
117+
The process of parameter usage is as follows:
118+
- `pipeline` has the highest priority. If `pipeline` is defined, the jobs will be loaded with data from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that match specific _pipeline_.
119+
- `profile` has the second-highest priority. If `profile` is defined, the jobs will be loaded with data from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that match specific _profile_.
120+
- `agent` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
121+
- `strimzi_feature_gates` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
122+
- `strimzi_rbac_scope` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
123+
- `cluster_operator_install_type` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
124+
- `parallel` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
125+
- `architecture` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
126+
- `groups` are passed directly to `mvn` command. It should be used mostly only with `custom` pipeline as otherwise a specific `pipeline` could have the `groups` excluded.
127+
- `tests` are passed directly to `mvn` command. It should be used mostly only with `custom` pipeline as otherwise a specific `pipeline` could have the `tests` excluded.
128+
- `kubeVersion` is used for setup `kind` for every running jobs.
129+
- `kafkaVersion` is used for every running job.
120130

121131
### Matrix generation
122-
Once the event is parsed the mechanism will decide whether Strimzi should be built or already existing images will be used (release rc for example).
123-
After that job matrix will be generated [run-system-tests](../workflows/run-system-tests.yml) workflow will be invoked for each job defined in the matrix.
132+
Once the event is parsed the mechanism will decide whether Strimzi should be built or already existing images will be used (release RC for example).
133+
After that job matrix will be generated and [run-system-tests](../workflows/run-system-tests.yml) workflow will be invoked for each job defined in the matrix.
124134

125-
Matrix is generated by action [generate-matrix](../actions/systemtests/generate-matrix) either for custom pipeline mentioned above or for pipelines defined in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml).
135+
Matrix is generated by action [generate-matrix](../actions/systemtests/generate-matrix) either by filtering pipelines/profiles defined in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) or by creating a custom pipeline from passed params.
126136

127137
### Workflow depiction
128138
![sts-workflow.png](sts-workflow.png)
@@ -138,3 +148,8 @@ Each first-time contributor has to be approved by one of the maintainers which s
138148

139149
Regarding secrets, the forks are not allowed to use secrets by default.
140150
Every generated `GITHUB_TOKEN` has only read access to the repo/org without access to the secrets.
151+
152+
## Testing workflows and actions
153+
Unit and integration tests invoked via [actions-tests.yml](../workflows/actions-tests.yml) workflow.
154+
It uses files specified within [tests](../tests) folder and via [act](https://github.com/nektos/act) it tries to execute the actions and check the outputs.
155+
Currently, we tests `check-permissions`, `generate-matrix`, and `parse-comment` actions.

‎.github/workflows/run-system-tests.yml‎

Lines changed: 5 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -79,35 +79,16 @@ on:
7979
description: "Repository ref that will be checkout"
8080
required: true
8181
type: string
82+
matrix:
83+
description: "JSON matrix generated by validate-matrix action"
84+
required: true
85+
type: string
8286

8387
jobs:
84-
generate-matrix:
85-
runs-on: ubuntu-latest
86-
outputs:
87-
matrix: ${{ steps.gen.outputs.matrix }}
88-
steps:
89-
- uses: actions/checkout@v5
90-
with:
91-
ref: ${{ inputs.ref }}
92-
- name: Generate Matrix
93-
id: gen
94-
uses: ./.github/actions/systemtests/generate-matrix
95-
with:
96-
pipelines: ${{ inputs.pipelineList }}
97-
profiles: ${{ inputs.profileList }}
98-
default_agent: ${{ inputs.agent }}
99-
default_arch: ${{ inputs.architecture }}
100-
default_strimzi_feature_gates: ${{ inputs.strimzi_feature_gates }}
101-
default_strimzi_rbac_scope: ${{ inputs.strimzi_rbac_scope }}
102-
default_cluster_operator_install_type: ${{ inputs.cluster_operator_install_type }}
103-
default_parallel: ${{ inputs.parallel }}
104-
runnerArch: "amd64"
105-
10688
run-tests:
107-
needs: generate-matrix
10889
strategy:
10990
matrix:
110-
config: ${{ fromJson(needs.generate-matrix.outputs.matrix) }}
91+
config: ${{ fromJson(inputs.matrix) }}
11192
fail-fast: false
11293
# Max timeout ise set for 10h for the whole job, test execution has lower timeouts to not block the job for the whole time
11394
# but have some additional time to collect logs, etc.

0 commit comments

Comments
 (0)