Describe the bug
The non-blocking error handling introduced in #2043 can mark an error as resolved even though the underlying operation is still failing. PersistentErrorTracker.UpdateFromRegistry treats an error as resolved whenever it's absent from a reconciliation cycle's ErrorRegistry. But a workflow run only exercises the reconcilers whose subscription matches the triggering event, so an error recorded by reconciler A gets cleared by a workflow triggered by an unrelated event B, where A never ran and never re-recorded its still-failing error. The retry loop can therefore give up before the underlying condition clears, and the error is dropped from the tracker without having actually been resolved.
Surfaced while validating #2088 (fix for #1578).
To Reproduce
Steps to reproduce the behavior:
- Deploy the operator to a cluster (
make local-setup).
- Remove the
create verb from the authorinos rule in the kuadrant-operator-manager-role ClusterRole, so the operator can no longer create Authorino resources.
# strip "create" from the authorinos rule in the operator's ClusterRole
kubectl get clusterrole kuadrant-operator-manager-role -o json | \
jq --arg idx "$(kubectl get clusterrole kuadrant-operator-manager-role -o json | jq '.rules | map(.resources // [] | contains(["authorinos"])) | index(true)')" \
'.rules[$idx|tonumber].verbs |= map(select(. != "create"))' | kubectl apply -f -
- Create a
Kuadrant CR (this also generates unrelated Limitador reconcile events):
kubectl apply -f - <<'EOF'
apiVersion: kuadrant.io/v1beta1
kind: Kuadrant
metadata:
name: kuadrant
namespace: kuadrant-system
spec: {}
EOF
- Watch the operator logs, do not restore the RBAC. The Authorino create keeps failing, yet the error is marked
resolved in between failures:
{"level":"error","ts":"2026-07-10T09:00:05.232Z","logger":"kuadrant-operator.AuthorinoReconciler","msg":"failed to create authorino resource","error":"...forbidden..."}
{"level":"info","ts":"2026-07-10T09:00:05.310Z","logger":"kuadrant-operator.PersistentErrorTracker","msg":"non-blocking error resolved","key":"Authorino.operator.authorino.kuadrant.io/kuadrant-system/authorino/create","attempts":2}
{"level":"error","ts":"2026-07-10T09:00:05.377Z","logger":"kuadrant-operator.AuthorinoReconciler","msg":"failed to create authorino resource","error":"...forbidden..."}
{"level":"error","ts":"2026-07-10T09:00:07.387Z","logger":"kuadrant-operator.AuthorinoReconciler","msg":"failed to create authorino resource","error":"...forbidden..."}
{"level":"info","ts":"2026-07-10T09:00:07.465Z","logger":"kuadrant-operator.PersistentErrorTracker","msg":"non-blocking error resolved","key":"Authorino.operator.authorino.kuadrant.io/kuadrant-system/authorino/create","attempts":2}
Expected behavior
An error should only be marked resolved when the operation that recorded it actually succeeds (or is genuinely no longer needed) not merely because a reconcile cycle triggered by an unrelated event didn't re-record it. Errors whose recording reconciler didn't run in a given cycle should be left untouched, so retries continue until the underlying condition actually clears.
Additional context
The non-blocking error resolved line in the reproduction is emitted by PersistentErrorTracker.UpdateFromRegistry in internal/controller/reconciliation_errors.go, which removes any tracked error that is not present in the current reconciliation cycle's ErrorRegistry.
This is shared error-handling code introduced in #2043; it is not specific to the Authorino or Limitador reconcilers. Because the resolution logic operates on the shared tracker, it applies to any reconciler that records errors into it (e.g. AuthConfigsReconciler and LimitadorLimitsReconciler, also wired in #2043).
The reproduction uses the Authorino create path (from #2088) only because it is the simplest way to inject a persistent, easily-triggered failure.
Describe the bug
The non-blocking error handling introduced in #2043 can mark an error as
resolvedeven though the underlying operation is still failing.PersistentErrorTracker.UpdateFromRegistrytreats an error as resolved whenever it's absent from a reconciliation cycle'sErrorRegistry. But a workflow run only exercises the reconcilers whose subscription matches the triggering event, so an error recorded by reconciler A gets cleared by a workflow triggered by an unrelated event B, where A never ran and never re-recorded its still-failing error. The retry loop can therefore give up before the underlying condition clears, and the error is dropped from the tracker without having actually been resolved.Surfaced while validating #2088 (fix for #1578).
To Reproduce
Steps to reproduce the behavior:
make local-setup).createverb from theauthorinosrule in thekuadrant-operator-manager-roleClusterRole, so the operator can no longer create Authorino resources.KuadrantCR (this also generates unrelated Limitador reconcile events):resolvedin between failures:{"level":"error","ts":"2026-07-10T09:00:05.232Z","logger":"kuadrant-operator.AuthorinoReconciler","msg":"failed to create authorino resource","error":"...forbidden..."} {"level":"info","ts":"2026-07-10T09:00:05.310Z","logger":"kuadrant-operator.PersistentErrorTracker","msg":"non-blocking error resolved","key":"Authorino.operator.authorino.kuadrant.io/kuadrant-system/authorino/create","attempts":2} {"level":"error","ts":"2026-07-10T09:00:05.377Z","logger":"kuadrant-operator.AuthorinoReconciler","msg":"failed to create authorino resource","error":"...forbidden..."} {"level":"error","ts":"2026-07-10T09:00:07.387Z","logger":"kuadrant-operator.AuthorinoReconciler","msg":"failed to create authorino resource","error":"...forbidden..."} {"level":"info","ts":"2026-07-10T09:00:07.465Z","logger":"kuadrant-operator.PersistentErrorTracker","msg":"non-blocking error resolved","key":"Authorino.operator.authorino.kuadrant.io/kuadrant-system/authorino/create","attempts":2}Expected behavior
An error should only be marked
resolvedwhen the operation that recorded it actually succeeds (or is genuinely no longer needed) not merely because a reconcile cycle triggered by an unrelated event didn't re-record it. Errors whose recording reconciler didn't run in a given cycle should be left untouched, so retries continue until the underlying condition actually clears.Additional context
The
non-blocking error resolvedline in the reproduction is emitted byPersistentErrorTracker.UpdateFromRegistryininternal/controller/reconciliation_errors.go, which removes any tracked error that is not present in the current reconciliation cycle'sErrorRegistry.This is shared error-handling code introduced in #2043; it is not specific to the Authorino or Limitador reconcilers. Because the resolution logic operates on the shared tracker, it applies to any reconciler that records errors into it (e.g.
AuthConfigsReconcilerandLimitadorLimitsReconciler, also wired in #2043).The reproduction uses the Authorino create path (from #2088) only because it is the simplest way to inject a persistent, easily-triggered failure.