Skip to content

fix: preserve metric snapshots when async metric tasks fail in indicator#2560

Open
aerosta wants to merge 1 commit intoconfident-ai:mainfrom
aerosta:fix/async-metric-failure-snapshot
Open

fix: preserve metric snapshots when async metric tasks fail in indicator#2560
aerosta wants to merge 1 commit intoconfident-ai:mainfrom
aerosta:fix/async-metric-failure-snapshot

Conversation

@aerosta
Copy link
Copy Markdown
Contributor

@aerosta aerosta commented Mar 18, 2026

Summary

measure_metrics_with_indicator used bare asyncio.gather(*tasks). If one metric raises an exception, sibling metric tasks could be canceled before completing. The caller then snapshots the metric state in finally, which could leave unfinished metrics with None values.

This change uses return_exceptions=True, waits for all metric tasks to finish, and re-raises the first exception. This preserves existing error propagation while ensuring metric snapshots reflect the final task state.

Context

PRs #2136 and #2247 applied the same pattern in execute.py, but the two gather calls in indicator.py were still using bare asyncio.gather(...). This change closes that remaining gap.

Related: #2128, #2298

Changes

  • add wait_for_metric_tasks in deepeval/metrics/indicator.py
  • update both gather call sites in measure_metrics_with_indicator
  • add a regression test showing that a slow metric's final state is captured even when a sibling metric raises

@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 18, 2026

@aerosta is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant