Skip to content

Feat: Add MLflow Analytics provider#6266

Open
WeichenXu123 wants to merge 3 commits intoFlowiseAI:mainfrom
WeichenXu123:mlflow-tracing
Open

Feat: Add MLflow Analytics provider#6266
WeichenXu123 wants to merge 3 commits intoFlowiseAI:mainfrom
WeichenXu123:mlflow-tracing

Conversation

@WeichenXu123
Copy link
Copy Markdown

Summary

Adds MLflow as an analytics/tracing provider in Flowise, allowing users to send LLM traces to an MLflow tracking server. Traces are logged as MLflow spans with proper parent-child hierarchy (CHAIN → LLM/TOOL), token usage, and model metadata.

Uses the official https://www.npmjs.com/package/@mlflow/core TypeScript SDK (^0.2.0).

Motivation

Flowise already supports analytics providers like LangSmith, Langfuse, LunaryAI, and Opik, but lacked support for the popular MLflow tracing provider.

Provider setup

130f0577-363c-4592-b31f-c9b84dadcecd

Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
Copilot AI review requested due to automatic review settings April 22, 2026 11:46
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates MLflow as an analytic provider, adding credential definitions, UI assets, and tracing logic within the AnalyticHandler for chains, LLMs, and tools. The review feedback highlights a critical concurrency issue where the global initialization of MLflow could lead to configuration leaks between requests. Additionally, several bugs were identified where span maps are overwritten rather than merged, potentially causing data loss in complex execution flows, and a recommendation was made to improve type safety for configuration options.

if (mlflowUsername) mlflowOptions.trackingServerUsername = mlflowUsername
if (mlflowPassword) mlflowOptions.trackingServerPassword = mlflowPassword

mlflow.init(mlflowOptions)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The mlflow.init() function configures a global singleton in the MLflow SDK. Since AnalyticHandler instances are created per chatId, concurrent requests with different MLflow configurations (e.g., different tracking URIs or credentials) will overwrite each other's settings. This can lead to traces being sent to the wrong tracking server in a multi-user environment. If the MLflow SDK supports it, consider using a client-based approach to create independent instances instead of global initialization.

}

const spanId = chainSpan.spanId
this.handlers['mlflow'].chainSpan = { [spanId]: chainSpan }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current implementation overwrites the chainSpan map every time a new chain starts. This will cause parent spans or concurrent spans to be lost, preventing them from being correctly retrieved (e.g., for nesting) or ended. Use object spreading to preserve existing spans in the map.

Suggested change
this.handlers['mlflow'].chainSpan = { [spanId]: chainSpan }
this.handlers['mlflow'].chainSpan = { ...this.handlers['mlflow'].chainSpan, [spanId]: chainSpan }
References
  1. Spreading null or undefined within an object literal is safe and does not require a ?? {} fallback, as it evaluates to an empty object.

})

const spanId = llmSpan.spanId
this.handlers['mlflow'].llmSpan = { [spanId]: llmSpan }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The llmSpan map is being overwritten, which will cause issues if multiple LLM calls occur within the same trace (e.g., in a loop or multi-agent setup). Use object spreading to maintain all active spans.

Suggested change
this.handlers['mlflow'].llmSpan = { [spanId]: llmSpan }
this.handlers['mlflow'].llmSpan = { ...this.handlers['mlflow'].llmSpan, [spanId]: llmSpan }
References
  1. Spreading null or undefined within an object literal is safe and does not require a ?? {} fallback, as it evaluates to an empty object.

})

const spanId = toolSpan.spanId
this.handlers['mlflow'].toolSpan = { [spanId]: toolSpan }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The toolSpan map is being overwritten. This will lead to data loss if multiple tools are executed within the same request context. Use object spreading to preserve the spans.

Suggested change
this.handlers['mlflow'].toolSpan = { [spanId]: toolSpan }
this.handlers['mlflow'].toolSpan = { ...this.handlers['mlflow'].toolSpan, [spanId]: toolSpan }
References
  1. Spreading null or undefined within an object literal is safe and does not require a ?? {} fallback, as it evaluates to an empty object.

const mlflowPassword = getCredentialParam('mlflowPassword', credentialData, this.nodeData)
const mlflowExperimentId = providerConfig.experimentId as string

const mlflowOptions: any = {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using any for mlflowOptions. It is better to use a specific type from the @mlflow/core package or at least Record<string, any> to maintain better type safety and clarity.

Suggested change
const mlflowOptions: any = {
const mlflowOptions: Record<string, any> = {

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds MLflow as a new analytics/tracing provider alongside existing providers (LangSmith/Langfuse/Lunary/LangWatch/Arize/Phoenix/Opik), enabling Flowise to emit chain/LLM/tool traces to an MLflow tracking server via the @mlflow/core SDK.

Changes:

  • Add @mlflow/core dependency and lockfile updates for MLflow + transitive OpenTelemetry/Databricks SDK packages.
  • Add MLflow provider configuration to the UI analytics settings (including optional experiment ID).
  • Implement MLflow span lifecycle wiring in AnalyticHandler (CHAIN/LLM/TOOL spans + flush on shutdown) and add an MLflow credential definition.

Reviewed changes

Copilot reviewed 4 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pnpm-lock.yaml Adds @mlflow/core and transitive dependencies needed for MLflow tracing
packages/components/package.json Declares @mlflow/core dependency for components package
packages/components/src/handler.ts Implements MLflow provider init + CHAIN/LLM/TOOL span start/end/error handling
packages/components/credentials/MLflowApi.credential.ts Adds credential schema for MLflow tracking server connection/auth
packages/ui/src/ui-component/extended/AnalyseFlow.jsx Adds MLflow as an analytics provider option in the UI
packages/ui/src/assets/images/mlflow.svg Adds MLflow icon asset
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pnpm-lock.yaml
Comment on lines +3825 to +3828
'@databricks/sdk-experimental@0.15.0':
resolution: {integrity: sha512-HkoMiF7dNDt6WRW0xhi7oPlBJQfxJ9suJhEZRFt08VwLMaWcw2PiF8monfHlkD4lkufEYV6CTxi5njQkciqiHA==}
engines: {node: '>=22.0', npm: '>=10.0.0'}

Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mlflow/core pulls in @databricks/sdk-experimental@0.15.0, which declares engines: { node: '>=22.0' }. This repo targets Node 20 (.nvmrc v20.19.2; root package.json allows ^20 and even 18.x), so enabling MLflow/Databricks tracing may break at runtime on supported Node versions. Consider using an @mlflow/core (or Databricks SDK) version compatible with Node 20/18, or add a pnpm override to a Node-20-compatible Databricks SDK (and verify MLflow still works), rather than introducing a Node >=22-only transitive dependency.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants