Feat: Add MLflow Analytics provider#6266
Feat: Add MLflow Analytics provider#6266WeichenXu123 wants to merge 3 commits intoFlowiseAI:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request integrates MLflow as an analytic provider, adding credential definitions, UI assets, and tracing logic within the AnalyticHandler for chains, LLMs, and tools. The review feedback highlights a critical concurrency issue where the global initialization of MLflow could lead to configuration leaks between requests. Additionally, several bugs were identified where span maps are overwritten rather than merged, potentially causing data loss in complex execution flows, and a recommendation was made to improve type safety for configuration options.
| if (mlflowUsername) mlflowOptions.trackingServerUsername = mlflowUsername | ||
| if (mlflowPassword) mlflowOptions.trackingServerPassword = mlflowPassword | ||
|
|
||
| mlflow.init(mlflowOptions) |
There was a problem hiding this comment.
The mlflow.init() function configures a global singleton in the MLflow SDK. Since AnalyticHandler instances are created per chatId, concurrent requests with different MLflow configurations (e.g., different tracking URIs or credentials) will overwrite each other's settings. This can lead to traces being sent to the wrong tracking server in a multi-user environment. If the MLflow SDK supports it, consider using a client-based approach to create independent instances instead of global initialization.
| } | ||
|
|
||
| const spanId = chainSpan.spanId | ||
| this.handlers['mlflow'].chainSpan = { [spanId]: chainSpan } |
There was a problem hiding this comment.
The current implementation overwrites the chainSpan map every time a new chain starts. This will cause parent spans or concurrent spans to be lost, preventing them from being correctly retrieved (e.g., for nesting) or ended. Use object spreading to preserve existing spans in the map.
| this.handlers['mlflow'].chainSpan = { [spanId]: chainSpan } | |
| this.handlers['mlflow'].chainSpan = { ...this.handlers['mlflow'].chainSpan, [spanId]: chainSpan } |
References
- Spreading null or undefined within an object literal is safe and does not require a ?? {} fallback, as it evaluates to an empty object.
| }) | ||
|
|
||
| const spanId = llmSpan.spanId | ||
| this.handlers['mlflow'].llmSpan = { [spanId]: llmSpan } |
There was a problem hiding this comment.
The llmSpan map is being overwritten, which will cause issues if multiple LLM calls occur within the same trace (e.g., in a loop or multi-agent setup). Use object spreading to maintain all active spans.
| this.handlers['mlflow'].llmSpan = { [spanId]: llmSpan } | |
| this.handlers['mlflow'].llmSpan = { ...this.handlers['mlflow'].llmSpan, [spanId]: llmSpan } |
References
- Spreading null or undefined within an object literal is safe and does not require a ?? {} fallback, as it evaluates to an empty object.
| }) | ||
|
|
||
| const spanId = toolSpan.spanId | ||
| this.handlers['mlflow'].toolSpan = { [spanId]: toolSpan } |
There was a problem hiding this comment.
The toolSpan map is being overwritten. This will lead to data loss if multiple tools are executed within the same request context. Use object spreading to preserve the spans.
| this.handlers['mlflow'].toolSpan = { [spanId]: toolSpan } | |
| this.handlers['mlflow'].toolSpan = { ...this.handlers['mlflow'].toolSpan, [spanId]: toolSpan } |
References
- Spreading null or undefined within an object literal is safe and does not require a ?? {} fallback, as it evaluates to an empty object.
| const mlflowPassword = getCredentialParam('mlflowPassword', credentialData, this.nodeData) | ||
| const mlflowExperimentId = providerConfig.experimentId as string | ||
|
|
||
| const mlflowOptions: any = { |
There was a problem hiding this comment.
There was a problem hiding this comment.
Pull request overview
Adds MLflow as a new analytics/tracing provider alongside existing providers (LangSmith/Langfuse/Lunary/LangWatch/Arize/Phoenix/Opik), enabling Flowise to emit chain/LLM/tool traces to an MLflow tracking server via the @mlflow/core SDK.
Changes:
- Add
@mlflow/coredependency and lockfile updates for MLflow + transitive OpenTelemetry/Databricks SDK packages. - Add MLflow provider configuration to the UI analytics settings (including optional experiment ID).
- Implement MLflow span lifecycle wiring in
AnalyticHandler(CHAIN/LLM/TOOL spans + flush on shutdown) and add an MLflow credential definition.
Reviewed changes
Copilot reviewed 4 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds @mlflow/core and transitive dependencies needed for MLflow tracing |
| packages/components/package.json | Declares @mlflow/core dependency for components package |
| packages/components/src/handler.ts | Implements MLflow provider init + CHAIN/LLM/TOOL span start/end/error handling |
| packages/components/credentials/MLflowApi.credential.ts | Adds credential schema for MLflow tracking server connection/auth |
| packages/ui/src/ui-component/extended/AnalyseFlow.jsx | Adds MLflow as an analytics provider option in the UI |
| packages/ui/src/assets/images/mlflow.svg | Adds MLflow icon asset |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| '@databricks/sdk-experimental@0.15.0': | ||
| resolution: {integrity: sha512-HkoMiF7dNDt6WRW0xhi7oPlBJQfxJ9suJhEZRFt08VwLMaWcw2PiF8monfHlkD4lkufEYV6CTxi5njQkciqiHA==} | ||
| engines: {node: '>=22.0', npm: '>=10.0.0'} | ||
|
|
There was a problem hiding this comment.
@mlflow/core pulls in @databricks/sdk-experimental@0.15.0, which declares engines: { node: '>=22.0' }. This repo targets Node 20 (.nvmrc v20.19.2; root package.json allows ^20 and even 18.x), so enabling MLflow/Databricks tracing may break at runtime on supported Node versions. Consider using an @mlflow/core (or Databricks SDK) version compatible with Node 20/18, or add a pnpm override to a Node-20-compatible Databricks SDK (and verify MLflow still works), rather than introducing a Node >=22-only transitive dependency.
Summary
Adds MLflow as an analytics/tracing provider in Flowise, allowing users to send LLM traces to an MLflow tracking server. Traces are logged as MLflow spans with proper parent-child hierarchy (CHAIN → LLM/TOOL), token usage, and model metadata.
Uses the official https://www.npmjs.com/package/@mlflow/core TypeScript SDK (^0.2.0).
Motivation
Flowise already supports analytics providers like LangSmith, Langfuse, LunaryAI, and Opik, but lacked support for the popular MLflow tracing provider.
Provider setup