Agent Optimization Page

[dbczumar]

No description provided.

[github-actions]

🚀 Netlify Preview Deployed!

Preview URL: https://pr-514--test-mlflow-website.netlify.app

Details

PR: #514
Build Action: https://github.com/mlflow/mlflow-website/actions/runs/24216524355
Deploy Action: https://github.com/mlflow/mlflow-website/actions/runs/24216597905

This preview will be updated automatically on new commits.

[AnanyaDBJ]

PR Review: Agent Optimization Page (#514)

Nice work getting these pages up — the SEO structure (JSON-LD, Open Graph, canonical URLs) and overall layout are solid. A few suggestions to strengthen the content and code quality:

Content

1. "Agent Optimization Techniques" section reads as problems, not techniques

The "Why Agent Optimization Matters" section already presents the Problem/Solution pairs (compounding costs, unreliable decision-making, cascading latency, too many knobs). The "Techniques" section then repeats the same framing: "Cut compounding costs", "Fix unreliable decision-making", "Reduce cascading latency." This is redundant — it restates the challenges rather than introducing distinct techniques.

Suggestion: Restructure this section around concrete, actionable techniques that MLflow actually supports today. The reader should leave this section knowing what to do, not what's wrong.

2. Use cases are too narrow and the page over-indexes on prompt optimization

The use cases section lists 4 specific patterns (RAG agents, multi-step tool use, chatbots, agents at scale), but misses the core argument: agent optimization matters most for compound systems with multiple moving parts — LLMs, tools, MCPs, sub-agents, retrieval — where errors propagate and small inefficiencies multiply. Autonomous agentic workflows should be front and center here. The more compound the system, the more optimization matters — the goal is agents making better decisions right the first time.

The page also leans heavily on prompt optimization as the technique. But agent optimization is broader: tool selection strategy, sub-agent orchestration, model routing per step, caching, execution graph optimization. The LLM optimization page already covers prompt optimization — repeating it here blurs the distinction between the two pages.

3. Missing key differentiator: human feedback-driven optimization

MLflow's agent optimization is grounded in human judgment — humans review traces, score agent outputs, and that human signal drives the optimization loop. This is completely absent from the page and it's arguably our strongest differentiator.

Other approaches rely solely on automated metrics or LLM-as-judge scores. MLflow uniquely closes the loop with human assessments: humans evaluate agent decisions → those judgments become training signal → optimization algorithms improve the agent based on what humans actually want. For agents making consequential decisions (not just generating text), optimization backed by human feedback is fundamentally more trustworthy than optimization backed only by another LLM's opinion.

This should be called out prominently — both in the techniques section and in the Open Source vs. Proprietary comparison.

4. Open Source vs. Proprietary section undersells MLflow's position

The current copy frames this as a generic open-source-vs-SaaS tradeoff (cost, flexibility, data ownership). That's an observability argument, not an optimization argument.

MLflow's actual differentiator is that it's the only tool — open or proprietary — that offers a complete agent optimization stack: tracing to identify problems, evaluation (with human feedback) to measure them, prompt optimization algorithms to fix them, and AI Gateway to control costs. Other tools (LangSmith, Braintrust, Arize) offer agent observability — they help you watch agents. None of them close the loop with human feedback-driven automated optimization. The copy should make this distinction explicitly and lead with it.

Code Quality

5. ~200 lines of CSS duplicated between both pages

agent-optimization.tsx and llm-optimization.tsx have nearly identical <style> blocks. This should be extracted to a shared CSS module or stylesheet — either in this PR or a fast follow-up.

6. 38 !important declarations across the two files

This many overrides suggest the styles are fighting Docusaurus globals rather than working with them. CSS modules or scoped styles would be a cleaner approach and more maintainable long-term.

7. Missing /llm-optimization link from FAQ page

The faq.tsx update adds a card for /agent-optimization but not /llm-optimization. Looks like an oversight — both pages should be discoverable from the FAQ.