[Feature] Add optional trace output to the RAG answer API for retrieval debugging

[LRriver]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Feature Description (功能描述)

Feature Description

The /rag answer endpoint currently returns final answer fields only. For
retrieval debugging, users need an optional trace/debug response that includes
intermediate graph-recall information such as extracted keywords, matched vertex
IDs, generated Gremlin, graph recall results, and related graph debug fields.

The trace should be opt-in so the default response remains compact and backward
compatible. Vector recall and rerank traces can be handled as follow-up work
after the graph trace contract is stable.

Current verification

• /rag has an explicit TODO saying more response information is needed for users to understand query logic in hugegraph-llm/src/hugegraph_llm/api/rag_api.py:79.
• /rag currently returns only query and requested answer fields in hugegraph-llm/src/hugegraph_llm/api/rag_api.py:80.
• /rag/graph already returns graph-recall intermediate fields including query, keywords, match_vids, graph_result_flag, gremlin, graph_result, and vertex_degree_list in hugegraph-llm/src/hugegraph_llm/api/rag_api.py:131.
• RAGRequest does not currently define an include_trace or similar option.

Suggested scope

• Add an opt-in request field, for example include_trace: bool = False.
• Extend the graph-related RAG flow return contract or add a structured side channel so /rag can include graph trace data without parsing logs.
• Reuse existing /rag/graph field names where possible.
• Keep vector recall and rerank trace fields out of the required first version unless the implementation already exposes them cleanly.
• Avoid leaking secrets or full prompt contents by default.
• Keep the default /rag response compatible for existing users.

Mermaid reference

flowchart LR
    Request[/POST /rag include_trace/]
    Request --> RAG[RAG answer flow]
    RAG --> Answer[Answer fields]
    RAG --> Trace{include_trace?}
    Trace -->|false| Compact[Current response shape]
    Trace -->|true| Debug[trace: keywords, match_vids, gremlin, graph_result]
    Answer --> Compact
    Answer --> Debug

Loading

Acceptance criteria

• /rag without include_trace keeps the current response shape.
• /rag with include_trace=true returns a structured trace object.
• Trace includes graph recall fields when graph_only or graph_vector_answer is used.
• Vector recall and rerank trace fields are optional follow-up fields, not required for the first version.
• Missing unavailable trace sections are omitted or returned as empty structured values consistently.

Suggested tests

• API contract test for default /rag response compatibility.
• API test for include_trace=true.
• Flow-level test showing trace data is populated from graph recall.
• Test that sensitive config values are not included in trace.

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct