Support citations and source attribution in MCP responses for RAG-backed apps

[sanpatricky]

Self Checks

• I have read the Contributing Guide and Language Policy.
• I have searched for existing issues, including closed ones.
• I confirm that I am using English to submit this report.
• I have not modified the feature request template and have filled in all required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

When an Advanced Chat app backed by Knowledge Retrieval is published as an MCP server, tools/call returns only the generated answer text. The retrieved source metadata is persisted by Dify in messages.message_metadata.retriever_resources, but it is not exposed through the MCP response.

This makes downstream MCP clients such as Microsoft Copilot Studio and Teams unable to show citations, document attribution, page/section information, or other provenance to end users. For enterprise RAG use cases, users need to know which manual, policy, or document supports an answer.

For example, the current MCP response is effectively:

{
  "content": [
    {
      "type": "text",
      "text": "Move your hand up or down below the inside rearview mirror to turn the reading light on or off."
    }
  ]
}

However, the corresponding Dify message already contains retrieval metadata such as:

• dataset name
• document name
• segment content and position
• retrieval score
• document metadata (for example page, model, version, or other custom metadata)

The current MCP adapter for chat modes extracts only response["answer"], and structured output contains only { "answer": ... }. As a result, the provenance is lost at the MCP boundary.

It would be helpful if MCP responses for RAG-backed Chat / Advanced Chat apps could expose citations and source attribution, for example:

{
  "answer": "Move your hand up or down below the inside rearview mirror to turn the reading light on or off.",
  "citations": [
    {
      "dataset_name": "vehicle-manuals",
      "document_name": "vehicle-manual.xlsx",
      "title": "Turning the reading light on and off",
      "page": 272,
      "score": 0.70,
      "metadata": {
        "model": "example-model",
        "model_year": "example-year"
      }
    }
  ]
}

Suggested behavior:

1. Include deduplicated retrieval sources in structuredContent.citations for MCP clients that support structured output.
2. Optionally append a human-readable source section to TextContent for clients that only render text.
3. Avoid exposing internal-only identifiers by default, such as tenant IDs or segment internals; expose user-facing document metadata instead.
4. Preserve backward compatibility for apps without retrieval results and for older MCP protocol versions.
5. Support both blocking Chat / Advanced Chat responses and streaming Agent Chat responses where retrieval metadata is available.

2. Additional context or comments

Tested with self-hosted Dify 1.16.1 and an Advanced Chat app published as an MCP server. The knowledge retrieval itself works and retriever_resources are correctly saved in message_metadata; only the MCP response omits them.

Related but separate MCP transaction bug: #39787 and #40013. Those address tools/call failing with a closed SQLAlchemy transaction, not citation propagation.

This feature would improve trust, auditability, and compliance for enterprise MCP integrations, especially Microsoft Copilot Studio / Teams, where the MCP tool response is the only source available to the orchestrator.

3. Can you help us with this feature?

• I am interested in contributing to this feature.

[sanpatricky]

I have a focused implementation with tests ready for this feature. Could a maintainer please assign #40481 to me so I can open a PR in accordance with the contribution guidelines? The implementation preserves backward compatibility, adds structured citations, and includes a text fallback for MCP clients that only render TextContent.

[sanpatricky]

The implementation is ready for review while I wait for assignment:

• Branch: https://github.com/sanpatricky/dify/tree/feat/mcp-citations-40481
• Commit: sanpatricky@5c28910
• Compare: main...sanpatricky:dify:feat/mcp-citations-40481

Implementation summary:

• Extracts metadata.retriever_resources already present in blocking Chat / Advanced Chat responses.
• Collects citations from streaming message_end metadata without consuming the stream twice.
• Adds document-level deduplicated human-readable Sources as a second TextContent block for text-only MCP clients.
• Adds segment-level structuredContent.citations for MCP 2025-06-18 clients.
• Excludes internal dataset/document/segment IDs from the MCP citation payload.
• Preserves the existing response shape when no retrieval sources exist.
• Covers legacy MCP clients with text sources and no structuredContent.

Validation:

• MCP streamable HTTP unit tests: 58 passed.
• make lint: passed.
• make type-check: passed (Pyrefly + Mypy, 1,670 source files).
• End-to-end tools/call: HTTP 200, isError=false, one deduplicated text source, and three structured retrieval-segment citations.

I will open the PR with Fixes #40481 after a maintainer assigns the issue, per the contribution guidelines.