{"id":1145803,"date":"2025-08-05T09:00:00","date_gmt":"2025-08-05T16:00:00","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=1145803"},"modified":"2026-01-28T07:45:50","modified_gmt":"2026-01-28T15:45:50","slug":"veritrail-detecting-hallucination-and-tracing-provenance-in-multi-step-ai-workflows","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/veritrail-detecting-hallucination-and-tracing-provenance-in-multi-step-ai-workflows\/","title":{"rendered":"VeriTrail: Detecting hallucination and tracing provenance in multi-step AI workflows"},"content":{"rendered":"\n
\"Alt<\/figure>\n\n\n\n
\n
Watch VeriTrail Explainer<\/a><\/div>\n<\/div>\n\n\n\n

This research was accepted by the 2026 International Conference on Learning Representations (ICLR), a premier conference in artificial intelligence.\u00a0<\/em><\/p>\n\n\n\n

Many applications of language models (LMs) involve generating content based on source material, such as answering questions, summarizing information, and drafting documents. A critical challenge for these applications is that LMs may produce content that is not supported by the source text \u2013 a phenomenon known as \u201cclosed-domain hallucination.\u201d1<\/sup><\/a><\/p>\n\n\n\n

Existing methods for detecting closed-domain hallucination typically compare a given LM output to the source text, implicitly assuming that there is only a single output to evaluate. However, applications of LMs increasingly involve processes with multiple generative steps: LMs generate intermediate outputs that serve as inputs to subsequent steps and culminate in a final output. Many agentic workflows follow this paradigm (e.g., each agent is responsible for a specific document or sub-task, and their outputs are synthesized into a final response).\u202f <\/p>\n\n\n\n

In our paper \u201cVeriTrail: Closed-Domain Hallucination Detection with Traceability<\/a>,\u201d we argue that, given the complexity of processes with multiple generative steps, detecting hallucination in the final output is necessary but not sufficient. We also need traceability<\/strong>, which has two components: <\/p>\n\n\n\n

    \n
  1. Provenance: <\/strong>if the final output is supported by the source text, we should be able to trace its path through the intermediate outputs to the source. <\/li>\n\n\n\n
  2. Error Localization: <\/strong>if the final output is not supported by the source text, we should be able to trace where the error was likely introduced.<\/li>\n<\/ol>\n\n\n\n

    Our paper presents VeriTrail, the first closed-domain hallucination detection method designed to provide traceability for processes with any number of generative steps. We also demonstrate that VeriTrail outperforms baseline methods commonly used for hallucination detection. In this blog post, we provide an overview of VeriTrail\u2019s design and performance.2<\/sup><\/a><\/p>\n\n\n\n

    VeriTrail\u2019s hallucination detection process<\/h2>\n\n\n\n

    A key idea leveraged by VeriTrail is that a wide range of generative processes can be represented as a directed acyclic graph (DAG)<\/strong>. Each node <\/strong>in the DAG represents a piece of text (i.e., source material, an intermediate output, or the final output) and each edge <\/strong>from node A to node B indicates that A was used as an input to produce B. Each node is assigned a unique ID, as well as a stage<\/strong> reflecting its position in the generative process.\u202f <\/p>\n\n\n\n

    An example of a process with multiple generative steps is GraphRAG<\/a>. A DAG representing a GraphRAG run is illustrated in Figure 1, where the boxes and arrows correspond to nodes and edges, respectively.3<\/sup><\/a><\/p>\n\n\n\n

    \"A
    Figure 1: GraphRAG splits the source text into chunks (Stage 1). For each chunk, an LM extracts entities and relationships (the latter are denoted by \u201c\u2b64 \u201c), along with short descriptions (Stage 2). If an entity or a relationship was extracted from multiple chunks, an LM summarizes the descriptions (Stage 3). A knowledge graph is constructed from the final set of entities and relationships, and a community detection algorithm, such as Leiden clustering, groups entities into communities. For each community, an LM generates a \u201ccommunity report\u201d that summarizes the entities and relationships (Stage 4). To answer a user\u2019s question, an LM generates \u201cmap-level answers\u201d based on groups of community reports (Stage 5), then synthesizes them into a final answer (Stage 6).<\/figcaption><\/figure>\n\n\n\n

    VeriTrail takes as input a DAG representing a completed generative process and aims to determine whether the final output is fully supported by the source text. It begins by extracting claims (i.e., self-contained, verifiable statements) from the final output using Claimify<\/a>. VeriTrail verifies claims in the reverse order of the generative process: it starts from the final output and moves toward the source text. Each claim is verified separately. Below, we include two case studies that illustrate how VeriTrail works, using the DAG from Figure 1.\u202f<\/p>\n\n\n\n

    Case study 1: A \u201cFully Supported\u201d claim<\/h3>\n\n\n\n
    \"An
    Figure 2: Left: GraphRAG as a DAG. Right: VeriTrail\u2019s hallucination detection process for a \u201cFully Supported\u201d claim.<\/figcaption><\/figure>\n\n\n\n

    Figure 2 shows an example of a claim that VeriTrail determined was not hallucinated:\u202f<\/p>\n\n\n\n