How can we ask the model to provide sources or cite the documents it used in its answer, and what are the challenges in evaluating such citations for correctness?

To ask a model to provide sources or cite documents in its answers, developers can use explicit prompts that specify the requirement for citations. For example, a query like, “Explain how neural networks work and cite the research papers you used,” directly instructs the model to include references. Structured prompts, such as “Answer in the format: [Response] Sources: [Document1, Document2],” can also enforce consistency. Additionally, when using retrieval-augmented models (e.g., those that access external databases), developers can programmatically require the model to reference retrieved documents by including instructions like, “Base your answer on the provided documents and list their IDs.” These methods rely on the model’s ability to recognize and follow citation guidelines embedded in the prompt.

A key challenge in evaluating citations is verifying their correctness. For instance, a model might accurately cite a real paper but misrepresent its findings (e.g., claiming “Document A shows X” when the source actually says Y). This requires manual cross-checking against the original material, which is time-intensive. Another issue is relevance: a citation might exist but not directly support the claim. For example, a model might reference a general overview of machine learning when asked for specifics about transformer architectures. Automated checks (e.g., keyword matching) can flag missing citations but struggle to assess contextual relevance. Furthermore, models may “hallucinate” plausible-sounding but fake sources, such as inventing a paper title or attributing a statement to the wrong author. Detecting this requires access to a verified database of sources, which may not always be available.

Technical limitations also complicate evaluation. If the model cites internal documents (e.g., “Document ID:123”), reviewers need access to the exact version of the referenced material to confirm accuracy. Broken links, outdated references, or formatting inconsistencies (e.g., citing a section number that doesn’t exist) add overhead. Scalability is another hurdle: manually validating citations for large outputs is impractical, but automated systems lack the nuance to judge whether a citation adequately supports a claim. For example, a model might correctly cite three sources for a fact, but only one is sufficient, making it hard to automate “sufficiency” checks. Developers must balance rigorous validation with practical constraints, often relying on sampling or hybrid human-AI workflows to audit citations efficiently.