The RAG (Retrieval-Augmented Generation) triad of metrics—answer relevance, support relevance, and correctness—is a framework for evaluating systems that combine document retrieval and text generation. These metrics assess how well a system retrieves appropriate information, generates answers aligned with the query, and ensures factual accuracy. Together, they address the core challenges of RAG systems: ensuring the output is useful, grounded in valid sources, and factually reliable. By measuring these three aspects, developers can pinpoint weaknesses in retrieval, generation, or validation steps.
Answer relevance evaluates whether the generated answer directly addresses the user’s query. For example, if a user asks, “What causes solar eclipses?” an answer explaining lunar phases would score low here, even if accurate. This metric ensures the system stays on-topic and avoids tangential or generic responses. Support relevance measures how well the retrieved documents (e.g., database entries or web pages) relate to the query. If the system fetches articles about lunar eclipses for a question about solar eclipses, this metric would flag the mismatch. Finally, correctness checks if the answer is factually consistent with the retrieved sources and external knowledge. An answer stating “solar eclipses occur monthly” would fail here, as it contradicts known astronomical facts. These three metrics work in tandem: even if an answer is correct, poor relevance to the query or supporting documents makes it unhelpful.
For a comprehensive evaluation, all three metrics must be tracked. For instance, a travel assistant RAG system answering “What’s the best time to visit Japan?” might retrieve outdated blog posts (low support relevance), leading to an answer suggesting travel during a restricted season (low correctness). Alternatively, it might retrieve valid sources but generate a vague answer like “Japan has four seasons” (low answer relevance). By analyzing all three metrics, developers can identify whether failures stem from retrieval (e.g., improving document indexing), generation (e.g., fine-tuning the model), or validation (e.g., adding fact-checking layers). This triad provides a balanced view of performance, ensuring systems are accurate, context-aware, and user-focused.
