Why might a RAG-generated answer score well on BLEU/ROUGE against a reference answer but still be considered a poor response in practice?

A RAG-generated answer might score well on BLEU/ROUGE metrics but still be a poor response because these metrics focus on surface-level text overlap rather than semantic accuracy, coherence, or practical utility. BLEU measures n-gram overlap between generated and reference texts, while ROUGE emphasizes recall (how much of the reference is covered). However, they don’t assess whether the answer is factually correct, logically structured, or contextually appropriate. For example, a RAG model might generate a response that includes many correct phrases from a reference answer but combines them in a way that misrepresents the core idea. It could also include irrelevant details that technically match the reference but fail to address the user’s intent. This creates a mismatch between metric performance and real-world usefulness.

One key issue is that BLEU/ROUGE ignore factual consistency. Suppose a user asks, “How does HTTPS ensure security?” A RAG answer might mention “encryption,” “SSL/TLS,” and “data integrity” (matching reference keywords) but incorrectly state that HTTPS uses symmetric encryption alone. The metrics would reward the n-gram overlap, but the answer is misleading. Similarly, a response might paraphrase a reference answer but omit critical steps—like explaining SSL handshakes without mentioning certificate verification. Developers relying on such answers could implement insecure systems despite high metric scores. The metrics also fail to penalize hallucinations—plausible-sounding but false claims that coincidentally share terminology with the reference.

Another problem is coherence and relevance. BLEU/ROUGE don’t evaluate whether the answer flows logically or addresses the query’s nuances. For instance, a technical question like “How to optimize SQL queries?” might receive a RAG-generated list of general tips (e.g., “index columns,” “avoid SELECT *”) that matches a reference answer. However, if the response doesn’t explain when indexing is counterproductive or how to analyze query plans, it’s incomplete for a developer seeking actionable guidance. Similarly, a response might overemphasize minor points (e.g., syntax details) while underselling critical best practices (e.g., normalization). In practice, users need answers that prioritize accuracy, depth, and applicability—qualities that BLEU/ROUGE don’t measure, leading to inflated scores for inadequate responses.