AI Quick Reference

Looking for fast answers or a quick refresher on AI-related topics? The AI Quick Reference has everything you need—straightforward explanations, practical solutions, and insights on the latest trends like LLMs, vector databases, RAG, and more to supercharge your AI projects!

• How do you prevent an LLM from drifting off-topic in a multi-step retrieval scenario (ensuring each step’s query remains relevant to the original question), and how would that be evaluated?
• How might we modify the RAG pipeline to reduce the incidence of hallucinations (for instance, retrieving more relevant information, or adding instructions in the prompt)?
• What strategies exist to give partial responses or stream the answer as it's being generated to mask backend latency in a RAG system?
• What prompt instructions can be given to reduce the chance of the LLM hallucinating, by explicitly telling it to stick to the provided information?
• What strategies could be used to scale the vector store component for a RAG system dealing with a very large knowledge base or high query volume (sharding, indexing optimizations, etc.)?
• How can we simulate a realistic scenario when measuring RAG latency (for example, including the time to fetch documents, model loading time, etc., not just the core algorithmic time)?
• In an evaluation setup, how would you simulate worst-case scenarios for the vector store (like cache misses, very large index sizes, complex filters) to ensure the RAG system is robust?
• How can we test a RAG system for consistency across different phrasings of the same question or slight variations, to ensure the answer quality remains high?
• How can we test whether a RAG system properly handles queries requiring multiple pieces of evidence? (Consider having test queries where leaving out one retrieved piece would make the answer incorrect.)
• In what situations would training a custom embedding model be worthwhile for RAG, and how would you go about evaluating its improvements over pre-trained embeddings?
• What are the trade-offs of using a cloud-based vector store service in a RAG system evaluation (in terms of latency variance, network costs, etc.) versus a local in-memory store?
• How can using multiple embedding models improve RAG retrieval (for instance, combining dense and sparse embeddings), and what complexity does this add to the system?
• How does using only a dense vector retriever compare to using a hybrid retriever (dense + lexical) in terms of coverage of information and system complexity?
• How can the use of smaller or distilled language models in RAG help with latency, and what is the impact on answer quality to consider?
• What happens if the retrieval strategy returns contradictory information from different sources? How should the LLM handle it, and how do we evaluate whether it handled it correctly?
• When comparing two RAG systems or configurations, what qualitative aspects of their answers would you examine, beyond just whether the answer is correct?
• For a given compute budget, how would you reason about investing in a larger, more powerful LLM versus investing in a more sophisticated retrieval system? What evaluation results would inform this decision?
• When evaluating different RAG architectures, how do differences in latency influence the practicality of each (for example, one might be more accurate but too slow for real-time use)?
• When evaluating a RAG system’s overall performance, how would you combine metrics for retrieval and metrics for generation? (Would you present them separately, or is there a way to aggregate them?)
• In comparing two vector stores or ANN algorithms for use in RAG, what performance and accuracy metrics should be part of the evaluation to make an informed choice?
• What are the potential failure modes when the integration between retrieval and generation is not well-tuned (like the model ignoring retrieval, or mis-associating which document contains the answer)?
• In what ways might prompt engineering differ for RAG when using a smaller or less capable LLM versus a very large LLM? (Think about explicit instructions and structure needed.)
• How do we measure the effect of vector store speed on the overall throughput of a RAG system (for example, could a slow retriever limit how many questions per second the whole pipeline can handle even if the LLM is fast)?
• What factors should be considered when selecting an embedding model for a RAG pipeline (such as the model’s domain training data, embedding dimensionality, and semantic accuracy)?
• What strategies can be used to update or improve embeddings over time as new data becomes available, and how would that affect ongoing RAG evaluations?
• How can prompt engineering help mitigate hallucinations? (E.g., telling the LLM “if the information is not in the provided text, say you don’t know.”)
• How can multi-hop retrieval potentially increase grounding quality? (E.g., by fetching intermediate facts, can it reduce the chance the model makes something up?)
• What are some failure modes of grounding (like contradictory documents retrieved, or no relevant document retrieved) and how do these manifest in the final answer?
• What does “answer relevancy” mean in the context of RAG evaluation, and how can it be measured? (Consider metrics or evaluations that check if the answer stays on topic and uses the retrieved info.)
• In evaluating answer quality, how can human evaluation complement automated metrics for RAG (e.g., judges rating clarity, correctness, and usefulness of answers)?
• What impact does an incoherent or disorganized retrieved context have on the coherence of the generated answer, and how might a model be guided to reorganize information?
• How might the decoding parameters of the LLM (temperature, top-k, etc.) affect the consistency and quality of the answers in a RAG system?
• How can we detect if a RAG system’s answer, while factually correct, might be incomplete or not sufficiently detailed? (Does it leave out relevant info that was in the sources?)
• In what ways can an answer be considered high-quality in RAG aside from factual correctness? (Think of readability, conciseness, directness, and user satisfaction.)
• How does the complexity of queries (or the need for multiple retrieval rounds) affect the system’s latency, and how can a system decide to trade off complexity for speed?
• How can the prompt be designed to handle contradictory information in retrieved documents (for example, guiding the model on how to reconcile conflicts)?
• What techniques can be applied if the retrieved text is too large to fit in the prompt (such as summarization or selecting key sentences), and how do we evaluate the impact of those on answer accuracy?
• How is a metric like BLEU calculated for an answer, and would a higher BLEU score correlate with a more factually correct or just a more lexically similar answer?
• How does one measure the “faithfulness” of an answer to the provided documents? Are there automated metrics (like those in RAGAS or other tools) to do this?
• Why might human evaluation be necessary for RAG outputs even if we have automated metrics, and what criteria would human evaluators assess (e.g., correctness, justification, fluency)?
• What are the two main ways to integrate retrieval with an LLM (prompting a frozen model with external info versus fine-tuning the model on a corpus), and what are the benefits of each approach?
• What role do frameworks like LangChain or HuggingFace’s RAG implementation play in simplifying the integration of retrieval and generation components?
• How can we evaluate whether the vector database or search index is the bottleneck in a RAG pipeline? (E.g., measuring query latency of the vector search separately from generation time.)
• When would a single-step retrieval strategy fail where a multi-step strategy would succeed, and how can those scenarios be detected and used as benchmarks?
• How might user expectations differ for multi-hop questions (like expecting more detailed answers) and how should evaluation metrics reflect satisfaction for these complex queries?
• In what scenario might it be better to rely on the LLM’s parametric knowledge rather than retrieving from an external source (e.g., very simple common knowledge questions), and how to detect those?
• How does quantum computing impact industries like cryptography, finance, and healthcare?
• What are qubits, and how do they differ from classical bits?
• What is quantum supremacy, and has it been achieved yet?
• What is a quantum Fourier transform, and how is it used in quantum algorithms?
• What is a quantum algorithm, and how does it work?
• What is a quantum annealer, and how does it differ from a universal quantum computer?
• How does a quantum computer use interference to amplify the correct solution?
• What is a quantum oracle, and how is it used in algorithms like Grover’s search?
• What is a quantum register, and how does it store quantum information?
• What is the difference between a quantum simulator and a quantum computer?
• What is the concept of a quantum wavefunction, and how is it used in quantum computing?
• What is the role of classical computation in hybrid quantum systems?
• What are the limitations of current quantum computing hardware?
• What is Grover's algorithm, and what is its purpose?
• How do you measure the performance of quantum algorithms?
• What is a quantum key distribution (QKD), and how does it work?
• What are quantum algorithms for optimization, and how do they work?
• How do quantum algorithms handle random walks?
• How does quantum annealing work in solving optimization problems?
• What are quantum circuits, and how do they work?
• What is the significance of quantum coherence in building a reliable quantum computer?
• What is the significance of quantum coherence time?
• What are the different models of quantum computation (e.g., gate model, adiabatic model)?
• How do quantum computers achieve parallelism in computation?
• How do quantum computers address problems related to big data analytics?
• How can quantum computers enhance AI training processes?
• How do quantum computers implement secure multi-party computation?
• How do quantum computers handle data encryption and decryption?
• How do quantum computers handle problems like searching and optimization?
• How do quantum computers affect the development of artificial intelligence?
• How do quantum computers perform matrix multiplication?
• How do quantum computers simulate molecular systems for drug discovery?
• How do quantum computers solve linear systems of equations?
• How do quantum computers utilize the concept of entanglement to speed up computations?
• How do quantum computing techniques enable faster solution generation in combinatorial optimization?
• How does quantum computing help solve optimization problems faster than classical systems?
• How is quantum computing applied in machine learning?
• What are the practical challenges of quantum computing in real-world applications?
• How does quantum computing handle quantum state manipulation?
• What are the applications of quantum computing in cryptography and cybersecurity?
• How does quantum computing interact with classical machine learning methods?
• What is quantum computing, and how does it differ from classical computing?
• What is quantum cryptography, and how does it improve security?
• How does quantum cryptography provide unbreakable encryption?
• How does quantum entanglement enable quantum communication?
• What is quantum error correction, and why is it important for quantum computing?
• What are the methods used for quantum error correction, and how do they work?
• What is the role of quantum error correction codes like the surface code?
• How do quantum error correction schemes like the Shor code work?
• What is the difference between quantum gates and classical logic gates?
• What are the different types of quantum gates, and how do they manipulate qubits?
• What are the basic quantum gates (Hadamard, Pauli, etc.)?
• What are quantum gates like X, Y, Z, and how do they affect quantum states?