Use open-source embeddings (e.g., BGE, Voyage, Nomic) for cost control; ensure embedding and Scout are from the same training foundation for alignment.
Your embedding model converts documents and queries to vectors that Milvus indexes and retrieves. Misalignment between embeddings and generation models causes semantic drift: if you embed with a model trained on generic web text but generate with Scout (trained on code, math, reasoning), retrieved chunks may not match what Scout "understands". BGE (BAAI General Embeddings) is popular for Milvus + open-source LLM stacks because it’s freely available, fast, and aligns well with Llama models. Voyage and Nomic are also strong. For proprietary embeddings, OpenAI’s text-embedding-3 works but introduces API dependency and cost per query.
Optimization: test embedding dimension vs. accuracy trade-off. BGE-large (1024 dims) is slower to index but retrieves with higher precision than BGE-base (768 dims). Milvus supports GPU-accelerated indexing, so dimension choice affects query latency more than throughput. With Scout’s 10M context window, retrieval quality matters—better embeddings mean fewer false positives that Scout must filter. Run offline benchmarks: embed your actual domain docs, create a test query set, measure recall with different embeddings, then choose the best accuracy/speed trade-off for your infrastructure.
Related Resources
- RAG with LlamaIndex — embedding integration examples
- Enhance RAG Performance — embedding selection strategies
- Milvus Blog — embedding benchmarks and case studies