Can using an inappropriate distance metric for a given embedding lead to poorer results (for example, using Euclidean on embeddings where only the direction matters)?

Yes, using an inappropriate distance metric for a given embedding can significantly degrade results. Distance metrics quantify how vectors relate in a vector space, and the choice depends on how the embedding was designed. For example, embeddings where direction matters (like word embeddings trained with cosine similarity) encode information in the angles between vectors, not their magnitudes. Using Euclidean distance here would misrepresent relationships because it measures straight-line distances, conflating direction and magnitude. This mismatch can lead to incorrect similarity rankings or clustering, undermining tasks like semantic search or recommendation systems.

Consider word embeddings like Word2Vec or GloVe, which are often evaluated using cosine similarity. These embeddings normalize vectors during training to emphasize directional alignment. If you compute Euclidean distance instead, vectors with the same direction but different magnitudes (e.g., [1, 1] and [2, 2]) will appear farther apart, even though they’re semantically identical. In contrast, cosine similarity would correctly score them as identical (similarity=1). Similarly, in recommendation systems, user/item embeddings often rely on direction to represent preferences. Using Euclidean here might group users based on activity levels (magnitudes) rather than shared interests (direction), leading to irrelevant recommendations. These examples highlight how the metric must align with the embedding’s intended interpretation.

To avoid poor results, always validate the distance metric against the embedding’s properties. If the embedding’s magnitude is noisy or irrelevant (e.g., in NLP tasks), cosine similarity is safer. If magnitude matters (e.g., in anomaly detection where outliers have large norms), Euclidean might work. Some embeddings, like those from contrastive learning, are explicitly designed for specific metrics—check the training objective. Tools like scikit-learn let you experiment with metrics (e.g., metric="cosine" in clustering). When in doubt, test multiple metrics using small-scale validation: compare their performance on a subset of your task (e.g., retrieval accuracy) to identify the best fit.