How do embeddings support zero-shot learning?

Embeddings support zero-shot learning by enabling models to generalize to unseen tasks or categories through semantic relationships encoded in vector spaces. Embeddings represent data—like words, images, or concepts—as dense vectors that capture their meaning and context. In zero-shot learning, a model leverages these precomputed embeddings to recognize or classify new examples without explicit training on them. This works because embeddings place semantically similar items (e.g., “cat” and “dog”) closer in the vector space, allowing the model to infer relationships between known and unknown classes based on proximity or similarity. For example, a language model trained on embeddings can infer that “kitten” relates to “cat” even if it wasn’t explicitly shown the word “kitten” during training.

A key application is cross-modal embedding alignment, where different data types (e.g., text and images) are mapped to a shared vector space. Models like CLIP (Contrastive Language-Image Pre-training) use this approach: images and their text descriptions are embedded into the same space during training. At inference time, a zero-shot image classifier can compare an input image’s embedding to text embeddings of class labels (e.g., “a photo of a zebra”) to predict the correct class, even if zebras weren’t in the training data. This works because the model understands the semantic connection between the image’s visual features and the text description’s meaning, all within the shared embedding space.

Embeddings also encode hierarchical or relational structures, which helps models generalize. For instance, if a model’s embeddings capture that “mammal” is a broader category containing “dog” and “cat,” it can infer that a new animal like “raccoon” belongs to the same category if its embedding aligns with the “mammal” cluster. Similarly, in multilingual models, embeddings align words across languages, enabling zero-shot translation between language pairs not seen during training. By structuring knowledge in this way, embeddings act as a bridge between known and unknown tasks, allowing models to extrapolate using semantic similarity rather than relying solely on explicit training examples. This approach reduces the need for task-specific data while maintaining robustness.