clip-vit-base-patch32 embeds images and text together by processing them through two separate neural networks that are trained to produce comparable vectors. Images are split into fixed-size patches (32×32 pixels), linearly projected, and passed through a Vision Transformer. Text is tokenized and processed through a transformer-based text encoder. Despite using different encoders, both outputs are mapped into the same vector space and normalized to comparable scales.
The training process is contrastive. Each batch contains matched image–text pairs, and the model computes similarity scores across all combinations in the batch. The objective function pushes correct pairs closer together while pushing incorrect pairs further apart. Over time, this forces semantically related images and text to occupy nearby positions in the vector space. As a result, an image of a dog and the text “a photo of a dog” will have a high cosine similarity, while unrelated pairs will not.
In implementation, developers typically do not need to interact with this training logic. They load the pretrained clip-vit-base-patch32 model and call the image or text encoder APIs. The resulting vectors can be stored and queried efficiently. When these vectors are indexed in a system like Milvus or Zilliz Cloud, developers can run fast similarity searches across modalities. This unified embedding approach is what enables text-to-image and image-to-text retrieval without additional alignment steps.
For more information, click here：https://zilliz.com/ai-models/text-embedding-3-large