Gemma 4 runs on NVIDIA Jetson Orin Nano for edge devices up to Blackwell GPUs for data centers, plus AMD ROCm and Google TPUs.
Gemma 4’s on-device optimization makes it unique among large multimodal models. The smaller variants (E2B, E4B) can run on NVIDIA Jetson Orin Nano, enabling deployment on edge devices for local inference without cloud dependencies. For more demanding workloads, the 26B and 31B variants scale across professional and datacenter GPUs.
Gemma 4 supports multiple accelerator platforms beyond NVIDIA: AMD ROCm-compatible GPUs and Google TPUs. This hardware flexibility means you can choose infrastructure based on your cost, availability, and geographic requirements rather than being locked into a single vendor ecosystem.
When building vector search applications with Milvus and Gemma 4, consider your deployment target. For on-premise systems, Jetson Orin Nano runs lightweight inference pipelines that feed embeddings into Milvus. For larger-scale operations, Blackwell GPUs or TPUs generate embeddings at high throughput while Milvus handles indexing and retrieval—both running on the same hardware or separately depending on your architecture.
Related Resources