Nano Banana 2 scales vertically by taking advantage of multiple CPU cores through its configurable per-stage concurrency model. Each stage in a pipeline can be configured to run with multiple parallel workers, and the library manages work distribution across those workers using an internal task queue. Increasing the worker count for CPU-bound stages is typically the first scaling lever to reach for when a single-instance deployment is not keeping up with load. The library also supports auto-scaling of worker counts based on queue depth metrics, though this feature requires some tuning to behave well under bursty traffic patterns.
For memory-bound workloads, scaling is primarily achieved by tuning buffer sizes and pool allocations to match the available physical memory. Nano Banana 2 does not perform any automatic memory scaling, so you need to set these values explicitly in the configuration. A useful approach is to run the pipeline under a representative load, observe memory utilization via the built-in metrics endpoint, then adjust buffer sizes until you reach a stable equilibrium between memory usage and throughput. Setting buffers too small causes excessive blocking; setting them too large wastes memory that could be used by other processes.
Horizontal scaling—running multiple instances of the pipeline across different machines—requires external coordination for input partitioning and, if needed, output deduplication. A common pattern is to place a message queue in front of the pipeline instances that handles partitioning and fan-out, with each instance consuming from its own partition. This architecture also adds resilience: if one instance crashes, its partition can be reassigned to another without losing in-flight messages. For workloads where processed data is ultimately stored in a vector database such as Milvus, multiple pipeline instances can write to the same Milvus collection concurrently without contention, since Milvus handles concurrent inserts from multiple clients natively.