Amazon Bedrock imposes specific limitations and quotas on model usage, request rates, and payload sizes to ensure service reliability and fair resource allocation. These constraints vary depending on the model provider (e.g., Anthropic, Cohere, Amazon Titan) and the AWS region you’re using. Developers need to be aware of these limits to design applications that scale effectively and avoid service interruptions.
For model usage, Bedrock enforces per-minute request quotas and token-based limits. For example, Anthropic’s Claude models might allow 1,000 requests per minute and 100,000 input tokens per minute by default, while Amazon Titan Text could have different thresholds. These quotas prevent a single user from monopolizing shared resources. Token limits also cap the input and output length for individual requests. Claude models, for instance, may restrict input to 10,000 tokens and output to 4,000 tokens per request. Exceeding these triggers an error, requiring developers to truncate or split data. AWS allows quota increases via support tickets, but approval depends on capacity.
Request rate limits control how frequently you can call Bedrock APIs. Each model has a transactions-per-second (TPS) cap, such as 10 TPS for Cohere Command in certain regions. Burst capacity might temporarily allow higher rates, but sustained overages result in throttling (HTTP 429 errors). For example, a real-time translation app sending 15 requests per second to Titan Text would need to implement retry logic with exponential backoff or distribute traffic across multiple AWS accounts. Rate limits are often tied to the model’s complexity—larger models like Claude 2 typically have stricter TPS caps than smaller ones.
Payload size restrictions apply to both input and output data. Most Bedrock models enforce maximum payload sizes of 8-16 MB per request, including text, images, or embeddings. For instance, Amazon Titan Multimodal Embeddings might reject image inputs larger than 5 MB. Additionally, some models impose context window limits—Claude 3’s 200,000-token context window requires splitting lengthy documents into chunks. Developers must preprocess data (e.g., compressing images, truncating text) and handle errors like ValidationException for oversized payloads. These constraints ensure low-latency responses and prevent network bottlenecks, but they require careful handling in data-intensive workflows like document analysis or batch processing.