Amazon Bedrock allows developers to choose between streaming responses incrementally or receiving the full output at once, depending on the API method and model used. When you call Bedrock’s InvokeModel API, it returns the complete generated text in a single response after processing the entire input. However, if you use the InvokeModelWithResponseStream API, the model streams the output token-by-token or in chunks as they are generated. This streaming approach is supported by specific foundation models in Bedrock, such as Anthropic’s Claude, and requires explicit configuration in the API request. For example, specifying "stream": true in the request body (for compatible models) or using the dedicated streaming API method triggers this behavior.
The distinction between streaming and non-streaming modes affects how you handle responses. In streaming mode, the API sends events over an HTTP connection as text is generated, allowing applications to process partial results immediately. For instance, in a Python script using the AWS SDK, you might iterate over a response_stream object and append tokens to a buffer in real time. This is useful for interactive applications like chatbots, where displaying text incrementally improves user experience. In contrast, non-streaming mode requires waiting for the entire generation process to complete before receiving the output. This is simpler to implement—for example, using response.get('body').read() in a single call—but introduces latency for longer responses.
Your choice depends on the use case. Streaming is ideal for scenarios requiring low perceived latency, such as real-time chat interfaces or live translation tools. For example, a developer building a customer support bot might stream responses to simulate natural conversation. However, streaming adds complexity, as you must manage partial outputs, handle connection interruptions, and reassemble tokens. Non-streaming is better for batch processing, such as generating summaries for thousands of documents, where immediate feedback isn’t critical. Always check the Bedrock documentation for your chosen model to confirm streaming support, as not all models offer both modes. For example, as of 2023, Amazon Titan Text supports streaming, but some third-party models in Bedrock may not. Use the AWS SDK’s retry logic and error handling in either mode to ensure reliability.