TypeError: a bytes-like object is required, not 'StreamingChecksumBody'

[crabhi]

Describe the bug

When the server returns an unexpected response, but still correct XML and HTTP headers, boto3 fails to parse it.

I encountered it using Backblaze, but I think this is a bug in boto3. Backblaze returning a bit different response just uncovered a code path that's not usually taken.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Boto3 raises a descriptive exception reflecting the server response.

In version 1.35 and below, the reproduction code raises the correct ClientError.

$ pip list | grep boto
boto3           1.35.99
botocore        1.35.99
$ python3 repro_checksum_bug.py
127.0.0.1 - - [14/Apr/2026 10:52:12] "GET /test-bucket/test-key HTTP/1.1" 400 -
Got different error: ClientError: An error occurred (InternalError) when calling the GetObject operation: Some error

In version 1.36 and later, including 1.42.89, the code raises a TypeError.

Current Behavior

$ python3 repro_checksum_bug.py
127.0.0.1 - - [14/Apr/2026 10:53:34] "GET /test-bucket/test-key HTTP/1.1" 400 -
Bug reproduced!
  TypeError: a bytes-like object is required, not 'StreamingChecksumBody'

This is the original stack trace from our app:

TypeError: a bytes-like object is required, not 'StreamingChecksumBody'

[...]

    self.client.download_fileobj(self.bucket, key_prefix + key, f)
  File "botocore/context.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "boto3/s3/inject.py", line 859, in download_fileobj
    return future.result()
  File "s3transfer/futures.py", line 111, in result
    return self._coordinator.result()
  File "s3transfer/futures.py", line 287, in result
    raise self._exception
  File "s3transfer/tasks.py", line 142, in __call__
    return self._execute_main(kwargs)
  File "s3transfer/tasks.py", line 165, in _execute_main
    return_value = self._main(**kwargs)
  File "s3transfer/download.py", line 582, in _main
    response = client.get_object(
  File "botocore/client.py", line 602, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "botocore/context.py", line 123, in wrapper
    return func(*args, **kwargs)
  File "botocore/client.py", line 1060, in _make_api_call
    http, parsed_response = self._make_request(
  File "botocore/client.py", line 1084, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "botocore/endpoint.py", line 119, in make_request
    return self._send_request(request_dict, operation_model)
  File "botocore/endpoint.py", line 197, in _send_request
    success_response, exception = self._get_response(
  File "botocore/endpoint.py", line 239, in _get_response
    success_response, exception = self._do_get_response(
  File "botocore/endpoint.py", line 313, in _do_get_response
    parsed_response = parser.parse(
  File "botocore/parsers.py", line 265, in parse
    parsed = self._do_error_parse(response, shape)
  File "botocore/parsers.py", line 1426, in _do_error_parse
    return self._parse_error_from_body(response)
  File "botocore/parsers.py", line 1450, in _parse_error_from_body
    root = self._parse_xml_string_to_dom(xml_contents)
  File "botocore/parsers.py", line 542, in _parse_xml_string_to_dom
    parser.feed(xml_string)

Reproduction Steps

import base64
import threading
import zlib
from http.server import BaseHTTPRequestHandler, HTTPServer

import boto3
from botocore.config import Config

ERROR_BODY = b"""<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>InternalError</Code>
  <Message>Some error</Message>
  <RequestId>fake-request-id</RequestId>
</Error>"""


class FakeS3Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        crc = zlib.crc32(ERROR_BODY).to_bytes(4, "big")
        checksum = base64.b64encode(crc).decode()

        # Use 400 (not 500) so the parser goes through _do_error_parse →
        # _parse_error_from_body → parser.feed() and hits the TypeError,
        # matching the production stack trace.  Status >= 500 takes a
        # different path (_is_generic_error_response) that hits an
        # AttributeError on .strip() instead.
        self.send_response(400)
        self.send_header("Content-Type", "application/xml")
        self.send_header("Content-Length", str(len(ERROR_BODY)))
        self.send_header("x-amz-checksum-crc32", checksum)
        self.send_header("x-amz-request-id", "fake-request-id")
        self.end_headers()
        self.wfile.write(ERROR_BODY)

def main():
    server = HTTPServer(("127.0.0.1", 0), FakeS3Handler)
    port = server.server_address[1]
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()

    client = boto3.client(
        "s3",
        endpoint_url=f"http://127.0.0.1:{port}",
        aws_access_key_id="fake",
        aws_secret_access_key="fake",
        region_name="us-east-1",
        config=Config(
            retries={"max_attempts": 1},
        ),
    )

    try:
        client.get_object(Bucket="test-bucket", Key="test-key")
    except TypeError as e:
        print(f"Bug reproduced!\n  {type(e).__name__}: {e}")
    except Exception as e:
        print(f"Got different error: {type(e).__name__}: {e}")
    else:
        print("No error — bug not reproduced")
    finally:
        server.shutdown()


if __name__ == "__main__":
    main()

Possible Solution

I assume this is caused by an interplay between

        response_dict = convert_to_response_dict(
            http_response, operation_model
        )
        handle_checksum_body(
            http_response,
            response_dict,
            context,
            operation_model,
        )

in Endpoint._do_get_response. handle_checksum_body overwrites what convert_to_response_dict has set to body for HTTP status >= 300. The error response parser then chokes on unexpected object.

I'm not sure about the fix but it should be probably be either to make sure parser can work with streaming body or to keep the type of body invariant.

Additional Information/Context

Probably related to #4754

SDK version used

1.42.89

Environment details (OS name and version, etc.)

N/A (both Debian, MacOS)

[adev-code]

Hello @crabhi, thank you for reaching out! Repro'ing the issue with boto3 and doing a request to S3,

• When S3 returns a successful response or 200, the checksum is returned in the response headers (e.g. x-amz-checksum-crc32: 123ABCxyz==)
• When 4XX or 5XX occurs from S3, the checksum is not included.

As per S3 documentations, GetObject API, Error responses and Checking object integrity for data uploads in Amazon S3, S3 only gives the checksum when action is successful or during downloads.

Looking at your repro code, it looks like it is giving back checksum header on a 400. Could you repro this issue with a real S3 request and using boto3 only, and please let me know if there's an occasion where S3 gives checksum header on a 400?

[crabhi]

Hi, I'm not sure.

The overwriting of "body" of the response with different object types seems to be inherently error prone. And raising a TypeError from an internal object signalizes mishandling data somewhere during processing. But it's very much possible that this error can't be triggered with the current S3 API behavior.