Building RAM requirements

[pboguslawski]

Building manticoresearch 6.0.4 in Debian 12 (gcc 12.2.0 from standard debian packages) with 4GB of RAM works fine using

cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTING=0 -DUSE_SYSLOG=1 -DWITH_SSL=0 -DWITH_GALERA=0 -DWITH_MYSQL=0 -DWITH_POSTGRESQL=0 -DWITH_STEMMER=0 -DWITH_ICU=0 -DWITH_ICONV=0 -DWITH_EXPAT=0 -DWITH_ODBC=0 ..
cmake --build . --config RelWithDebInfo
cmake --install . --config RelWithDebInfo

Bulding manticoresearch 6.2.12 in the same OS requires 10GB of RAM (OOM is thrown during compilation even with 8GB of RAM).

After changing build mode from RelWithDebInfo to Release, it is possible to compile manticoresearch 6.2.12 in the same OS with 6GB of RAM.

Is 10 GB of RAM requirement for building 6.2.12 in RelWithDebInfo mode possible or a bug?

[sanikolaev]

Manticore builds fine for all OS using the standard github-hosted runner Ubuntu 22.04 with 7G of RAM -

manticoresearch/.github/workflows/build_template.yml

Line 57 in 53daccb

runs-on: ubuntu-22.04

RelWithDebInfo is a default and is not overridden in the CI:
https://github.com/manticoresoftware/manticoresearch/blob/53daccbab0e53c8b580e75a02da8e50238265826/.github/workflows/pack_publish.yml

so perhaps it's specific to your use case or perhaps the github runner has more RAM than announced. Anyway, we never had a task to optimize RAM consumption during compiling Manticore and haven't calculated the requirements.

BTW are you using the special docker image which is used in the CI? (https://manual.manticoresearch.com/dev/Installation/Compiling_from_sources#Building-using-CI-Docker)

[pboguslawski]

are you using the special docker image which is used in the CI?

No, RAM usages as above were noticed in Debian 12 image built from scratch, with standard Debian 12 packages.

[sanikolaev]

Then perhaps it has smth to do with the cmake version or smth. I'd suggest trying to reproduce the same issue using our official docker that we build for preparing packages. It may help to localize the source of the RAM overconsumption. I can also assume that it may also have smth to do with the number of threads used during compilation. I'm not sure what cmake uses as the default value, perhaps it's automatically calculated based on the number of CPU cores, that's perhaps why we don't have any problems with the github shared runners.