Building C++ package with pybind11 works with plain cmake, fails with pixi, need help!

[marcuscollins]

I am trying to make it easy to install python bindings for various protein science code that's in C++. I've successfully build the library with CMake on Ubuntu , but when I try to do this with pixi, using pixi-build, it fails, apparently unable to find the libraries, even when I'm telling it exactly where they are in the pyproject.toml file. I'm pretty new to CMake, but I imagine this is an issue where pixi is trying to build everything within its own environment, and not linking to "external" (/usr/lib/*) libraries. Can anyone help me fix/work around this?

My code is all here: https://github.com/diff-use/density-fitness/tree/debugging-pixi-install and the README includes information on the libraries that have to be installed first (all with apt). Full error message below.

[marcuscollins]

│ ╭─ Running build script
 │ │ INFO: activate-binutils_linux-64.sh made the following environmental changes:
 │ │ +ADDR2LINE=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-addr2line
 │ │ +AR=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-ar
 │ │ +AS=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-as
 │ │ +CXXFILT=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-c++filt
 │ │ +ELFEDIT=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-elfedit
 │ │ +GPROF=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-gprof
 │ │ +LD=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-ld
 │ │ +NM=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-nm
 │ │ +OBJCOPY=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-objcopy
 │ │ +OBJDUMP=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-objdump
 │ │ +RANLIB=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-ranlib
 │ │ +READELF=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-readelf
 │ │ +SIZE=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-size
 │ │ +STRINGS=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-strings
 │ │ +STRIP=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-strip
 │ │ INFO: activate-gcc_linux-64.sh made the following environmental changes:
 │ │ +BUILD=x86_64-conda-linux-gnu
 │ │ +CC=x86_64-conda-linux-gnu-cc
 │ │ +CC_FOR_BUILD=x86_64-conda-linux-gnu-cc
 │ │ +CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem $PREFIX/include -fdebu
 │ │ g-prefix-map=$SRC_DIR=/usr/local/src/conda/py_density_fitness-0.1.0 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix
 │ │ +CMAKE_ARGS=-DCMAKE_AR=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-ar -DCMAKE_CXX_COMPILER_AR=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_C_COMPIL
 │ │ ER_AR=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-gcc-ar -DCMAKE_RANLIB=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-ranlib -DCMAKE_CXX_COMPILER_RANLIB=$BUILD_PRE
 │ │ FIX/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_C_COMPILER_RANLIB=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-gcc-ranlib -DCMAKE_LINKER=$BUILD_PREFIX/bin/x8
 │ │ 6_64-conda-linux-gnu-ld -DCMAKE_STRIP=$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-strip -DCMAKE_BUILD_TYPE=Release -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -
 │ │ DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY -DCMAKE_FIND_ROOT_PATH=$PREFIX;$BUILD_PREFIX/x86_64-conda-linux-gnu/sysroo
 │ │ t -DCMAKE_INSTALL_PREFIX=$PREFIX -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_PROGRAM_PATH=$BUILD_PREFIX/bin;$PREFIX/bin
 │ │ +CMAKE_PREFIX_PATH=$PREFIX:$BUILD_PREFIX/x86_64-conda-linux-gnu/sysroot/usr
 │ │ +CONDA_BUILD_CROSS_COMPILATION=
 │ │ +CONDA_BUILD_SYSROOT=$BUILD_PREFIX/x86_64-conda-linux-gnu/sysroot
 │ │ +CONDA_TOOLCHAIN_BUILD=x86_64-conda-linux-gnu
 │ │ +CONDA_TOOLCHAIN_HOST=x86_64-conda-linux-gnu
 │ │ +CPP=x86_64-conda-linux-gnu-cpp
 │ │ +CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem $PREFIX/include
 │ │ +CPP_FOR_BUILD=x86_64-conda-linux-gnu-cpp
 │ │ +DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction
 │ │ -sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/py_density_fitness-0.1.0 -fdebug-prefix-map=$PREFIX=/usr/local/sr
 │ │ c/conda-prefix
 │ │ +DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og -isystem $PREFIX/include
 │ │ +GCC=x86_64-conda-linux-gnu-gcc
 │ │ +GCC_AR=x86_64-conda-linux-gnu-gcc-ar
 │ │ +GCC_NM=x86_64-conda-linux-gnu-gcc-nm
 │ │ +GCC_RANLIB=x86_64-conda-linux-gnu-gcc-ranlib
 │ │ +HOST=x86_64-conda-linux-gnu
 │ │ +LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath
 │ │ ,$PREFIX/lib -Wl,-rpath-link,$PREFIX/lib -L$PREFIX/lib
 │ │ +LDFLAGS_LD=-O2 --sort-common --as-needed -z relro -z now --disable-new-dtags --gc-sections --allow-shlib-undefined -rpath $PREFIX/lib -rpath-link $PREFIX/
 │ │ lib -L$PREFIX/lib
 │ │ +MESON_ARGS=-Dbuildtype=release --prefix=$PREFIX -Dlibdir=lib
 │ │ +build_alias=x86_64-conda-linux-gnu
 │ │ +host_alias=x86_64-conda-linux-gnu
 │ │ -BUILD=x86_64-conda_cos6-linux-gnu
 │ │ -CONDA_BUILD_CROSS_COMPILATION=0
 │ │ INFO: activate-gxx_linux-64.sh made the following environmental changes:
 │ │ +CXX=x86_64-conda-linux-gnu-c++
 │ │ +CXXFLAGS=-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffuncti
 │ │ on-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/py_density_fitness-0.1.0 -fdebug-prefix-map=$PREFIX=/usr/local/
 │ │ src/conda-prefix
 │ │ +CXX_FOR_BUILD=x86_64-conda-linux-gnu-c++
 │ │ +DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -W
 │ │ all -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/py_density_fitne
 │ │ ss-0.1.0 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix
 │ │ +GXX=x86_64-conda-linux-gnu-g++
 │ │ 1.13.2
 │ │ cmake version 3.25.3
 │ │ CMake suite maintained and supported by Kitware (kitware.com/cmake).
 │ │ ~/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/density-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/work/build ~/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/de
 │ │ nsity-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/work
 │ │ -- Git hash not found
 │ │ -- density-fitness: Not using resources since mrc was not found
 │ │ -- Found pybind11: $PREFIX/include (found version "3.0.1")
 │ │ -- Checking for module 'libpcre2-8'
 │ │ --   Package 'libpcre2-8', required by 'virtual:world', not found
 │ │ -- Looking for ccp4/ccp4_general.h - /usr/include/ccp4
 │ │ -- Found CCP4: /usr/lib/libccp4c.so  
 │ │ -- Set CCP4_INCLUDE_DIRS to /usr/include/ccp4
 │ │ -- Looking for clipper/clipper.h - /usr/include
 │ │ -- Found CCP4: /usr/lib/x86_64-linux-gnu/libclipper-core.so  
 │ │ -- Set CCP4_INCLUDE_DIRS to /usr/include/ccp4;/usr/include
 │ │ -- Need rfftw fftw
 │ │ -- FFTW2 libraries - /usr/lib/libfftw.so
 │ │ --                 - /usr/lib/librfftw.so
 │ │ CMake Error at /usr/share/ccp4/ccp4-config.cmake:234 (message):
 │ │ -- FFTW2 header directory - /usr/include
 │ │   Linking with clipper-core failed.
 │ │ -- trying with extra libs
 │ │ Call Stack (most recent call first):
 │ │   .pixi/build/work/py_density_fitness-hLp9NHZTR6U/work/build/_deps/pdb-redo-src/CMakeLists.txt:110 (find_package)
 │ │ -- added threads: _LINKING_WITH_CLIPPER_CORE_AND_THREADS
 │ │ 
 │ │ -- Configuring incomplete, errors occurred!
 │ │ See also "$SRC_DIR/build/CMakeFiles/CMakeOutput.log".
 │ │ See also "$SRC_DIR/build/CMakeFiles/CMakeError.log".
 │ │ × error Script failed with status 1
 │ │ × error 
 │ │ × error Script execution failed.
 │ │ × error 
 │ │ × error   Work directory: /data/users/marcus/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/density-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/work
 │ │ × error   Prefix: /data/users/marcus/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/density-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/host_placehold_pla
 │ │ × error cehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_
 │ │ × error   Build prefix: /data/users/marcus/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/density-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/bld
 │ │ × error 
 │ │ × error To run the script manually, use the following command:
 │ │ × error 
 │ │ × error   cd "/data/users/marcus/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/density-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/work" && ./conda_build
 │ │ × error .sh
 │ │ × error 
 │ │ × error To run commands interactively in the build environment:
 │ │ × error 
 │ │ × error   cd "/data/users/marcus/CLionProjects/tmp/tmp/tmp.U5EjOmokNr/density-fitness/.pixi/build/work/py_density_fitness-hLp9NHZTR6U/work" && source build_
 │ │ × error env.sh
 │ │
 │ ╰─────────────────── (took 1 second)
 │
 ╰─────────────────── (took 1 second)
Error:   × failed to build 'py_density_fitness' from '.'
  ╰─▶   × Script failed to execute