How to reuse work dir for pixi-build projects that have a cached build directory?

[matthewfeickert]

Question

If you have a Python project with compiled dependencies that is using scikit-build-core as the build-system, and you are using scikit-build-core's build-dir to establish a local build cache

diff --git a/pyproject.toml b/pyproject.toml
index f256e95..e1e2705 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -12,3 +12,5 @@ dependencies = ["torch"]
 [tool.scikit-build]
 cmake.version = ">=3.18"
 wheel.packages = []
+# cache build in a local directory
+build-dir = "build/{wheel_tag}"

you will get the expected build caching when using pixi-build, but it also results in the Python package needing to get reinstalled from the cache as a conda package every time the environment is activated.

Is it possible to define a particular build cache for pixi-build so that if my Python project's build-system is using a local build cache pixi-build can be aware of it and detect if there are changes that require rebuilding the conda package?

It was mentioned in prefix-dev/pixi-build-backends#86 that

we already have the option to re-use the work dir.

but I'm not sure how to do that.

Files

(This is from a colleague's private work, so can't just share the repository — sorry.)

pyproject.toml

[build-system]
requires = ["scikit_build_core", "pybind11"]
build-backend = "scikit_build_core.build"

[project]
name = "example_pkg"
version = "0.1.0"
description = "example package"
requires-python = ">=3.10"
dependencies = ["torch"]

[tool.scikit-build]
cmake.version = ">=3.18"
wheel.packages = []
# cache build in a local directory
build-dir = "build/{wheel_tag}"

pixi.toml

[workspace]
channels = ["conda-forge"]
name = "example_pkg"
platforms = ["linux-64", "osx-arm64"]
version = "0.1.0"
preview = ["pixi-build"]

[workspace.build-variants]
python = ["3.10.*", "3.11.*", "3.12.*", "3.13.*", "3.14.*"]

[tasks]

[dependencies]
example_pkg = { path = "." }

[package]
name = "example_pkg"
version = "0.1.0"

[package.build]
backend = { name = "pixi-build-python", version = "0.4.*" }
# c.f. https://pixi.prefix.dev/latest/build/backends/pixi-build-python/#compilers
config = { compilers = ["cxx"] }

[package.build-dependencies]
cmake = ">=4.2.1"

[package.host-dependencies]
python = "*"
uv = "*"
scikit-build-core = ">=0.11.6"
pybind11 = ">=3.0.1"
pytorch = ">=2.9.1"
geant4 = ">=11.3.2"

[package.run-dependencies]
pytorch = ">=2.9.1"
geant4 = ">=11.3.2"

[feature.py310.dependencies]
python = "3.10.*"

[feature.py311.dependencies]
python = "3.11.*"

[feature.py312.dependencies]
python = "3.12.*"

[feature.py313.dependencies]
python = "3.13.*"

[feature.py314.dependencies]
python = "3.14.*"

[feature.testing.dependencies]
pytest = "*"

[feature.testing.tasks.tests]
description = "Run tests with pytest"
cmd = "pytest tests/ -v"
default-environment = "py314"

[environments]
py310 = { features = ["py310", "testing"], solve-group = "py310" }
py311 = { features = ["py311", "testing"], solve-group = "py311" }
py312 = { features = ["py312", "testing"], solve-group = "py312" }
py313 = { features = ["py313", "testing"], solve-group = "py313" }
py314 = { features = ["py314", "testing"], solve-group = "py314" }

Commands

$ pixi clean && rm -rf ./build/*
$ time pixi install -v &> install.txt

real	1m8.530s
user	1m6.254s
sys	0m20.184s

install.txt

$ time pixi install -v &> install_rerun.txt

real	0m6.262s
user	0m3.421s
sys	0m2.339s

install_rerun.txt

$ time pixi run -v python -c 'import example_pkg; print(example_pkg)' &> python_import.txt

real	0m6.253s
user	0m5.146s
sys	0m1.467s

python_import.txt

[matthewfeickert]

@baszalmstra 👋 Moved this question here from the Discord.