Skip to content

linalg/mmm: L3-resident outer tier for the single-thread block walk#2350

Open
czoli1976 wants to merge 2 commits into
sonos:mainfrom
czoli1976:feature/mmm-st-l3-tier
Open

linalg/mmm: L3-resident outer tier for the single-thread block walk#2350
czoli1976 wants to merge 2 commits into
sonos:mainfrom
czoli1976:feature/mmm-st-l3-tier

Conversation

@czoli1976

@czoli1976 czoli1976 commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Stacked on #2349 (the reusable cache module). GitHub can't target a fork branch as the base, so this PR is opened against main and currently shows both commits — the first is #2349's cache module, the second (linalg/mmm: add an L3-resident outer tier…) is the actual content here. Once #2349 merges I'll rebase and the diff will drop to the single L3 commit. Please review only the top commit; merge #2349 first.


Adds a second cache-blocking tier to the single-thread MMM tile walk: the existing L2-resident inner block is wrapped in an outer super-block sized to L3, so a group of inner blocks stays L3-resident as the walk sweeps the grid (instead of re-fetching shared A/B panels from DRAM at large k).

The outer tier engages only when cache::cache_info() reports an L3 larger than L2; otherwise the outer edge spans the whole grid and the walk is byte-for-byte the current single-tier loop. Either way it's pure tile reordering — each tile still computes its own full-k reduction into a disjoint output region — so it stays bit-exact with the naive loop. The nesting is extracted into for_each_blocked_tile and unit-tested (visits every tile exactly once across single-tier / two-tier / degenerate edges; the no-L3 path matches the original order exactly).

Not yet benchmarked — hence draft. Apple Silicon doesn't expose hw.l3cachesize, so the outer tier is a guaranteed no-op on the hardware I can test; it only engages on x86/Linux boxes that report L3 via /sys. It's regression-safe by construction (no-op without L3, bit-exact with it), but the speedup claim needs an A/B on an L3-reporting machine at a large-k shape (grid > L2, super-block ⊂ L3).

Verified on Apple M-series: full tract-linalg suite green (incl. the real blocked-path bit-exactness tests + the new nesting tests), cargo fmt --all clean, no new clippy findings.

Citation for standard multi-level cache-blocked GEMM:

  • Goto & van de Geijn, Anatomy of High-Performance Matrix Multiplication, ACM TOMS 2008 — the MC/KC/NC blocking; this outer tier is the NC (L3) block.
  • BLIS (Van Zee & van de Geijn, ACM TOMS 2015); GotoBLAS/OpenBLAS; ATLAS (Whaley & Dongarra) — empirical block-size auto-tuning.
  • Eigen manage_caching_sizes — derives GEMM block sizes from detected L1/L2/L3 (closest precedent for cache-derived blocking).

Contrast: Frigo et al., Cache-Oblivious Algorithms, FOCS 1999 — the no-cache-sizes alternative this PR deliberately doesn't take.

@czoli1976

Copy link
Copy Markdown
Contributor Author

When #2349 merges, ping me and I'll rebase #2350.

@czoli1976

czoli1976 commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

tested on a Xeon container(*)

image

(*) = Intel Xeon @ 2.80 GHz, 4 cores, AVX2 + AVX-512 (F/DQ/BW/VL/CD) + AVX-512-VNNI, F16C/FMA. Caches: L1d 32 KB/core, L2 1 MB/core, L3 33 MB shared.

@czoli1976

Copy link
Copy Markdown
Contributor Author

To set expectations:

Expect gain on Cortex-A7X (A72…A78 / A710 / A715 — big cores) and any other Big Boy CPU (not on A5x as no L3 Cache over there)

czoli1976 and others added 2 commits June 17, 2026 20:04
The single-thread MMM block-budget probed L2 with detection logic inlined in
frame/mmm/mod.rs, reusable nowhere and limited to macOS/Linux L2. Move it into a
cache module that exposes L1d/L2/L3 through one memoised probe (macOS/iOS via
sysctlbyname, Linux/Android via /sys, Windows via wmic) and have the block
budget read it. The existing L2 budget is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…walk

The single-thread tile walk blocked one level, sizing panel blocks to L2 only;
at large k a grid that exceeds L2 still re-fetches shared A/B panels from DRAM as
it sweeps. Wrap the L2 inner block in an outer super-block sized to L3 (from the
crate::cache probe) so a group of inner blocks stays L3-resident across the
sweep. The outer tier engages only when an L3 larger than L2 is detected;
otherwise the edge is the whole grid and the walk is identical to before. Still
pure tile reordering, so bit-exact with the naive loop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kali kali force-pushed the feature/mmm-st-l3-tier branch from 61d01bc to d30c3de Compare June 17, 2026 18:04
@kali

kali commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Rebased!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants