Skip to content

Add m4sme_p and m4sme_e sub-configurations#921

Open
Luislo1 wants to merge 28 commits into
flame:masterfrom
Luislo1:master
Open

Add m4sme_p and m4sme_e sub-configurations#921
Luislo1 wants to merge 28 commits into
flame:masterfrom
Luislo1:master

Conversation

@Luislo1

@Luislo1 Luislo1 commented Mar 16, 2026

Copy link
Copy Markdown

This PR adds two new arm64 sub-configurations to BLIS, called m4sme_p and m4sme_e. They are built for the ARM instruction set architecture and are optimized for the Apple Silicon M4 processor, utilizing the Scalable Matrix Extension 2(SME2). The m4sme_p sub-configuration is configured for execution in performance cores and m4sme_e in efficiency cores. Included in the armsme kernels folder are implementations for packing (1m) and the level 3 gemm kernel programmed with intrinsics.

The m4sme_p sub-configuration is chosen through a hardware detection heuristic if the machine supports the SME2 feature. The m4sme_e sub-configuration must be selected manually. The sub-configurations are temporarily blacklisted for all compilers except Clang 17 or later on Darwin.

Regarding the implementation details, the kernel lengths are SVL (Streaming Vector Length) agnostic. We have included 3 different gemm kernels for single precision and 4 for double precision based on the possibilities afforded by the ZA storage. Please note that despite the kernels themselves being agnostic, the bli_cntx_init file is specifically configured for the M4 silicon; Apple silicon currently only supports an SVL of 512b (64B) with an SME 2D array size of 4KiB.

Developers for the m4sme implementation:
@Luislo1
@figual
@luismacostero

We welcome your thoughts and look forward to your feedback on this implementation.

Luislo1 and others added 11 commits March 16, 2026 12:09
@Luislo1
Create m4sme_p configuration and first kernels
6b49e47 
- The configuration is optimal for the Apple M4 chip's performance cores
with a SVL of 512 bits.
- The sgemm kernel's size is 2SVLx2SVL and the dgemm's is 4SVLx2SVL
@Luislo1
Add packing routines for sgemm and dgemm kernels
c8a29f8 
- Use the ZA tiles' reading capabilities for more efficient packing
@Luislo1
Add SVLx4SVL sgemm and SVLx8SVL dgemm kernels
294afc6 
- Include packing routines for both kernels
- Disable new kernels using an #if 0 block in the m4sme_p subconfig
@Luislo1
Add 4SVLxSVL sgemm and 8SVLxSVL dgemm kernels
0cfed38 
- Reuse the SVLx4SVL and SVLx8SVL packing routines
@Luislo1
Add 2SVLx4SVL dgemm kernel
e839c2e 
- Reuse the 4SVLx2SVL packing routine
@Luislo1
Add m4sme_e configuration optimized for E-cores
491f1a3 
- Adjust block size to optimal value for the smaller SME Engine shared
by the efficiency core cluster of the Apple M4 chip.
@Luislo1
Fix sysctl include
18e8a87
@Luislo1
Fix register size values
562a2b8
@devinamatthews
Merge branch 'master' into m4sme
34f2fe3
@devinamatthews
Add CI tests for m4sme.
e8b5ccc 
Details:
- Add m4sme_p and m4sme_e tests in CI.
- Streamline aarch64 testing with QEMU to use single arm64 build.
- Add clang-based QEMU tests.
- Update compiler toolchains: gcc14 and llvm17 needed for SME2.
@devinamatthews
Update CREDITS.
e10a5b8
@devinamatthews

devinamatthews commented Jun 27, 2026

Copy link
Copy Markdown
Member

I made some tweaks to the compiler support and added tests in CI. It may take a couple passes to get CI working since I changed how aarch64 w/ qemu is done.

@devinamatthews

devinamatthews commented Jun 27, 2026

Copy link
Copy Markdown
Member

@Luislo1 @figual @luismacostero this is almost ready to merge I just want to cross all the i's.

  • Can you take a look at the testsuite failure in build-6? This is on actual M4 Pro hardware and it looks like it is crashing in the microkernel tests.
  • I'm not sure why it says the flag -mno-unaligned-access is unknown in build-8. Maybe it is a flag ordering thing? If you can try gcc please take a look.
  • I'm still working on getting clang cross compilation working (build-9), but if for some reason it just won't work there is still build-6.

@jeffhammond

jeffhammond commented Jun 28, 2026

Copy link
Copy Markdown
Member

Given Apple doesn't support core affinity, does it make sense to have separate configs for P/E-cores? Should we not have a config for M4 SME and then figure out the P/E dispatch internally?

@jeffhammond

jeffhammond commented Jun 28, 2026

Copy link
Copy Markdown
Member

I tried this on M5 and it hit SIGILL. I am skeptical that M5 is backwards-incompatible with M4.

bash: line 1: 36586 Illegal instruction: 4  ./obj/m4sme_p/blastest/cblat3.x < ./blastest/input/cblat3.in
make: *** [run-cblat3] Error 132
make: *** Waiting for unfinished jobs....
bash: line 1: 36608 Illegal instruction: 4  ./obj/m4sme_p/blastest/dblat3.x < ./blastest/input/dblat3.in
make: *** [run-dblat3] Error 132
bash: line 1: 36651 Illegal instruction: 4  ./obj/m4sme_p/blastest/sblat3.x < ./blastest/input/sblat3.in
make: *** [run-sblat3] Error 132
bash: line 1: 36220 Illegal instruction: 4  ./test_libblis.x -g ./testsuite/input.general -o ./testsuite/input.operations > output.testsuite
make: *** [testsuite-run] Error 132

@devinamatthews

devinamatthews commented Jun 28, 2026

Copy link
Copy Markdown
Member

Same on CircleCI. Could be due to the build flag changes I made, since it was complaining about SVE not being enabled, but then SME implementations don't actually have to support SVE in non-streaming mode...

@devinamatthews

devinamatthews commented Jun 28, 2026

Copy link
Copy Markdown
Member

Given Apple doesn't support core affinity, does it make sense to have separate configs for P/E-cores? Should we not have a config for M4 SME and then figure out the P/E dispatch internally?

We need better support for run-time selection of different kernels (within the same sub-configuration) anyways.

@Luislo1

Luislo1 commented Jun 29, 2026

Copy link
Copy Markdown
Author

@Luislo1 @figual @luismacostero this is almost ready to merge I just want to cross all the i's.

* Can you take a look at the testsuite failure in build-6? This is on actual M4 Pro hardware and it looks like it is crashing in the microkernel tests.

Fixed! It was a compiler flags problem (at least for build-6). (we are checking build-8 now)

* I'm not sure why it says the flag `-mno-unaligned-access` is unknown in build-8. Maybe it is a flag ordering thing? If you can try gcc please take a look.

Sure! We'll try during this week

* I'm still working on getting clang cross compilation working (build-9), but if for some reason it just won't work there is still build-6.

@jeffhammond can you try the latest commit on an M5 processor? We don't have one available right now (working on it).

@jeffhammond

jeffhammond commented Jun 30, 2026

Copy link
Copy Markdown
Member

check-blastest.sh: All BLAS tests passed! with the latest commit. Thanks!

@jeffhammond

jeffhammond commented Jun 30, 2026

Copy link
Copy Markdown
Member

I used AI to do performance analysis versus Apple's BLAS on the Mac M5 Pro CPU (4 S/P-core, 6 E-core). Everything looks great.

  BLIS vs Apple Accelerate — benchmark results
  
  Setup
  - Machine: Apple M5, 10 cores (4 P), macOS 26.5.1
  - BLIS: config m4sme_p, single-threaded, SME kernels (lib/m4sme_p/libblis.a)
  - Accelerate: the test/ drivers compiled on their non-BLIS code path (sgemm_/dgemm_…), linked with -framework Accelerate (the framework's BLAS,
  AMX/SME-backed)
  - Precision: double (FP64) — the drivers hardcode BLIS_DOUBLE
  - Sizes: square 200→2000, best of 3 repeats; table shows peak GFLOPS
  
  I skipped configure/the Makefile vendor machinery as you asked — just compiled each test_<op>.c twice (once -DBLIS, once -DBLAS="accelerate")
  against the already-built libblis.a.
  
  Peak GFLOPS (B/A = BLIS ÷ Accelerate; >1 means BLIS faster)

  ┌─────────┬────────────┬────────────┬──────┐
  │   op    │ BLIS (SME) │ Accelerate │ B/A  │
  ├─────────┼────────────┼────────────┼──────┤
  │ Level-3 │            │            │      │
  ├─────────┼────────────┼────────────┼──────┤
  │ gemm    │      466.4 │      468.9 │ 0.99 │
  ├─────────┼────────────┼────────────┼──────┤
  │ hemm    │      468.2 │      462.6 │ 1.01 │
  ├─────────┼────────────┼────────────┼──────┤
  │ herk    │      426.4 │      428.1 │ 1.00 │
  ├─────────┼────────────┼────────────┼──────┤
  │ her2k   │      422.1 │      366.5 │ 1.15 │
  ├─────────┼────────────┼────────────┼──────┤
  │ trmm    │      395.2 │      420.2 │ 0.94 │
  ├─────────┼────────────┼────────────┼──────┤
  │ trsm    │      153.4 │      157.2 │ 0.98 │
  ├─────────┼────────────┼────────────┼──────┤
  │ Level-2 │            │            │      │
  ├─────────┼────────────┼────────────┼──────┤
  │ gemv    │        8.2 │        8.2 │ 1.00 │
  ├─────────┼────────────┼────────────┼──────┤
  │ ger     │        3.6 │        3.6 │ 0.99 │
  ├─────────┼────────────┼────────────┼──────┤
  │ hemv    │        5.9 │        5.9 │ 0.99 │
  ├─────────┼────────────┼────────────┼──────┤
  │ her     │        2.1 │        2.2 │ 0.96 │
  ├─────────┼────────────┼────────────┼──────┤
  │ her2    │        3.2 │        2.8 │ 1.15 │
  ├─────────┼────────────┼────────────┼──────┤
  │ trmv    │        8.2 │        8.2 │ 1.00 │
  ├─────────┼────────────┼────────────┼──────┤
  │ trsv    │        8.0 │        7.9 │ 1.01 │
  ├─────────┼────────────┼────────────┼──────┤
  │ Level-1 │            │            │      │
  ├─────────┼────────────┼────────────┼──────┤
  │ dotv    │        3.4 │        3.7 │ 0.90 │
  ├─────────┼────────────┼────────────┼──────┤
  │ axpyv   │        2.1 │        2.2 │ 0.95 │
  └─────────┴────────────┴────────────┴──────┘

  gemm scaling (GFLOPS by size) — essentially overlapping curves:
  
  size    BLIS   Accel
   200   208.7   210.1
   400   369.2   374.7
   800   448.5   446.2
  1200   432.9   435.4
  1600   459.5   460.2
  2000   466.4   468.9
  
  Takeaways
  - It's basically a dead heat. Across the compute-bound Level-3 kernels BLIS and Accelerate are within ~1% of each other (gemm 466 vs 469 GFLOPS
  FP64). BLIS wins her2k/her2 notably (+15%); Accelerate edges trmm (+6%).
  - Both saturate the same shared matrix coprocessor, so BLIS being single-threaded isn't a handicap here — extra threads don't help AMX/SME-bound
  GEMM much. Level-1/2 are memory-bound and both sit near the bandwidth ceiling.
  
  Complex GEMM: BLIS (SME) vs Apple Accelerate
  
  Same setup as before (Apple M5, FP-complex, peak of best-of-3, sizes 200→2000). GFLOPS include the standard ×4 complex-flop factor, so they're
  directly comparable to the earlier real-GEMM numbers.
  
  Peak GFLOPS
  
  ┌───────┬────────────────┬────────────┬────────────┬──────┐
  │  op   │   precision    │ BLIS (SME) │ Accelerate │ B/A  │
  ├───────┼────────────────┼────────────┼────────────┼──────┤
  │ cgemm │ single complex │     1094.8 │     1099.8 │ 1.00 │
  ├───────┼────────────────┼────────────┼────────────┼──────┤
  │ zgemm │ double complex │      384.1 │      386.8 │ 0.99 │
  └───────┴────────────────┴────────────┴────────────┴──────┘
  
  Scaling (effective GFLOPS):
  
  cgemm   size    BLIS   Accel        zgemm   size    BLIS   Accel
           600   670.7   721.9                 600   306.8   314.3
          1000   868.7   903.0                1000   333.1   335.5
          1400  1035.9  1009.8                1400   343.7   357.2
          2000  1084.2  1094.0                2000   384.1   386.8
  
  Takeaways 
  - Another dead heat — within ~1% at peak for both cgemm and zgemm. Accelerate is a hair ahead at mid sizes (cgemm@600: +8%), BLIS pulls even by
  large sizes.
  - The cgemm/zgemm ≈ 2.8× ratio (1095 vs 385) is the expected FP32-vs-FP64 throughput gap on the matrix unit.
  - cgemm hits ~1.1 TFLOPS effective — both libraries are extracting essentially the same peak from the M5's matrix coprocessor.

@devinamatthews

devinamatthews commented Jul 7, 2026

Copy link
Copy Markdown
Member

@Luislo1 given that the config does work on the actual HW, I would be OK with semi-temporarily disabling the cross-compiler tests. Please let me know if you would like to do that or soldier on and try to fix them.

@Luislo1

Luislo1 commented Jul 8, 2026

Copy link
Copy Markdown
Author

@devinamatthews we've been trying to compile on the Apple M4 Pro with GCC but haven't gotten it to work with Apple's assembler with the SME intrinsics.

If you could point us to the docker container for build-8, we could investigate that issue further, but otherwise we have no issue with leaving it as is for now.

@devinamatthews

devinamatthews commented Jul 9, 2026

Copy link
Copy Markdown
Member

I don't suppose you've tried non-Apple clang? We might need to blacklist that too.

@Luislo1

Luislo1 commented Jul 10, 2026

Copy link
Copy Markdown
Author

We have only tried Homebrew clang in the M4 Pro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Luislo1 @devinamatthews @jeffhammond