I'm on Asahi 42. I've done as much research as I can, and so far, llama.cpp seems like the only general model worker that works in Asahi. So I downloaded the latest git repo and compiled it with Vulkan support.
I've got the Unsloth Qwen3-Coder-30B-A3B-Instruct-Q4_K_M gguf downloaded, and been playing around with settings to try to get it faster.
First off : -ngl 40 is the highest I can go without running out of memory; for whatever reason, it just can't load any more than this (<50%) into memory. So I know this is gonna limit speed a bunch.
I run llama-bench, and it only shows about 40-60 tps pp, 18-22 tps tg. So about 20 t/s. This same model in MacOS on the same machine gives 54 t/s, using mlx. Now I know mlx is a bit more efficient than gguf, but three times more?! I don't think so...
Either Asahi is just not ready to run LLMs fast, or I'm doing something wrong. Is anyone getting faster, and does anyone have more guidance on setup please?