r/LocalAIServers • • 5d ago

LocalAIServers Reproducible Benchmark Chart

1 Upvotes

Qwen3.8-27B in FP16, plus local RTN8 and RTN4 builds. MI50 32GB (gfx906) · V620 (gfx1030) TP2 / TP4 / TP8 · native MTP2 on and off

A strict correctness gate decides whether speed counts. Speed is the exact 2K sample; 10K is diagnostic. Failed rows stay published, marked.

FP16, MTP off, 2K sample, direct comparison:

ConfigMI50 tok/sV620 tok/sStrict
TP219.913.1PASS
TP453.633.5PASS

Validation: 47 of 48 configurations verified end to end · 191 of 192 command executions. One capacity-blocked configuration is published as a reviewed failure. Three V620 TP8 rows are unmeasured and say so.

Every row carries a pinned image and the exact commands to reproduce it. Run one. Post your numbers in this thread.

Full results, images, and reproduction commands


r/LocalAIServers • • Jun 20 '26

Start Here: LocalAIServers Community AI Navigation & Hands-On Local AI Learning

9 Upvotes

Start Here: LocalAIServers

LocalAIServers is a 501(c)(3) public charity providing public education and open-source infrastructure for locally hosted AI systems.

Our mission is to help people move from AI curiosity to AI agency.

This community helps learners, small business owners, nonprofit operators, educators, builders, and community technologists understand:

  • where AI runs,

  • what data it can see,

  • what systems it can touch,

  • when cloud AI may be appropriate,

  • when local or controlled AI may be safer,

  • what hardware is realistic,

  • how to evaluate benchmark claims,

  • and how to learn by building real local AI systems.

What LocalAIServers does

LocalAIServers provides:

  • community AI navigation,

  • secure local-AI education,

  • hands-on local AI learning resources,

  • reproducible runtime artifacts,

  • benchmark literacy,

  • QC and hardware-verification methodology,

  • open-source documentation,

  • and public support resources for locally hosted AI systems.

Affordable GFX906-class hardware matters because it gives people a realistic way to learn AI infrastructure hands-on. People learn more by building, testing, troubleshooting, and verifying real systems than they can learn from passive videos or articles alone.

Public proof and documentation

Website:

https://localaiservers.com

GitHub:

https://github.com/joe2gaan/localaiservers

GitHub Releases:

https://github.com/joe2gaan/localaiservers/releases

Docker Hub:

https://hub.docker.com/r/joe2gaan/localaiservers

Canonical Qwen / GFX906 deployment notes:

https://github.com/joe2gaan/localaiservers/blob/main/qwen36-gfx906/README.md

Important boundaries

LocalAIServers is not:

  • a public login service,

  • a public cloud provider,

  • a managed inference service,

  • a hardware reseller,

  • a procurement channel,

  • a fulfillment program,

  • a hardware discount program,

  • or a private-benefit program.

The controlled GFX906 compute site is used as a verification and reproducibility testbed. Public benefit is delivered through published outputs: guides, documentation, reproducible artifacts, benchmark reports, QC methods, hardware-verification standards, and source-level findings.

How to participate

Ask questions, share builds, discuss local AI tradeoffs, post benchmark questions, and help turn recurring community questions into durable public guides.

Please do not post secrets, private keys, private network details, addresses, payment information, vendor pricing, or sensitive logs.


r/LocalAIServers • • 3h ago

2× R9700 vs. 2× RTX 5090 experience. Anyone Tried Both?

Post image
21 Upvotes

I'm from Autonomous and we’ve been building NVIDIA rigs for clients for a while. Lately, GPU prices have been making these builds harder for clients to justify.

thinking about building a dual Radeon AI PRO R9700 rig to see how it compares with our dual RTX 5090 setup for local AI since they have the same 32GB VRAM.

I know lots of folks here are running on AMD cards. How’s the experience compared with NVIDIA, especially getting both GPUs working and keeping inference stable day to day? What's your advice for starters?


r/LocalAIServers • • 17h ago

My beast of burden

Thumbnail
gallery
129 Upvotes

I've been putting this together the last couple months.

  • 5950x CPU + 128GiB DDR4 3600Mhz + 2TB NVME

  • 3x v620 32GiB running Qwen 3.8 flash next Q4 in strata, EmbeddingGemma2 in extra VRAM

  • rtx pro 5000 embedded blackwell 24GiB running Qwen 3.8 27B nvfp4 in ninfer

  • rtx 5000 embedded ada 16GiB running Gemma 4 12B, whisper large

I offset the V620s 40mm to avoid smashing the motherboard IO headers. The blackwell is connected through the chipset m.2 slot. The ada is shamefully crammed into the HDD cavity behind the motherboard, held in place by zip ties.


r/LocalAIServers • • 5h ago

The best (and AMD-est) computer I've ever owned

Post image
6 Upvotes

r/LocalAIServers • • 21m ago

Small question

1 Upvotes

Can I pair a 5060ti 8gb with an Intel pro b60? I want to be ready for the 27b version of qwen4


r/LocalAIServers • • 9h ago

Which Model Should I Run on My Tesla P100 16GB?

Post image
5 Upvotes

r/LocalAIServers • • 5h ago

Best direction to go for my setup EPYC 32core + 512GB DDR4 + 44GB GPU ?

1 Upvotes

Just looking for some advice on what is possble with my setup and speeds I should be gettingI know it won't be super fast tps but I was more after getting the bigger models to run.

This is my spec:

Epyc 7502p - 32 Core

Gigabyte MZ32-AR0 - 8 channel memory

512GB DDR4

1x 4070 12GB

2x 4060 16GB

quite a few implementations I see which can be confusing.

Is this build even useful now? that we have Project Maya which you can stream from the NVME drive?

Anyone have similar setups and what to expect, what software best to use


r/LocalAIServers • • 10h ago

Turning my AI workstation into a home server

Post image
2 Upvotes

I have a couple of ideas. Would it be possible to use different fingerprints at startup to log into different operating systems? Or during everyday use, could scanning a particular finger quickly launch a different app? As for the Copilot key, does anyone actually use it? Can it be remapped to do something else? This is a nimo laptop, and I couldn't find anything about this in the documentation they provided. Curious to hear what everyone thinks.


r/LocalAIServers • • 13h ago

About to add a 5060 Ti 16GB to my 4060 Ti 16GB for local agentic coding (Qwen3.8-27B, 128K ctx or similar ) — suggestions? Help out a newbie!

Thumbnail reddit.com
2 Upvotes

r/LocalAIServers • • 18h ago

Building my first duel 5080 Nalvo server

4 Upvotes

r/LocalAIServers • • 14h ago

Train AI to run AI: introducing Apareto loading - a vLLM plugin

Post image
1 Upvotes

r/LocalAIServers • • 21h ago

[FS][WW] NEW Kioxia CD8P-R 15.36TB PCIe 5.0 Enterprise NVMe SSD (x3)

1 Upvotes

r/LocalAIServers • • 1d ago

Ai Rig finally done and stable, perfect winter heater

Post image
16 Upvotes

r/LocalAIServers • • 1d ago

Strata Performance Setup (TUI performance optimization)

Post image
3 Upvotes

r/LocalAIServers • • 1d ago

Elastic KV cache on RDNA4 (2× R9700): anyone else tried kvcached or HIP VMM ballooning on ROCm?

Thumbnail
reddit.com
3 Upvotes

r/LocalAIServers • • 1d ago

Local Ai on asus rog strix g18 laptop

Thumbnail reddit.com
1 Upvotes

r/LocalAIServers • • 2d ago

Zikporx AI, my local LLM chat app for Mac, now does voice both ways: talk to it, and it talks back. Everything stays on your Mac.

Post image
3 Upvotes

r/LocalAIServers • • 1d ago

inferzero update: Intel Arc from a laptop iGPU to 4× Pro B60. DeepSeek-V4.1-Flash over several cards, Flash Next on a 32 GB laptop, a Linux server

Thumbnail
reddit.com
1 Upvotes

r/LocalAIServers • • 1d ago

I built a gpu hosting marketplace and it’s cooking

Thumbnail
reddit.com
1 Upvotes

r/LocalAIServers • • 2d ago

I have a PC with 32gb of RAM, a 16gb 5080 and an AMD 9900 X

5 Upvotes

I’m interested in running local AI models. I have a 1 Tb NVME SSD that I was going to install Linux on. Any recommendations on a distro? I’ve been an Ubuntu guy for about 16 years.


r/LocalAIServers • • 2d ago

Mac Studio 256 vs 512

5 Upvotes

Hi,
I am planning to get a Mac studio 256/512 for hosting LLMs for a scientific product I am building.

I work in synthetic biology and frequently get requests denied by Anthropic / OpenAI frontier. (Due to the nature of my research)

Now I am thinking to get this to be able to run a LLM locally and not send research data to a provider.

Now my question, what sort of models can you run on a 512 vs the 256? I have tried qwen 3.8 flash next on another machine and that was decent but I am kinda trying to figure out if it’s worth paying the premium for larger parameter models.
How much more intelligence/ performance do you actually get ? Is it fine to use SSD for larger than 256gb MOE models or does that make them practically useless in terms of t/s. Also what about future proofing? Any specific model recommendations ?

Would love some recommendations. Looking into strix halo 495 and dgx spark but when I add it up to 256/512gb it kinda is not a big price difference between those and the Mac.

Thanks in advance !


r/LocalAIServers • • 2d ago

[FS] 4x Radeon Pro V620 32GB - 128GB set with blower fans

Post image
5 Upvotes

r/LocalAIServers • • 2d ago

A year ago I couldn't get vLLM running on my 5090. Now work gave me a DGX Station for a month. What should I run?

Thumbnail gallery
24 Upvotes

r/LocalAIServers • • 2d ago

I built a gpu hosting marketplace and it’s cooking

Thumbnail
reddit.com
0 Upvotes