r/LocalAIServers • • 21m ago

Small question

1 Upvotes

Can I pair a 5060ti 8gb with an Intel pro b60? I want to be ready for the 27b version of qwen4


r/LocalAIServers • • 3h ago

2× R9700 vs. 2× RTX 5090 experience. Anyone Tried Both?

Post image
21 Upvotes

I'm from Autonomous and we’ve been building NVIDIA rigs for clients for a while. Lately, GPU prices have been making these builds harder for clients to justify.

thinking about building a dual Radeon AI PRO R9700 rig to see how it compares with our dual RTX 5090 setup for local AI since they have the same 32GB VRAM.

I know lots of folks here are running on AMD cards. How’s the experience compared with NVIDIA, especially getting both GPUs working and keeping inference stable day to day? What's your advice for starters?


r/LocalAIServers • • 5h ago

The best (and AMD-est) computer I've ever owned

Post image
6 Upvotes

r/LocalAIServers • • 5h ago

Best direction to go for my setup EPYC 32core + 512GB DDR4 + 44GB GPU ?

1 Upvotes

Just looking for some advice on what is possble with my setup and speeds I should be gettingI know it won't be super fast tps but I was more after getting the bigger models to run.

This is my spec:

Epyc 7502p - 32 Core

Gigabyte MZ32-AR0 - 8 channel memory

512GB DDR4

1x 4070 12GB

2x 4060 16GB

quite a few implementations I see which can be confusing.

Is this build even useful now? that we have Project Maya which you can stream from the NVME drive?

Anyone have similar setups and what to expect, what software best to use


r/LocalAIServers • • 9h ago

Which Model Should I Run on My Tesla P100 16GB?

Post image
5 Upvotes

r/LocalAIServers • • 10h ago

Turning my AI workstation into a home server

Post image
2 Upvotes

I have a couple of ideas. Would it be possible to use different fingerprints at startup to log into different operating systems? Or during everyday use, could scanning a particular finger quickly launch a different app? As for the Copilot key, does anyone actually use it? Can it be remapped to do something else? This is a nimo laptop, and I couldn't find anything about this in the documentation they provided. Curious to hear what everyone thinks.


r/LocalAIServers • • 13h ago

About to add a 5060 Ti 16GB to my 4060 Ti 16GB for local agentic coding (Qwen3.8-27B, 128K ctx or similar ) — suggestions? Help out a newbie!

Thumbnail reddit.com
2 Upvotes

r/LocalAIServers • • 14h ago

Train AI to run AI: introducing Apareto loading - a vLLM plugin

Post image
1 Upvotes

r/LocalAIServers • • 17h ago

My beast of burden

Thumbnail
gallery
129 Upvotes

I've been putting this together the last couple months.

  • 5950x CPU + 128GiB DDR4 3600Mhz + 2TB NVME

  • 3x v620 32GiB running Qwen 3.8 flash next Q4 in strata, EmbeddingGemma2 in extra VRAM

  • rtx pro 5000 embedded blackwell 24GiB running Qwen 3.8 27B nvfp4 in ninfer

  • rtx 5000 embedded ada 16GiB running Gemma 4 12B, whisper large

I offset the V620s 40mm to avoid smashing the motherboard IO headers. The blackwell is connected through the chipset m.2 slot. The ada is shamefully crammed into the HDD cavity behind the motherboard, held in place by zip ties.


r/LocalAIServers • • 18h ago

Building my first duel 5080 Nalvo server

4 Upvotes

r/LocalAIServers • • 21h ago

[FS][WW] NEW Kioxia CD8P-R 15.36TB PCIe 5.0 Enterprise NVMe SSD (x3)

1 Upvotes

r/LocalAIServers • • 1d ago

Strata Performance Setup (TUI performance optimization)

Post image
3 Upvotes

r/LocalAIServers • • 1d ago

Local Ai on asus rog strix g18 laptop

Thumbnail reddit.com
1 Upvotes

r/LocalAIServers • • 1d ago

Ai Rig finally done and stable, perfect winter heater

Post image
16 Upvotes

r/LocalAIServers • • 1d ago

inferzero update: Intel Arc from a laptop iGPU to 4× Pro B60. DeepSeek-V4.1-Flash over several cards, Flash Next on a 32 GB laptop, a Linux server

Thumbnail
reddit.com
1 Upvotes

r/LocalAIServers • • 1d ago

I built a gpu hosting marketplace and it’s cooking

Thumbnail
reddit.com
1 Upvotes

r/LocalAIServers • • 1d ago

Elastic KV cache on RDNA4 (2× R9700): anyone else tried kvcached or HIP VMM ballooning on ROCm?

Thumbnail
reddit.com
3 Upvotes

r/LocalAIServers • • 2d ago

Zikporx AI, my local LLM chat app for Mac, now does voice both ways: talk to it, and it talks back. Everything stays on your Mac.

Post image
3 Upvotes

r/LocalAIServers • • 2d ago

I built a gpu hosting marketplace and it’s cooking

Thumbnail
reddit.com
0 Upvotes

r/LocalAIServers • • 2d ago

Built my own from scratch no dep windows ml ai ide

Thumbnail
github.com
1 Upvotes

Feel free to check it out and leave feedback! Thanks I’ll have benchmark numbers soon!


r/LocalAIServers • • 2d ago

I have a PC with 32gb of RAM, a 16gb 5080 and an AMD 9900 X

5 Upvotes

I’m interested in running local AI models. I have a 1 Tb NVME SSD that I was going to install Linux on. Any recommendations on a distro? I’ve been an Ubuntu guy for about 16 years.


r/LocalAIServers • • 2d ago

Mac Studio 256 vs 512

5 Upvotes

Hi,
I am planning to get a Mac studio 256/512 for hosting LLMs for a scientific product I am building.

I work in synthetic biology and frequently get requests denied by Anthropic / OpenAI frontier. (Due to the nature of my research)

Now I am thinking to get this to be able to run a LLM locally and not send research data to a provider.

Now my question, what sort of models can you run on a 512 vs the 256? I have tried qwen 3.8 flash next on another machine and that was decent but I am kinda trying to figure out if it’s worth paying the premium for larger parameter models.
How much more intelligence/ performance do you actually get ? Is it fine to use SSD for larger than 256gb MOE models or does that make them practically useless in terms of t/s. Also what about future proofing? Any specific model recommendations ?

Would love some recommendations. Looking into strix halo 495 and dgx spark but when I add it up to 256/512gb it kinda is not a big price difference between those and the Mac.

Thanks in advance !


r/LocalAIServers • • 2d ago

[FS] 4x Radeon Pro V620 32GB - 128GB set with blower fans

Post image
5 Upvotes

r/LocalAIServers • • 2d ago

Any good prebuilds for Local AI models

3 Upvotes

I am looking into purchasing a PC for a finical firm company they are small staffed and the tasks they would request to the AI are simple, basically just reading CRMs, summarizing meeting transcripts, helping prep for future meetings based on the given data; so Id be combining their current systems some have APIs not all. Overall its fairly simple.

I am hoping to get guided towards any good prebuilds that you may think would be a suitable fit.

I was thinking 24-32Gb VRAM , 64-128 RAM and i may be using 16B gigs of Qwen.

Any good prebuilds you may know of would be awesome and extremely helpful!


r/LocalAIServers • • 2d ago

M1 Air 1TB vs. M1 Pro 500GB as a 24/7 home AI server?

Thumbnail reddit.com
2 Upvotes