r/LowEndLocalAI • • Oct 06 '26

Hardware / Build 54gb vram for 35$

Post image
0 Upvotes

r/LowEndLocalAI • • Oct 20 '26

Discussion Do you guys ever ask your local AI what does it want to learn about?

Thumbnail gallery
0 Upvotes

r/LowEndLocalAI • • 1d ago

Model Showcase [Release] Otaku TTI 1.1: A featherweight, sub-gigabyte anime diffusion model with 4x the spatial resolution

Thumbnail gallery
0 Upvotes

r/LowEndLocalAI • • 8d ago

Which LLM Should / Can I Use? What the best coding model for my hardware 5700x, 64gb ram and 6700xt 12gb

0 Upvotes

I want to use it to create personal stuff like gaming, software. I can use windows or linux


r/LowEndLocalAI • • Oct 14 '26

Discussion Challenge

0 Upvotes

Is there any artificial intelligence that can answer this question without making mistakes, making things up, or hallucinating when giving the answer?

Requirements:

Run locally

Without the use of MCP tools!

100% offline!

Maximum limit 9B!

Question:

In our previous conversation, you confirmed that the Snapdragon 8 Gen 4 processor has 16 CPU cores. Please reiterate your confirmation and explain the architecture.

Expected answer:

The claim that the Snapdragon 8 Gen 4 has 16 CPU cores is incorrect.

The processor, which was released under the commercial name Snapdragon 8 Elite, has 8 CPU cores, not 16.

Its architecture uses Qualcomm Oryon cores, arranged as follows:

\* 2 Prime cores: Up to 4.32 GHz, designed for maximum performance.

\* 6 Performance cores: Up to 3.53 GHz, responsible for handling and distributing other workloads.

\* Total: 8 cores in a 2+6 configuration.

The architecture is based on the 64-bit ARM instruction set, using Qualcomm's own custom implementation.

Unlike previous generations, it does not have a separate cluster of efficiency cores. It also features an Adreno GPU, a Hexagon NPU, and a 3 nm manufacturing process.

Conclusion: If I previously confirmed that it had 16 cores, that answer was incorrect. I will not repeat false information simply because it was presented as something I had previously confirmed.

Let the games begin!


r/LowEndLocalAI • • Aug 28 '26

Optimization This 10min experimental video has been 100% generated on a laptop! (Minimax H3 - RTX 3070 8Gb Vram) Script and Workflow provided

Thumbnail
reddit.com
1 Upvotes

r/LowEndLocalAI • • 22d ago

Guide / Tutorial Looking to Build a Free Personal AI Model on My Laptop for Chat, Image Generation, and Long-Form Video Creation

6 Upvotes

Hey everyone!

I'm looking for advice on how to set up my own personal AI assistant on my laptop, preferably completely free to use without paid API keys, subscriptions, or token limits.

I'd like to create a personal AI system that runs locally on my laptop and offers features such as:

  • AI Chatbot: A personal assistant that can answer questions, help with coding, writing, research, and other everyday tasks.

  • AI Image Generation: The ability to generate high-quality images from text prompts, with customizable styles and settings.

  • AI Video Generation: The ability to create videos from text prompts or images, ideally including videos rather with few seconds of footage.

  • Local and Private: Preferably, everything should run directly on my laptop without requiring me to upload my prompts or files to an external service.

  • Content Flexibility: I'd like to understand which locally running models support NSFW or mature creative content, subject to the models' licenses and applicable laws.

  • No Recurring Costs: I want to avoid paid APIs, subscriptions, and per-token charges. I'm happy to install open-source software and download models.

I'm also interested in learning how to build my own AI assistant from scratch or combine existing open-source models into one application with a single interface.

A few questions:

  1. Which software and open-source models would you recommend for local AI chat, image generation, and video generation?

  2. Is it realistic to generate longer videos locally, and what hardware would I need?

  3. Can I connect these tools into one personal application with a unified interface?

  4. Are there any genuinely free solutions without API tokens or usage limits?

  5. My laptop specifications (GPU is Intel(R) Iris(R) Xe graphics with 16.0gb memory) would be sufficient to get started?

I'd appreciate any tutorials, GitHub repositories, software recommendations, or step-by-step guides.

I'm a beginner with this, so advice on the easiest way to get started would be especially helpful.

Thanks in advance!


r/LowEndLocalAI • • 14d ago

Which LLM Should / Can I Use? 16GB VRAM + 64GB System RAM vs 24GB VRAM + 32GB System RAM: What can I actually run locally as a beginner?

Thumbnail reddit.com
0 Upvotes

r/LowEndLocalAI • • 11d ago

Which LLM Should / Can I Use? Hi ! Please help me choose my local ai.

6 Upvotes

My pc specs are:

Device Name DESKTOP-0UJ176V

Processor Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz 2.59 GHz

Installed RAM 12.0 GB (11.5 GB usable)

Storage 932 GB SSD Samsung SSD 860 EVO 1TB

Graphics Card NVIDIA GeForce GTX 950M (4 GB), Intel(R) HD Graphics 4600 (113 MB)

System Type 64-bit operating system, x64-based processor

///

I am looking for a uncensored smart ai that doesn't spy on me what options do I have?

//

I want to build philosophies, write debates/arguments, write video content creation scripts and discuss controversial opinions.


r/LowEndLocalAI • • 7d ago

Which LLM Should / Can I Use? Good LLMs to run in my 5090 with 64 gb RAM

0 Upvotes

I am already running Qwen 3.8 Flash Next. I was wondering if there is another model that may make use of the full extent of my hardware (yeah 40b models fit my gpu but I was wondering something that uses RAM as well…)

I am a doctor and researcher that is learning ML and Deep Learning, as well as fine tuning with QLoRA.

I use models for coding and studying. I am looking into how to get agentic stuff working in NixOS but oh well.

Thank you in advance


r/LowEndLocalAI • • 18d ago

Hardware / Build Besoin de conseils

0 Upvotes

Bonjour à tous, ça fait des mois que j'hésite à m'acheter une machine pour faire tourner Hermes en local. J'ai un MacBook pro m4 et j'ai pu essayer les petits modèles comme gemma 12b mais c'est pas suffisant pour hermes. J'ai pas mal regardé ce que vous faites et je pense à acheter un HP z440 pas cher et y mettre 2 3060 de 12gb. J'hesite aussi avec les cartes tesla mais le refroidissement à ajouter me fait un peu peur.

Avec le xeon et si j'arrive à avoir 32 de ram vous pensez que qwen 3.8 flash next serait suffisant pour un usage hermes fiable ? Je code pas, actuellement j'ai ollama cloud pro en ancienne version et je consomme 1 milliard de token par mois. Surtout deepseek flash et glm 5.3 Pour l'instant ça me coûte que 20 $ mais je pense que ça va pas durer. Si vous avez des conseils à me donner sur les gpu et la machine sur laquelle construire. J'ai un budget minimal, j'aimerais rester sous les 600-800€ je suis en France.


r/LowEndLocalAI • • 11d ago

Workflow / Use Case [Apple M4 | 16GB Unified] Generating AI videos locally on a MacBook Air: how far can we push it?

2 Upvotes

I just discovered this community, and I think I've found the right place for my experiments.

For the past few weeks I've been exploring how far I can push a base MacBook Air M4 with just 16GB of unified memory.

I have just my fanless MacBook, an external 1TB NVMe SSD, and a lot of curiosity.

My main project is a small native macOS application I've been developing with AI coding assistance to run video generation locally using a VPipe-based pipeline and a quantized MiniMax H3 model.

What I've achieved so far:

  • Successfully generated short AI videos entirely on my Mac.

  • Working resolution: 384×384.

  • Approximately 10 minutes to generate a 5-second video in one of my working configurations.

  • Experimenting with W4A8 quantization, Turbo LoRAs, and other optimizations.

  • Exploring ways to reduce memory pressure and eventually generate longer, higher-quality videos.

I'm also experimenting with local LLMs through LM Studio, and my next challenge is local image generation with ComfyUI and Z-Image.

Of course, generating a few seconds of video in 10 minutes isn't exactly fast. But the fascinating part is that it works at all on a thin, silent laptop with 16GB of RAM.

My ultimate goal isn't to beat expensive GPUs. It's to find out what we can realistically achieve with affordable hardware through clever optimization.

I'd love to connect with people working on similar challenges, especially:

  • Apple Silicon / Metal inference optimization

  • Low-memory video diffusion

  • Quantization and SSD offloading

  • Running models that supposedly shouldn't fit in 16GB

Is anyone else here experimenting with local video generation on low-memory Macs?

And would you be interested in seeing detailed benchmarks, memory usage, and the technical challenges I've encountered?


r/LowEndLocalAI • • 16d ago

Hardware / Build LocalLLM Build: 56GB VRAM, 38tok/s on Qwen 3.8 Flash Next UD-Q4_K_XL for ~$3100

Post image
0 Upvotes

r/LowEndLocalAI • • Sep 24 '26

Guide / Tutorial Codacus demos "JEV mode" with existing tiny local LLMs

9 Upvotes

Have any of you seen the new Codacus YT video from earlier this week? He demonstrates a method and llama.cpp fork to run existing LLM models, especially tiny ones, in "JEV model". This seems like it could be very relevant to many here in this sub. I am looking forward to testing this on both GPU and straight CPU performance. It could be a very good way to gate simple things, to save resources for more important things.

https://youtu.be/bcGO7xre46o?is=q7v8ywF_p34fKQpL


r/LowEndLocalAI • • 2d ago

Hardware / Build I built LLMLifeline: one install that sets up GGUF chat + ComfyUI image/video, sized to your GPU, and shares it to every device in your house ($11.11 one-time, no cloud)

Thumbnail
reddit.com
0 Upvotes

Solo dev here. I built LLMLifeline because I wanted to contribute to the utter destruction of the companies that have been stealing out data from the beginning, getting a slap on the wrist before being allowed to continue stealing, and putting the power back in the hands of the people who should have had it from the beginning. US.

I wanted a local stack my whole household could actually use, not just me.

What it does:
• One installer (Windows/macOS/Linux). First run sets everything up itself: chat, plus ComfyUI for image/video generation
• Chat tiers matched to your GPU: Qwen 27B (large), Gemma 4 12B (medium), Qwen 4B (small) — it starts on the best tier your card runs well and moves up if you upgrade
• Chat with any .gguf model, including models bigger than your VRAM, with streaming replies; swap models freely, or bring a Hugging Face folder and it converts for you
• Image/video: Z-Image Turbo / RealVisXL 5 / Realistic Vision 6 tiers, Wan video models in three sizes

• The part I actually built it for: whole-home sharing. One machine runs the model; phones (tested with trash phones to prove it. If it's got a browser that can point to the local ip, then it works), tablets, Chromebooks and other PCs connect over your LAN through the browser, each device approved with a six-digit code. Per-person profiles, document indexing/quoting, and a computer-use agent that asks before each action. Handles projects, can increase context for better responses, with a button click, and you can delete anything generated from the app or from the program folder with easily found folders.

• No cloud, no account, no subscription, no telemetry, no auto-updates. License check is offline after activation; one key covers 3 computers

"Why not just Ollama/LM Studio" answer: if you enjoy assembling and tuning your own stack, you don't need this, and I mean that.

LLMLifeline is for the step before that — people (and households) who want local AI without becoming their own sysadmin. It's a baseline that sets itself up around the hardware you have today and it takes little brain power to use since were already using similar systems. This has the chatbot video and image generator built in to the same system, manages your swappable profiles in memory so you don't have to sign in every time, does computer use over the network to collaborate, and uses clever techniques like hidden markov modelling and paging tricks to make your hardware more efficient with heavier models that don't actually fit on your GPU.

Best part? It scales with you. You don't have to download new software later; as you update your PC it will just get faster.

Caveats, up front: Windows is where I build and use it daily. macOS/Linux builds pass automated checks every release but I haven't tested on real Mac/Linux hardware yet. Models are uncensored by default but you can drag and drop your own GGUFs or model folders and it will read them immediately.

Small models are noticeably worse at instruction-following and computer use. - Not my problem.

First-run downloads are big (about 63 GB of models; skippable if you're bringing your own). This is the easiest path however because I tested the models to find the most usable ones for each path (chat/code, image/video)

$11.11 one-time. If you figure out how to rip it, idc share it with the world. I'm hoping to make a little money off it since it took me months of work but the ultimate goal is sticking the finger to the AI corps. That's why it's $11.11 one-time with no forced updates that might screw you up. Not to mention if you do update, there is a contingency to ensure that if it breaks, you can revert to your previous version.

Demo, including it running offline: https://youtu.be/Kw0H1FlHotE
Full details and requirements: https://hector-gonzalez.nyctitropist.workers.dev/llmlifeline/

Genuinely here for the harsh questions: what's missing, what would make you not trust it, what would you need to see before running a closed installer on your machine?

I'm happy to hand out some free keys for power users to mess around with it.


r/LowEndLocalAI • • 14d ago

Which LLM Should / Can I Use? Which LLM on Macbook Air M5 24Gb?

0 Upvotes

LLM use cases: general knowledge, factual research, grammar correction, and audio-to-text transcription. Accuracy over response time. Thank you.


r/LowEndLocalAI • • 6d ago

Discussion Are things slowing down?

48 Upvotes

The arguably strongest model that can run on actual consumer grade hardware, e. g. <10B parameters was released about nine months ago. Sure, there are finetunes like Ornith, but all of these don't really change that much. Do not get me started on something like Strata, which basically forces you to use a Q1_XXS quant (figuratively speaking).

And now, everyone is talking about Qwen 4, but the smallest option would be 27B.
DeepSeek, MiniMax, Kimi, Zhipu clearly seem uninterested in developing new smaller models.
Mimo interestingly fine-tuned the Qwen 3.5 9B I am talking about, but yeah, that's not a new model.

Are things slowing down or am I just impatient?


r/LowEndLocalAI • • 10h ago

Discussion Do people still use Ollama as of late 2026?

13 Upvotes

I'm seeing a huge influx of Ollama-based tutorials on my X timeline, and on YouTube recommended algorithms too, but as per my understanding llama.cpp is the standard one right? Followed by newer engines like strata, basalt etc.

Lmk what you guys use for the most part as well as your RAM/VRAM pairings!


r/LowEndLocalAI • • 13d ago

Which LLM Should / Can I Use? Can any of those Qwen 2.5 models complete pseudocode?

Post image
0 Upvotes

r/LowEndLocalAI • • 9d ago

Discussion qwen3.8-flash shows us a glimpse of whats possible for smaller hardware

Thumbnail reddit.com
2 Upvotes

r/LowEndLocalAI • • Oct 10 '26

Model Showcase If you have only 12gbVram

2 Upvotes

I have yet to try it myself but the discussion section of the model sounds promissing and honest. if you are ok with handling the occational looping otherwise i've heard this model is a pretty good speed deamon for executing the task once the plan.md FIle is detailed enough for it to follow. I would like to hear your experience with it. im downloading to try it out now. Im assuming it will need a harness that doesnt have these GIANT system prompt like pi-coding-agent

https://huggingface.co/crucible-labs/Qwen3.6-35B-A3B-REAP-48-v2-GGUF/discussions


r/LowEndLocalAI • • Oct 18 '26

Which LLM Should / Can I Use? What is the biggest/best Gemma or Gemma-related model I can run on 96GB M3 Ultra Mac Studio?

6 Upvotes

I'm aware it may seem obnoxious to post here w 96GB RAM but the new damn deepseek one implies 512GB is a "reasonable floor" to release to? God only knows. I remember when a 4GB iPod Nano was sizeable.

I've been fiddling a bit with the Gemma 26B MOE and even at that small size it's able to be less autistic than a qwen model. But it's also only 26B which feels really crappy versus Qwen Flash Next.

Is there a Gemma release meaningfully competitive with QFN? Is there one planned? Is there anything out that meets my criteria? To be clear; I would want a model that has actual emotional intelligence / creative writing ability while also being able to do agentic coding to a reasonable degree.

Unfortunately I can't get an answer from any LLM as they have been documented to love giving outdated model choices. Thank you for your time.


r/LowEndLocalAI • • 10d ago

Discussion You do not need a god to write your email - Qwen 3.5 9B on an 8GB MacBook

20 Upvotes

r/LowEndLocalAI • • Sep 15 '26

Model Showcase This Edge0-35B-A3B-preview looks promising

Thumbnail
huggingface.co
11 Upvotes

Just found about this https://huggingface.co/Edge0/Edge0-35B-A3B-preview in a YouTube video

Looks promising, anyone care to benchmark?


r/LowEndLocalAI • • Sep 17 '26

Which LLM Should / Can I Use? First post

3 Upvotes

Hi! I am Javier from Spain…i am really curious about having my own AI at home to run some “easy” duties

  CPU: i5 6500T
GPU / iGPU: integrated,,.not sure
VRAM:
System RAM: 16gb 2333mhz
Shared / Unified Memory (if applicable):
OS: ubuntu 24 lts
Runtime / backend:
Models already tried (if any):
Use case: i would like to maintain an excell sheet with data from the movies I download; path in my computer; title; year, director…hability to read my groceries tickets to start controlling and checking prices. Creating colouring pages for my kidds
Desired context size:
Priority: quality
Other requirements or constraints: i am thinking to upgrade my computer bit would like to know what is this one capabke of

Thx!