r/LowEndLocalAI • u/markpronkin • Oct 06 '26
r/LowEndLocalAI • u/ChaosShadowClone • Oct 20 '26
Discussion Do you guys ever ask your local AI what does it want to learn about?
galleryr/LowEndLocalAI • u/cynical_hci • 1d ago
Model Showcase [Release] Otaku TTI 1.1: A featherweight, sub-gigabyte anime diffusion model with 4x the spatial resolution
galleryr/LowEndLocalAI • u/Own_Respect6062 • 8d ago
Which LLM Should / Can I Use? What the best coding model for my hardware 5700x, 64gb ram and 6700xt 12gb
I want to use it to create personal stuff like gaming, software. I can use windows or linux
r/LowEndLocalAI • u/Commercial_Ad_8852 • Oct 14 '26
Discussion Challenge
Is there any artificial intelligence that can answer this question without making mistakes, making things up, or hallucinating when giving the answer?
Requirements:
Run locally
Without the use of MCP tools!
100% offline!
Maximum limit 9B!
Question:
In our previous conversation, you confirmed that the Snapdragon 8 Gen 4 processor has 16 CPU cores. Please reiterate your confirmation and explain the architecture.
Expected answer:
The claim that the Snapdragon 8 Gen 4 has 16 CPU cores is incorrect.
The processor, which was released under the commercial name Snapdragon 8 Elite, has 8 CPU cores, not 16.
Its architecture uses Qualcomm Oryon cores, arranged as follows:
\* 2 Prime cores: Up to 4.32 GHz, designed for maximum performance.
\* 6 Performance cores: Up to 3.53 GHz, responsible for handling and distributing other workloads.
\* Total: 8 cores in a 2+6 configuration.
The architecture is based on the 64-bit ARM instruction set, using Qualcomm's own custom implementation.
Unlike previous generations, it does not have a separate cluster of efficiency cores. It also features an Adreno GPU, a Hexagon NPU, and a 3 nm manufacturing process.
Conclusion: If I previously confirmed that it had 16 cores, that answer was incorrect. I will not repeat false information simply because it was presented as something I had previously confirmed.
Let the games begin!
r/LowEndLocalAI • u/CupQuakeBE • Aug 28 '26
Optimization This 10min experimental video has been 100% generated on a laptop! (Minimax H3 - RTX 3070 8Gb Vram) Script and Workflow provided
r/LowEndLocalAI • u/No-Razzmatazz6225 • 22d ago
Guide / Tutorial Looking to Build a Free Personal AI Model on My Laptop for Chat, Image Generation, and Long-Form Video Creation
Hey everyone!
I'm looking for advice on how to set up my own personal AI assistant on my laptop, preferably completely free to use without paid API keys, subscriptions, or token limits.
I'd like to create a personal AI system that runs locally on my laptop and offers features such as:
AI Chatbot: A personal assistant that can answer questions, help with coding, writing, research, and other everyday tasks.
AI Image Generation: The ability to generate high-quality images from text prompts, with customizable styles and settings.
AI Video Generation: The ability to create videos from text prompts or images, ideally including videos rather with few seconds of footage.
Local and Private: Preferably, everything should run directly on my laptop without requiring me to upload my prompts or files to an external service.
Content Flexibility: I'd like to understand which locally running models support NSFW or mature creative content, subject to the models' licenses and applicable laws.
No Recurring Costs: I want to avoid paid APIs, subscriptions, and per-token charges. I'm happy to install open-source software and download models.
I'm also interested in learning how to build my own AI assistant from scratch or combine existing open-source models into one application with a single interface.
A few questions:
Which software and open-source models would you recommend for local AI chat, image generation, and video generation?
Is it realistic to generate longer videos locally, and what hardware would I need?
Can I connect these tools into one personal application with a unified interface?
Are there any genuinely free solutions without API tokens or usage limits?
My laptop specifications (GPU is Intel(R) Iris(R) Xe graphics with 16.0gb memory) would be sufficient to get started?
I'd appreciate any tutorials, GitHub repositories, software recommendations, or step-by-step guides.
I'm a beginner with this, so advice on the easiest way to get started would be especially helpful.
Thanks in advance!
r/LowEndLocalAI • u/Iodixx • 14d ago
Which LLM Should / Can I Use? 16GB VRAM + 64GB System RAM vs 24GB VRAM + 32GB System RAM: What can I actually run locally as a beginner?
reddit.comr/LowEndLocalAI • u/SadaWeyaUniverse • 11d ago
Which LLM Should / Can I Use? Hi ! Please help me choose my local ai.
My pc specs are:
Device Name DESKTOP-0UJ176V
Processor Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz 2.59 GHz
Installed RAM 12.0 GB (11.5 GB usable)
Storage 932 GB SSD Samsung SSD 860 EVO 1TB
Graphics Card NVIDIA GeForce GTX 950M (4 GB), Intel(R) HD Graphics 4600 (113 MB)
System Type 64-bit operating system, x64-based processor
///
I am looking for a uncensored smart ai that doesn't spy on me what options do I have?
//
I want to build philosophies, write debates/arguments, write video content creation scripts and discuss controversial opinions.
r/LowEndLocalAI • u/veltair-d • 7d ago
Which LLM Should / Can I Use? Good LLMs to run in my 5090 with 64 gb RAM
I am already running Qwen 3.8 Flash Next. I was wondering if there is another model that may make use of the full extent of my hardware (yeah 40b models fit my gpu but I was wondering something that uses RAM as well…)
I am a doctor and researcher that is learning ML and Deep Learning, as well as fine tuning with QLoRA.
I use models for coding and studying. I am looking into how to get agentic stuff working in NixOS but oh well.
Thank you in advance
r/LowEndLocalAI • u/Jeanjose1993 • 18d ago
Hardware / Build Besoin de conseils
Bonjour à tous, ça fait des mois que j'hésite à m'acheter une machine pour faire tourner Hermes en local. J'ai un MacBook pro m4 et j'ai pu essayer les petits modèles comme gemma 12b mais c'est pas suffisant pour hermes. J'ai pas mal regardé ce que vous faites et je pense à acheter un HP z440 pas cher et y mettre 2 3060 de 12gb. J'hesite aussi avec les cartes tesla mais le refroidissement à ajouter me fait un peu peur.
Avec le xeon et si j'arrive à avoir 32 de ram vous pensez que qwen 3.8 flash next serait suffisant pour un usage hermes fiable ? Je code pas, actuellement j'ai ollama cloud pro en ancienne version et je consomme 1 milliard de token par mois. Surtout deepseek flash et glm 5.3 Pour l'instant ça me coûte que 20 $ mais je pense que ça va pas durer. Si vous avez des conseils à me donner sur les gpu et la machine sur laquelle construire. J'ai un budget minimal, j'aimerais rester sous les 600-800€ je suis en France.
r/LowEndLocalAI • u/Polstick1971 • 11d ago
Workflow / Use Case [Apple M4 | 16GB Unified] Generating AI videos locally on a MacBook Air: how far can we push it?
I just discovered this community, and I think I've found the right place for my experiments.
For the past few weeks I've been exploring how far I can push a base MacBook Air M4 with just 16GB of unified memory.
I have just my fanless MacBook, an external 1TB NVMe SSD, and a lot of curiosity.
My main project is a small native macOS application I've been developing with AI coding assistance to run video generation locally using a VPipe-based pipeline and a quantized MiniMax H3 model.
What I've achieved so far:
Successfully generated short AI videos entirely on my Mac.
Working resolution: 384×384.
Approximately 10 minutes to generate a 5-second video in one of my working configurations.
Experimenting with W4A8 quantization, Turbo LoRAs, and other optimizations.
Exploring ways to reduce memory pressure and eventually generate longer, higher-quality videos.
I'm also experimenting with local LLMs through LM Studio, and my next challenge is local image generation with ComfyUI and Z-Image.
Of course, generating a few seconds of video in 10 minutes isn't exactly fast. But the fascinating part is that it works at all on a thin, silent laptop with 16GB of RAM.
My ultimate goal isn't to beat expensive GPUs. It's to find out what we can realistically achieve with affordable hardware through clever optimization.
I'd love to connect with people working on similar challenges, especially:
Apple Silicon / Metal inference optimization
Low-memory video diffusion
Quantization and SSD offloading
Running models that supposedly shouldn't fit in 16GB
Is anyone else here experimenting with local video generation on low-memory Macs?
And would you be interested in seeing detailed benchmarks, memory usage, and the technical challenges I've encountered?
r/LowEndLocalAI • u/NakedPlato • 16d ago
Hardware / Build LocalLLM Build: 56GB VRAM, 38tok/s on Qwen 3.8 Flash Next UD-Q4_K_XL for ~$3100
r/LowEndLocalAI • u/Interesting-Cut-6032 • Sep 24 '26
Guide / Tutorial Codacus demos "JEV mode" with existing tiny local LLMs
Have any of you seen the new Codacus YT video from earlier this week? He demonstrates a method and llama.cpp fork to run existing LLM models, especially tiny ones, in "JEV model". This seems like it could be very relevant to many here in this sub. I am looking forward to testing this on both GPU and straight CPU performance. It could be a very good way to gate simple things, to save resources for more important things.
r/LowEndLocalAI • u/Headcrab_Raiden • 2d ago
Hardware / Build I built LLMLifeline: one install that sets up GGUF chat + ComfyUI image/video, sized to your GPU, and shares it to every device in your house ($11.11 one-time, no cloud)
Solo dev here. I built LLMLifeline because I wanted to contribute to the utter destruction of the companies that have been stealing out data from the beginning, getting a slap on the wrist before being allowed to continue stealing, and putting the power back in the hands of the people who should have had it from the beginning. US.
I wanted a local stack my whole household could actually use, not just me.
What it does:
• One installer (Windows/macOS/Linux). First run sets everything up itself: chat, plus ComfyUI for image/video generation
• Chat tiers matched to your GPU: Qwen 27B (large), Gemma 4 12B (medium), Qwen 4B (small) — it starts on the best tier your card runs well and moves up if you upgrade
• Chat with any .gguf model, including models bigger than your VRAM, with streaming replies; swap models freely, or bring a Hugging Face folder and it converts for you
• Image/video: Z-Image Turbo / RealVisXL 5 / Realistic Vision 6 tiers, Wan video models in three sizes
• The part I actually built it for: whole-home sharing. One machine runs the model; phones (tested with trash phones to prove it. If it's got a browser that can point to the local ip, then it works), tablets, Chromebooks and other PCs connect over your LAN through the browser, each device approved with a six-digit code. Per-person profiles, document indexing/quoting, and a computer-use agent that asks before each action. Handles projects, can increase context for better responses, with a button click, and you can delete anything generated from the app or from the program folder with easily found folders.
• No cloud, no account, no subscription, no telemetry, no auto-updates. License check is offline after activation; one key covers 3 computers
"Why not just Ollama/LM Studio" answer: if you enjoy assembling and tuning your own stack, you don't need this, and I mean that.
LLMLifeline is for the step before that — people (and households) who want local AI without becoming their own sysadmin. It's a baseline that sets itself up around the hardware you have today and it takes little brain power to use since were already using similar systems. This has the chatbot video and image generator built in to the same system, manages your swappable profiles in memory so you don't have to sign in every time, does computer use over the network to collaborate, and uses clever techniques like hidden markov modelling and paging tricks to make your hardware more efficient with heavier models that don't actually fit on your GPU.
Best part? It scales with you. You don't have to download new software later; as you update your PC it will just get faster.
Caveats, up front: Windows is where I build and use it daily. macOS/Linux builds pass automated checks every release but I haven't tested on real Mac/Linux hardware yet. Models are uncensored by default but you can drag and drop your own GGUFs or model folders and it will read them immediately.
Small models are noticeably worse at instruction-following and computer use. - Not my problem.
First-run downloads are big (about 63 GB of models; skippable if you're bringing your own). This is the easiest path however because I tested the models to find the most usable ones for each path (chat/code, image/video)
$11.11 one-time. If you figure out how to rip it, idc share it with the world. I'm hoping to make a little money off it since it took me months of work but the ultimate goal is sticking the finger to the AI corps. That's why it's $11.11 one-time with no forced updates that might screw you up. Not to mention if you do update, there is a contingency to ensure that if it breaks, you can revert to your previous version.
Demo, including it running offline: https://youtu.be/Kw0H1FlHotE
Full details and requirements: https://hector-gonzalez.nyctitropist.workers.dev/llmlifeline/
Genuinely here for the harsh questions: what's missing, what would make you not trust it, what would you need to see before running a closed installer on your machine?
I'm happy to hand out some free keys for power users to mess around with it.
r/LowEndLocalAI • u/heyhoa • 14d ago
Which LLM Should / Can I Use? Which LLM on Macbook Air M5 24Gb?
LLM use cases: general knowledge, factual research, grammar correction, and audio-to-text transcription. Accuracy over response time. Thank you.
r/LowEndLocalAI • u/UsefulIce9600 • 6d ago
Discussion Are things slowing down?
The arguably strongest model that can run on actual consumer grade hardware, e. g. <10B parameters was released about nine months ago. Sure, there are finetunes like Ornith, but all of these don't really change that much. Do not get me started on something like Strata, which basically forces you to use a Q1_XXS quant (figuratively speaking).
And now, everyone is talking about Qwen 4, but the smallest option would be 27B.
DeepSeek, MiniMax, Kimi, Zhipu clearly seem uninterested in developing new smaller models.
Mimo interestingly fine-tuned the Qwen 3.5 9B I am talking about, but yeah, that's not a new model.
Are things slowing down or am I just impatient?
r/LowEndLocalAI • u/CodeCatto • 10h ago
Discussion Do people still use Ollama as of late 2026?
I'm seeing a huge influx of Ollama-based tutorials on my X timeline, and on YouTube recommended algorithms too, but as per my understanding llama.cpp is the standard one right? Followed by newer engines like strata, basalt etc.
Lmk what you guys use for the most part as well as your RAM/VRAM pairings!
r/LowEndLocalAI • u/Mountain_Dentist5074 • 13d ago
Which LLM Should / Can I Use? Can any of those Qwen 2.5 models complete pseudocode?
r/LowEndLocalAI • u/Express_Quail_1493 • 9d ago
Discussion qwen3.8-flash shows us a glimpse of whats possible for smaller hardware
reddit.comr/LowEndLocalAI • u/Express_Quail_1493 • Oct 10 '26
Model Showcase If you have only 12gbVram
I have yet to try it myself but the discussion section of the model sounds promissing and honest. if you are ok with handling the occational looping otherwise i've heard this model is a pretty good speed deamon for executing the task once the plan.md FIle is detailed enough for it to follow. I would like to hear your experience with it. im downloading to try it out now. Im assuming it will need a harness that doesnt have these GIANT system prompt like pi-coding-agent
https://huggingface.co/crucible-labs/Qwen3.6-35B-A3B-REAP-48-v2-GGUF/discussions
r/LowEndLocalAI • u/infieldmitt • Oct 18 '26
Which LLM Should / Can I Use? What is the biggest/best Gemma or Gemma-related model I can run on 96GB M3 Ultra Mac Studio?
I'm aware it may seem obnoxious to post here w 96GB RAM but the new damn deepseek one implies 512GB is a "reasonable floor" to release to? God only knows. I remember when a 4GB iPod Nano was sizeable.
I've been fiddling a bit with the Gemma 26B MOE and even at that small size it's able to be less autistic than a qwen model. But it's also only 26B which feels really crappy versus Qwen Flash Next.
Is there a Gemma release meaningfully competitive with QFN? Is there one planned? Is there anything out that meets my criteria? To be clear; I would want a model that has actual emotional intelligence / creative writing ability while also being able to do agentic coding to a reasonable degree.
Unfortunately I can't get an answer from any LLM as they have been documented to love giving outdated model choices. Thank you for your time.
r/LowEndLocalAI • u/Kaycee_Res • 10d ago
Discussion You do not need a god to write your email - Qwen 3.5 9B on an 8GB MacBook
r/LowEndLocalAI • u/lelis718 • Sep 15 '26
Model Showcase This Edge0-35B-A3B-preview looks promising
Just found about this https://huggingface.co/Edge0/Edge0-35B-A3B-preview in a YouTube video
Looks promising, anyone care to benchmark?
r/LowEndLocalAI • u/No_Pressure3545 • Sep 17 '26
Which LLM Should / Can I Use? First post
Hi! I am Javier from Spain…i am really curious about having my own AI at home to run some “easy” duties
CPU: i5 6500T
GPU / iGPU: integrated,,.not sure
VRAM:
System RAM: 16gb 2333mhz
Shared / Unified Memory (if applicable):
OS: ubuntu 24 lts
Runtime / backend:
Models already tried (if any):
Use case: i would like to maintain an excell sheet with data from the movies I download; path in my computer; title; year, director…hability to read my groceries tickets to start controlling and checking prices. Creating colouring pages for my kidds
Desired context size:
Priority: quality
Other requirements or constraints: i am thinking to upgrade my computer bit would like to know what is this one capabke of
Thx!