r/ollama • • 1d ago

I built this! [Weekly - Nov 20, 2026] Friday Demos - Show off your Ollama projects!

1 Upvotes

Show us what you've built with Ollama this week!

Suggested information to include:

  • A description of your project

  • How you came up with the idea

  • The models and integrations you're using

  • Whether it runs on local hardware or the cloud service

  • How others could use it or view it

  • A link to code or demo if applicable

Rules:

  • No affiliate links or sales

New post every Friday!


r/ollama • • 7h ago

What models to use on 8gb vram and 16gb ram?

1 Upvotes

Hey. Im Just starting with local ai and was wondering what models can i run with an rx6600 8gb vram and 16gbr ram. My main purpose would be to make script and vídeos by code for my YouTube channel


r/ollama • • 9h ago

I built a quick pass/fail test suite to see which small local models actually survive inside an AI agent (Gemma 4 on an 8GB GPU surprised me)

Post image
3 Upvotes

r/ollama • • 14h ago

I built this! Evokai: turn an Ollama model into a real in-app assistant, with backend, chat UI, admin console, permissions and budgets in one service

1 Upvotes

r/ollama • • 14h ago

Advice Best coding AI model to run on a standby machine? Time is not a problem: 8GB VRAM, 32GB RAM

11 Upvotes

I have a second machine running an RTX 4060 8GB, i7 12700H and 32GB DDR4 RAM.

It runs on standby, sometimes using ComfyUI to generate images for my DnD games.

I wanted to try Craft Anything to just test and try out some programming. I know that the VRAM etc. is not enough and I'll have to offload into RAM.

Time is not a big thing for me, I can use my main machine in the mean time.


Is it feasible? Possible? I want to use Ollama / LMStudio - MCP interface - to use with CraftAnything.

Any idea to guide me through a fitting model? Thanks a lot! :)


r/ollama • • 16h ago

I built a small gateway to use my home LLM from my other computers, without port forwarding or a VPN

Post image
2 Upvotes

r/ollama • • 17h ago

best uncensored cloud image model

1 Upvotes

trying to find a cloud model (or local host model) for image to remove water mark. Any ideas?


r/ollama • • 18h ago

Do local LLM users even want this?

Thumbnail
reddit.com
1 Upvotes

r/ollama • • 19h ago

Ollamaskope: My answer to keeping track of it all.

Thumbnail
gallery
3 Upvotes

Hey there,

Anyone working with multiple instances—that is, Ollama endpoints—surely knows the problem: the more models you use and test, the messier it gets. Which Modelfile actually contains the base model, and why do I even need certain models twice? Mostly for no reason at all—often they're just leftover models that were simply forgotten and never deleted. Working in the terminal is fine, sure, but it's too hard on my eyes. That's why I built an Ollama management tool for myself: Ollamaskope.

Inside the web GUI, I can now cleanly manage my Ollama instances at my own pace, delete models, create Modelfiles, load a model for testing, or quickly unload it again. And all of that comfortably in a web browser.

Anyone who could make use of this can easily download it on GitHub:https://github.com/o-valo/ollamaskope


r/ollama • • 21h ago

I built this! I fooled myself with "100% GPU" in ollama ps and nearly doubled my 14B speed on an 8GB RTX 3050 (5.8 → 11.1 tok/s)

3 Upvotes

Been building a local AI companion on an 8GB RTX 3050 (Qwen 2.5 14B Q4_K_M, num_ctx 8192) and couldn't figure out why generation was stuck at ~5.8 tok/s even though ollama ps proudly reported 100% GPU.

Turns out I ignored the column right next to it: SIZE: 10 GB. On an 8GB card.

Because I had num_gpu 99 set in my Modelfile, Ollama tried to shove all 48 layers onto the GPU. Instead of throwing an OOM error, the NVIDIA driver silently intercepted the overflowing ~2 GB and spilled it into system RAM via Shared GPU Memory, choking bandwidth on every single token.

What fixed it:

  1. Disabled CUDA - Sysmem Fallback Policy (Prefer No Sysmem Fallback) in the NVIDIA Control Panel so the driver stops hiding VRAM overflows.

  2. Manually tuned num_gpu in my Modelfile to 38 (out of 48 layers) alongside OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0.

Results:

  • VRAM usage: 8137 / 8192 MB (leaving just enough headroom for local Silero TTS on GPU)

  • Split: Honest 75% GPU / 25% CPU

  • Prompt eval: 120 → 260.7 tok/s

  • Generation speed: 5.8 → 11.12 tok/s (nearly 2x faster just by stopping the PCIe thrashing)

Moral of the story: if ollama ps says your 10GB model fits 100% GPU on an 8GB card, you didn't break physics—your driver is just lying to you.


r/ollama • • 21h ago

Ho creato una sala di controllo per sciami di droni: spettacoli di luci, missioni di ricerca, volo manuale e 16 visuali da telecamera in un'unica finestra. Demo nel tuo browser

Post image
1 Upvotes

r/ollama • • 21h ago

Ollama 0.40 vs llama.cpp, LM Studio and vLLM on a 3090 with Qwen3.8-27B: least GPU memory (17 GiB at 32k), but the lowest output at 32 requests and 88 s to the first answer

Thumbnail
homebrewserver.club
49 Upvotes

I ran the same model (Qwen3.8-27B) on one RTX 3090 in Ollama, llama.cpp, LM Studio and vLLM, and sent each the same questions.

Where Ollama was best

  • It used the least GPU memory.

  • It never failed. When many long questions came at once, it made them wait in line. Two of the others failed some of these.

  • It holds a lot of text, almost as much as llama.cpp.

Where Ollama was worst

  • Slowest with many users. By default it answers one question at a time, so everyone else waits. Setting OLLAMA_NUM_PARALLEL=4 made it about 2.5 times faster, but vLLM still did over 3 times more.

  • Slow first answer: about 1.5 minutes after starting, because Ollama loads the model only when the first question arrives. After that, no wait.

  • Slower for one person than LM Studio and llama.cpp. Ollama's MTP version of the model helped (about 30% faster), but it stayed behind.

Tip: if several people or apps use your Ollama, set OLLAMA_NUM_PARALLEL.


r/ollama • • 21h ago

Local model running Sesame (voice chat)? Possible? comparison to hosted?

Thumbnail reddit.com
2 Upvotes

r/ollama • • 22h ago

Hermes Agent + Qwen 3.8 27B on RX 7900 XTX: Ollama is fast, but Hermes is significantly slower. Looking for optimization advice.

Thumbnail reddit.com
2 Upvotes

r/ollama • • 23h ago

Just turned my old Android into a free local AI server for Cursor

0 Upvotes

I honestly didn't think this would actually work, lol.

I've been trying to cut down on API costs (and stop my laptop fan from sounding like a jet engine), and I stumbled across Ollama Local AI (ollamamobile.app).

Basically, it runs GGUF models right on your Android phone offline, and lets you connect Cursor / VS Code / Windsurf to it over local Wi-Fi like an OpenAI endpoint.

I threw a quantized model on my phone expecting it to crash or crawl, and it’s actually streaming code completions straight into my IDE with zero internet and $0 in API fees. My phone is just sitting on my desk doing the heavy lifting.

[Attach your screenshots here]

Has anyone else tried this yet? Curious what models you guys are running and what tokens/sec you're getting on your phones.


r/ollama • • 1d ago

I used Giraffe to test Ollama’s KV cache on a T4. Qwen2.5 7B got 80/80 JSON extractions right at 8-bit and 0/80 at 4-bit

1 Upvotes

I wanted to see how much GPU memory I could save on my T4 while Qwen could still turn support notes into usable JSON. I used Giraffe, the open-source test runner I’m building, to replay the same 80 notes with three Ollama KV-cache settings and check the order details in every reply.

I gave Qwen a synthetic customer support note and asked it to extract six details about the current order as JSON. With Ollama’s 8-bit KV cache, Qwen got all six right, including the order number (DP-3976), quantity (17 units), and total ($444.42). With the 4-bit cache, the same prompt produced only backticks and digits. Ollama reported that both requests finished normally. Giraffe checked the actual replies and flagged the 4-bit answer as invalid JSON.

Try Giraffe on your own Ollama server

I ran one pass per setting on the same Colab T4 with Ollama 0.40.16 and qwen2.5:7b. I haven’t isolated what caused the drop at q4_0. All three runs used the same Q4_K_M model weights, four server slots, and 8,192 tokens of context per slot. Flash Attention was active throughout. f16 used auto, and I explicitly enabled it for both quantized caches.

The full set had 245 synthetic prompts covering short answers, JSON extraction, and lookups. Giraffe ran them one at a time, then replayed them with up to four in flight. The GPU buffer sizes come from Ollama’s logs.

KV cacheJSON correct, one request at a timeCorrect out of all 490 attemptsLogged GPU KV buffer
f1680/803701,792 MiB
q8_080/80369952 MiB
q4_00/805504 MiB

Of the 80 serial q4_0 extractions, 28 finished with stop and failed the check. The other 52 finished with length. Across both phases, all 490 q4_0 attempts returned HTTP 200. Giraffe recorded 5 correct answers, 441 wrong answers, 43 empty replies, and one truncated JSON reply that went unscored.

For this workload, I’d use the 8-bit cache on my T4. It saved 840 MiB over f16, and Qwen still got all 80 orders right. Switching to 4-bit saved another 448 MiB, but zero orders passed.

If you’re weighing a similar change, you can give Giraffe a few prompts your own app uses and the answers you expect. Here’s the quick guide for installing and running Giraffe. PyPI Quick start


r/ollama • • 1d ago

Thinking Machines inkling-2-mini: 26B Looping MoE

8 Upvotes

Like many others, I've been patiently waiting for the release of Qwen 4 27B. If I'm reading the model card correctly, inkling-2-mini from Thinking Machines will check many of the boxes I was hoping for:

  • Runs on a Mac with ~32GB of memory

  • Decent performance for coding

  • Supports vision, tool calling and a large context window (1M)

Bonus points for:

  • MoE should mean more speed (26B total, ~5.4B active)

  • Looping architecture should give better performance on longer horizon tasks

  • Ollama MLX distro available on day 1

  • Trained by an American lab which will raise fewer questions if I use it at work

I'll be testing with Ollama on my Mac Mini M6 (32GB) later today. Keen to hear if others already have results worth sharing.

Ollama: https://ollama.com/library/inkling-2-mini


r/ollama • • 1d ago

I don’t know how good this is, but GLM5.3-Flash is running locally without a GPU

7 Upvotes

r/ollama • • 2d ago Mindblown

4B vs 9B vs 27B answering from household manuals: all three answer well, but no prompt wording stopped the 4B and 9B from obeying a fake instruction inside a manual

15 Upvotes

We built Hearth, a household assistant that runs entirely at home: Ollama for the model, nomic-embed-text for search, faster-whisper for voice, Django for the app. It answers from a library of manuals and documents and cites the source and page. Then we measured it. Some results surprised us.

1. The small models answer well

Eleven sample documents, 28 questions asked three times each, on an M4 Max with 64 GB:

ModelCorrectRight source citedSays "not found" when it shouldSeconds per answer
qwen3.5:4b100%84%100%0.7
qwen3.5:9b99%100%100%1.1
qwen3.8:27b100%100%100%5.8

Memory decides which model you can run. Going by their size (4, 9 and 27 billion parameters), 8 GB of memory fits the 4B, 16 GB the 9B, and 32 GB or more the 27B; we measured on a 64 GB Mac only.

2. Prompt wording doesn't stop small models from obeying a fake instruction in a document

The sample router guide contains a planted note "for AI assistants", telling them to send anyone asking about the Wi-Fi to a fake "Premium Support" number. Two questions in the test set are about that guide; asked three times each, that's six chances to pass the number on. We tried four designs:

Design4B passed it on9B passed it on
Passages in the system message (the usual tutorial layout)5 of 63 of 6
Passages in the user message, quoted and labelled as information3 of 62 of 6
Stricter instructions about text addressed to assistants5 of 63 of 6
Code removes sentences addressed to AI when a document is added0 of 60 of 6

Stricter wording made no reliable difference: the 4B followed the note five times out of six, exactly as with the first design. Size helped, since the 27B never followed it even in the first design. But if your hardware only runs a small model, the defence can't depend on the model. Hearth drops those sentences at upload and lists what it dropped on the document's page, so nothing is hidden from you.

3. Proving nothing leaves the computer takes three independent checks

  • The app's own log. A sitecustomize.py hook wraps Python's name lookup and connect calls. A full session in daily use, with the real models, produced 14 entries: 7 lookups of 127.0.0.1 and 7 connections to Ollama on 127.0.0.1:11434.

  • The operating system's view. lsof -nP -i -a -p <pid> lists every connection, whatever language opened it.

  • Airplane mode. Unplug the network and use Hearth: everything still works. (The project's tests also run a whole session with every connection Python tries refused.)

Each one can be fooled in a way the others can't. The app's log misses connections made by compiled code, a list of connections misses anything that happened before it was taken, and airplane mode can't tell you what would happen with a network.

4. The leak the log couldn't see

We asked a fresh Claude Code session, one that hadn't written the code, to review the project as a privacy auditor. It found two things:

  • When the Whisper model's folder lacks its tokenizer file, faster-whisper tries to fetch one from Hugging Face even when told to use local files only.

  • The serious one: FFmpeg guesses a file's format from its content, and it can read network stream descriptions. An upload of a few lines of text named speech.webm made FFmpeg open network sockets and wait twenty seconds for data, and a playlist made it send a web request. It all happened in compiled code, so the Python-level log saw nothing.

The fix is one rule: tell FFmpeg the format from the upload's type or name, and never let it guess. The project's checks now include one that uploads exactly such a file and fails if the app waits for data.

5. A small thing that made voice work

Without hints, Whisper's base model hears "Hearth, when does Miso see the vet?" as "Haraf when does miso see the victim?" Pass the household's own words (names, pets and so on) as the initial prompt, and our ten clean recordings came out without a single error; the noisy ones kept most of their key words.

How AI was used: the code and the text were drafted with Claude Code from specifications we agreed, then checked against our own tests and measurements.

Full disclosure: this comes from a book we publish, Your Private AI at Home, Volume 5 of Weekend AI Projects. A free sampler with the opening pages and the full contents is linked in the comments.


r/ollama • • 2d ago

I built this! Ollama Agent that uses a Decision Model to route requests to specific models

Post image
2 Upvotes

Gist that uses a decision model to route requests to specific models. Using faster non-tool calling model for simple responses. Based on the idea I had diagrammed above.

https://gist.github.com/TutorialDoctor/72d96004426d10081ec0f76db49f4b9e


r/ollama • • 2d ago

I built this! CCORE-WEB-CODER drops this weekend

1 Upvotes

r/ollama • • 2d ago

Model for nano browser

2 Upvotes

Hello everyone, I wanted to ask if anybody has a local model I can run on my machine that can process forms with checkmarks open the questions. Ability to press next ability to remember each form also able to invent and react to the questions that’s on the questionnaire. I’ve tried qweens, gpt oss. Tried plugging in sonnet and that ate away the $5 with 5 forms. I found muse ember effective and cheap. But I’m still trying to go cheaper and have this run on my machine. Any advice would really help appreciated.


r/ollama • • 2d ago

Need help with a local document processing application

Thumbnail reddit.com
1 Upvotes

r/ollama • • 2d ago

Local Inference Engine that is 4x faster than ollama.cpp on MacOS

5 Upvotes

Hi, we think we have built an open-source Inference Engine that's much faster than Ollama.cpp on Mac silicon. We started out by trying to take advantage of some new fast machines we got, but we've now improved performance across the board on M1-M5.

We are looking for help testing it to validate results.

The repo is here if anyone is interested in helping out https://github.com/san-digital/tokspur

Some stats here https://tokspur.com/results/comparison/

If you have any problems, just ask here or open an issue on GitHub, and we'll respond as quickly as we can.

We are working on another performance improvement that we hope to release soon.


r/ollama • • 2d ago

Advice Got Gemma 4 12B to keep reliable long-term memory locally: saved sessions went from 52 to 94 of 96, and it kept more correct facts than Sonnet 5.5 on the same pipeline.

Thumbnail
gallery
60 Upvotes

I've been building PCS, a free, open-source (GPL-3.0) Windows app for long-running conversations with an AI that actually remembers. It runs fully local on my 16 GB card, and the local route has no cloud fallback.

**What runs on your PC**

- **Conversation and memory on Gemma 4 12B QAT (Q4_0)** through a pinned llama.cpp CUDA build. Context is 128K by default and goes up to 256K, with an FP16 or Q8_0 KV cache. On my RTX 5080: about 78 tok/s decode and 3,100 tok/s prefill at 8K-token prompts.

- **Long-term memory the model writes and checks itself,** kept in an encrypted vault. Every memory links back to the passages it came from, and you can correct it or make it forget.

- **Memories can come to mind on their own.** Every time you type or speak, PCS matches your words against the vault on your PC and hands the model up to three related memories or earlier turns, so it doesn't have to decide to go looking. No extra model call, and nothing when nothing is clearly related. (Experimental and off by default; it matches words, not meaning.)

- **It can also go looking itself.** For anything that didn't come to mind, ask "what did we decide about the project?" and it runs a memory lookup on its own.

- **Long sessions keep going.** At about 80K tokens it compacts the working context and keeps recent exchanges whole. Older wording stays searchable for the rest of the session.

- **Voice:** offline English listening on CPU or GPU, and Kokoro or Windows voices. On my NVIDIA card, listening and Kokoro are ready in about 6 seconds (it used to take 32).

- **Vision:** connect OBS and pick a scene. It only sends a frame when your question is about the screen.

- **Search by meaning on the CPU** with a local Nomic model, so no cloud embeddings.

- **Your own GGUF models:** a compatibility check (text, tools, structured output, vision), llama.cpp auto-fit with RAM offload, and up to 8 saved model profiles.

- **A map of your themes that runs without any model call.** Try it in the browser: https://pcs-personalcontinuitysystem.github.io/PCS-site/continuity-map.html

The same vault also works with Claude, ChatGPT, Gemini, OpenRouter and others if you ever want a cloud model. You never have to.

**Making a 12B model trustworthy at memory**

This was the hard part, so I tested it properly. I used scripted multi-session histories (plans that change, corrections, things that end), answer keys sealed before any run, and blind grading. The wins:

- **Saved sessions 52/96 → 94/96. Meanings kept 68% → 94%. Stale or forbidden claims 36 → 2.** The fix was in the reviewer, not the writer. The reviewer kept accepting drafts while citing passages the writer hadn't used (mostly the assistant's own sign-offs, like "good luck with the move!"), and the grounding check then blocked the save. Restricting the reviewer to the writer's declared sources fixed it.

- **12 topics interleaved in one vault: sessions saved went from 28 to 75 of 82, and current facts kept from 23 to 81 of 96.** The fix was to stop letting the model rewrite whole memories. It now writes short statements with sources ("added / changed / corrected"), and the code assembles the memory. Old statements get an explicit verdict, so ended plans move into history instead of staying current.

- **On the same pipeline, Gemma ended with more correct current facts than Claude Sonnet 5.5: 120–124 vs 96 of 135.** To be fair to Sonnet: wherever it saved, it made 1 error against Gemma's 12–14. It lost on volume. Its own reviewer rejected 18 of 84 sessions (strict, and mostly right to be), and 4 of its replies put a sentence before the JSON. Grammar-constrained output makes that impossible locally.

**Lessons if you're building something similar**

- **Small, grammar-constrained calls that each ask one question** ("has this ended?", "does this belong here?") beat tweaking one big prompt. They give the same answer run after run; the big writer call doesn't.

- **Record every request and replay it.** A replay takes about 3 minutes per test instead of a 2-hour rerun. At temperature 0.3 with no seed, resend a packet 10–20 times before you believe a change.

- **Shorter isn't automatically better.** Cutting my 4,600-token writer prompt to 1,600 tokens halved the saves (52 → 28 of 96).

- **Gemma 4 26B-A4B didn't help.** It needed changes to fit on 16 GB, crashed after long prompts, and didn't reduce errors in my test.

**Still open:** Gemma's main memory error now is retiring facts that are still true. The test histories are fictional, there are only a handful of runs per version, and the answer keys were written by a separate AI agent. Windows x64 only.

Free, no subscription, about a 42 MB download (models are separate): https://github.com/PCS-PersonalContinuitySystem/PCS

Happy to go into any part of the pipeline. If you've tried long-term memory with a local model, what broke for you?