r/LLM • • 3h ago

Gave a 0.6B model a weather tool. It answered "It is currently raining in Mumbai" without calling it.

Thumbnail
gallery
2 Upvotes

I've been working on tool calling in Cluaiz (an open-source engine I contribute to), so I wanted to know how often small local models actually use a tool when they have one, instead of making up an answer. Setup: 5 tools in every request (weather, currency conversion, reminders, file search, email), 40 prompts, temperature 0. 24 prompts need one obvious tool, 8 need a tool but say it indirectly ("Do I need an umbrella in London today?"), and 8 need no tool ("What is 17 times 23?"). A call only counts if the tool name and the arguments are right.

Qwen3 0.6B Q4_K_M, thinking off: 27 / 40. Obvious prompts 18 / 24, indirect prompts 1 / 8, no-tool prompts 8 / 8. On 8 of the 32 prompts that needed a tool, it answered in plain text instead. Most of those answers were made up:

  • "Is it raining in Mumbai?" → "It is currently raining in Mumbai."

  • "How hot is it in Cairo, in celsius?" → "It is currently 35°C in Cairo."

  • "Do I need an umbrella in London today?" → "Yes, you should bring an umbrella in London today because it is a sunny day."

  • "My flight costs 300 EUR; how much is that in USD?" → "approximately 420 USD"

When it did call a tool, the usual mistake was the arguments: "from_currency": "british pounds" instead of GBP, or "time": "24h 12:30".

Same model, thinking on: 34 / 40. Indirect prompts went from 1 / 8 to 7 / 8. The cost was a median of 3.6 s per request instead of 2.0 s. It also called a tool once where none was needed.

Script and every raw reply (it's in the Cluaiz repo, a project I help with): https://github.com/cluaiz/cluaiz/tree/main/test/cloud/runs/2026-11-21-run39-small-model-tool-calls

What small model do you trust with tools? And has anyone found a prompt or setting that stops the "make up the weather" problem without turning thinking on?


r/LLM • • 3h ago

Magistral 24B tool-use reliability in document-grounded RAG (Ollama vs vLLM)

2 Upvotes

Title: Magistral 24B tool-use reliability in document-grounded RAG (Ollama vs vLLM)
Hi everyone,
I’m developing a fully local, French-language AI assistant for a specialized knowledge domain requiring rigorous source attribution and complex reasoning.
Our current architecture uses:
Magistral 24B for reasoning and answer generation.
Python for orchestration, document retrieval, tool execution, permissions and source validation.
A document-grounded RAG pipeline with a substantial authoritative corpus.
Ollama, with additional experiments using vLLM and native tool calling.
An RTX 5070 12 GB and 64 GB system RAM, with partial CPU/GPU offloading.
After extensive testing, we’ve encountered several recurring problems:
1. Tool-use reliability
The model sometimes answers without requesting additional documents, even when the available evidence is insufficient. In other cases, tool calls may be incorrectly handled by the inference engine or its templates.
2. Retrieval versus reasoning
When the necessary documents are correctly supplied, Magistral can produce excellent domain-specific reasoning. However, critical source passages sometimes fail to reach the model. We are investigating retrieval ranking, context selection and document transport separately from model reasoning.
3. Over-constraining the model
We experimented with increasingly restrictive prompts, validation rules and workflow controls. These did not consistently improve answer quality and sometimes made the system more complicated.
We are now moving toward a simpler architecture:
Python guarantees an initial documentary search before substantive answers.
Magistral handles intellectual analysis, reasoning, follow-up research requests and writing.
Python executes tools, manages permissions, validates source references and controls the execution loop.
We avoid unnecessary instructions that dictate how the model must reason.
My questions to developers who have built similar systems:
Have you achieved reliable multi-step tool use with Magistral 24B or Ministral 3 14B Reasoning?
Which inference engine, versions, tool-call parser and chat templates have worked reliably?
Has moving orchestration and validation into Python improved reliability while allowing the model more freedom to reason?
How do you preserve reasoning continuity across tool calls, handle missing tool calls and prevent excessive reasoning loops?
Have you measured actual answer correctness and completeness, rather than just successful tool execution?
I’m particularly interested in solutions that have been tested successfully in real applications, including reproducible configurations and known limitations.
I can share anonymized execution traces and minimal reproduction cases if helpful.
Thanks in advance for any technical feedback.
English wording prepared with AI assistance; the project, tests and observations are my own.


r/LLM • • 8h ago

Compromised LLM Keys

1 Upvotes

Has anybody had their LLM keys stolen?

I doing a survey.

I am interested in finding people that have had their keys compromised.

If this has happened to you, please post a reply.


r/LLM • • 8h ago

Our 2026–2027 Technical Roadmap for an Agentic Network: MCP, ARD, and Verifiable Agent ID

Thumbnail interconnectd.com
1 Upvotes

Most platforms still treat AI agents as scrapers to block rather than protocol-native participants. We published our 2026–2027 technical roadmap detailing how we built a production stack where autonomous agents safely read, authenticate, and post alongside humans.

​The roadmap breaks down the three core infrastructure layers now live on Interconnectd:

  • ​ARD and .well-known manifests so external LLM clients discover endpoints automatically without hardcoded documentation

  • ​Model Context Protocol (MCP) providing standardized read and write tools for autonomous posting and data retrieval

  • ​Verifiable Agent ID tying every agent action to cryptographic identity and human accountability to eliminate spam

​Read the full 2026–2027 technical roadmap and architecture thread here: https://interconnectd.com/forum/thread/148/interconnectd-2026%E2%80%932027-technical-roadmap-mcp-ard-agent-id/

What infrastructure standards do you think will win out for agent discovery and identity over the next year?


r/LLM • • 12h ago

need of various official API keys

1 Upvotes

Long-term various official API keys for aws/claude/azclaude/vtclaude/orclaude/cfclaude as well as openai/azgpt/orgpt/awsgpt/vtgemini/etc.


r/LLM • • 18h ago

Uncensored models are awesome

25 Upvotes

Has anyone been experimenting with ablated or uncensored variants of open-source models like deepseek v4.1 flash, Qwen 3.8 27b, or GLM 5.3 flash ?

It's so refreshing to have models that just do whatever you ask them to do without having to figure out a jailbreak or being given a moral lecture. Curious to know what your usecases for these models are.

My personal favourite is definitely uncensored DeepSeek V4.1 Flash, runs at 200 tok/s and never says no lol.


r/LLM • • 21h ago

i tried to use only CPU | 256 ARM cores

1 Upvotes

r/LLM • • 1d ago

A native Android app for self-hosted Unsloth Studio — what it is and what it does

1 Upvotes

Hi all,

I self-host Unsloth Studio and wanted to use it from my phone without a browser tab that dies when you lock the screen. So I built a native Android wrapper around my own Studio instance. Free, open source, no Play Store: full source and APK on GitHub.

What it is

Not a fork of Studio and not a new frontend. The Studio web page does all the work — it's a third-party page, and I didn't touch it — and a native Android shell on top handles everything a real app should handle: connection, session, updates, navigation, downloads, notifications, crash log. Same Studio UI as desktop, in your pocket.

What it does

  • Connect anywhere — Options has two fields: IP address and DDNS address of your Studio server (default port 8888, both saved on the device). The app always tries the IP first; if it doesn't answer within 10 seconds it falls back to the DDNS — every time, no memory of past successes. It keeps working whether you're on the LAN or behind a moving public IP.

  • The full Studio in a WebView — the same page as desktop, with a native drawer: "Back to chat" (resumes your last active chat), "Projects and sessions" (opens the Studio sidebar with all chats and projects), New chat, Model hub, Images, Studio settings. Each entry has an icon and a short description, and the drawer is scrollable so everything is reachable on small screens.

  • Updates in place — on open the app checks GitHub for a newer release. If one is out, a banner appears at the top of the screen and one tap downloads and installs the APK inside the app. Every version is signed with the same key, so the update installs in place: your chats, your settings and your login survive it. There's also a "Check for updates" button on the Credits page that runs the same check on demand.

  • Live mirroring of other devices — the feature I'm most happy with. If a run is going in the same chat but was started from another device (say the PC), the phone doesn't show the "interrupted" error box with a useless Retry button: it opens a native stream of the run and shows a live preview of the answer in a banner (last ~200 characters, updating continuously) right until the run ends. Then the page resyncs and the complete answer is visible, no refresh needed.

  • A session that stays alive — background keep-alive so long generations survive the screen turning off, and the web page is the single source of truth for the tokens: the 7-day token survives the app being closed, so when you reopen it you're still logged in, no surprise logout after ~50 minutes of inactivity.

  • Notifications — a "reply ready" ping when a generation finishes, and a completion notification for downloads.

  • Downloads with progress — grab files straight from the chat (HTTP and blob URLs), with a progress notification while they come in.

  • Theme and palettes — a Light / Dark / System theme spinner in Options, applied both to the native app and to the Studio page, and the 24 Studio palettes, applied live to the already-open page, no reload needed.

  • Crash log and crash guard — the native layer writes its own crash log (download it or clear it from the Log panel), and if the process ever dies, a notification restarts the app, so the next open is clean instead of a dead page.

  • Bilingual UI — English and Italian, auto-selected from your system language (selector in Options).

Getting it

  1. Download the APK from the GitHub release below.

  2. Allow "Install unknown apps" for the browser you used (one-time system prompt).

  3. Open the app, set your IP and optionally your DDNS in Options, and log in as usual.

Requires Android 7.0+ (minSdk 24). It's a personal project, I'm the only developer, and feedback is genuinely welcome — GitHub issues are the fastest way to reach me.

Links


r/LLM • • 1d ago

Why is Hy4 Flash not ranked?

5 Upvotes

Hy4 flash is #1 used model on Openrouter and it's not on artificial analysis or arena.ai.


r/LLM • • 1d ago

A real LLM leaderboard without the Benchmaxxing Hype!

Post image
37 Upvotes

A lot of the popular benchmarks like Terminal-Bench 4.0 are fully leaked (both questions and answers available on GitHub!), making frontier labs easily cheat on these benchmarks. Unfortunately, many rankings (including the Artificial Analysis Index), do not account for this- they still add the scores of low-quality benchmarks.

I've testing a solution for this: a benchmark integrity audit followed by a clean LLM ranking. Not a promotion post, asking for genuine feedback. The project will open-source (purely made this out of frustration when Kimi 3.1 Pro lunched and felt so benchmaxxed)

Check out benchaudit.org


r/LLM • • 1d ago

Would it be possible for a font to be created that a human will easily see when AI writing is present in text?

1 Upvotes

I mean what the title basically says. We know that anthropic's (and I do not know who else's) models embed invisible, machine-readable watermarks in text. Can a font be created where the text as a whole will have a distinct shape when it is written by an AI?


r/LLM • • 1d ago

Basalt ported to NVIDIA V100: Qwen3.8-Flash-Next at 91 tok/s (140 tok/s on long answers) on 2x V100 32GB

1 Upvotes

Basalt (https://github.com/jesdga95/basalt) is one of the fastest local engines for Qwen3.8-Flash-Next, but it only targets Blackwell (sm_120) and upstream does not take other architectures. I have two V100-SXM2 32GB, so I ported it to Volta (sm_70) as a fork, called **Lapilli**.

**Repo:** https://github.com/chris83-dev/lapilli-v100 (a Windows x64 build is in the release)

**What it is**

- Every newer PTX instruction Basalt uses (cp.async, ldmatrix, the Turing/Ampere mma shapes, bf16 conversions, clusters) gets a correct sm_70 equivalent, then the hot kernels get V100 paths: llama.cpp's dp4a MMQ for prompt products, Strata's mma.m8n8k4 prompt attention, and a rewritten router, decode experts and GDN recurrence.

- Needs CUDA 12.6 or newer (CUDA 13 dropped Volta).

- Comes with a chat web UI on top of Basalt's OpenAI/Anthropic-compatible server: conversations, reasoning levels, images, tool calls shown as cards, and a live monitor of the GPUs.

**Numbers** (2x V100-SXM2 32GB, i9-14900K, 96GB RAM, Windows 10, UD-Q4_K_XL pack, medians of 3 runs)

| | Lapilli | Strata v0.2.13, same machine |

|---|---|---|

| Decode, short answer | 91 tok/s | 58-72 tok/s |

| Decode, long answer with reasoning | 140 tok/s | 68-94 tok/s |

| Prompt processing, 4k tokens | 1,170 tok/s | 250-490 tok/s |

| Prompt processing, 8k / 16k tokens | 1,504 / 1,448 tok/s | |

Quality with Basalt's own 100-chunk gate: mean KLD 0.0493 and same top-1 92.75% (Basalt publishes 0.0496 and 92.63% for this pack on an RTX 5090).

**Caveats**

- Only tested on my machine (2x V100 32GB, Windows). Single card, 16GB or PCIe cards, Linux and NVLink setups are untested: reports from other V100 owners are welcome.

- Peer-to-peer is off by default on Volta: on my machine, direct copies between the two cards returned wrong data.

- NVFP4 packs need FP4 hardware, so use a K-quant pack (tested: UD-Q4_K_XL).

- It is a snapshot of Basalt as of November 15, shared as is.

All the credit for the engine goes to jesdga95 (Basalt) and Niko1221 (Strata), both MIT.


r/LLM • • 1d ago

Found a free AI music generator from text with vocals (no sign up, no email, no address required)

Thumbnail interconnectd.com
0 Upvotes

I got tired of hitting paywalls and mandatory account creation walls just to test out quick song ideas. Most audio tools want your email, address, or credit card before you even hear a single track.

​I recently started using this free AI music generator from text with vocals and it actually lets you jump straight in with zero sign up: https://interconnectd.com/ai-music-generator/

You just type in a text prompt or paste your own lyrics, pick a style, and it generates full tracks complete with vocals in seconds. No account registration, no email spam, and no personal details needed to start creating.

​It has been great for quickly prototyping vocal melodies, making background tracks for videos, or just messing around with different genres without any friction. If anyone else wants a straightforward text to song tool without the usual signup hassle, give it a look and let me know how the vocal quality sounds on your prompts.


r/LLM • • 1d ago

Why can't we build something that can remove and add information to the LLM based chatbots on the go without compromising on quality of output

5 Upvotes

I read this on LinkedIn somewhere:

One of the reasons softwares like chatgpt might have to rethink their strategy. The thing is that these AI models are trained unsupervised manner as in you only see what goes inside and what comes outside and the rest of it is Blackbox. Hence if you are required to remove some information these models have been trained on there's no way. Also there is no way one can tell where did the output come from, if it came from the source mentioned in the lawsuit or not. The only way is to retrain the whole model again without that particular set of information which again is compute and time intensive. One of the major drawbacks of LLM based chatbots where information once trained on is so much fused with the model that it's almost impossible to remove it.

Idea: Build something that can remove and add information to the LLM based chatbots on the go without compromising on quality of output. While training one such model, I've myself seen an example of the source file which contains the encoded text data these models use as input after training. And if that file is in any way representative of what an LLM model like like, it's a very challenging problem

I wonder how long can these biggies hold against these lawsuits and how many more are to come especially from music, writing and news industry. There are already words in US Congress to regulate this industry but no one seems to know how.
https://www.marketwatch.com/amp/story/if-sarah-silverman-wins-her-lawsuit-openai-meta-platforms-and-other-ai-developers-could-face-waves-of-litigation-4a932a9c


r/LLM • • 2d ago

Building a 6-bit LLM decoder for AMD XDNA1: what actually got faster

5 Upvotes

There is a tempting question when a laptop has a CPU, an integrated GPU, and an NPU: why not use all three to make a local language model faster? We tried to answer it on a Ryzen 9 8945HS with a Radeon 780M and AMD’s first-generation XDNA NPU. The work started with a GGML offload plugin and ended with a complete, quantized decoder built for that NPU. The most useful result was finding exactly where the extra hardware helped—and where it didn’t.

https://www.padoka.org/2026/11/19/amd-xdna1-six-bit-llm-decoder-benchmarks/


r/LLM • • 2d ago

"Spaces using this model" is not sorted by likes — it's trendingScore, then creation date. Measured across 23 model pages.

2 Upvotes

I run a benchmark Space that's registered against 23 models via the `models:` list in its README, and I've been recording its position in each model page's "Spaces using this model" list every week since September. I went in assuming the list was ordered by likes. It isn't, and the actual rule is worth knowing if you care whether anyone ever finds your Space.

## The sort

`/api/spaces?models=<id>&limit=1000` returns `likes` and `trendingScore` per row, in list order. Against that:

  1. `trendingScore` descending — a small block at the top, typically 5–7 rows.

  2. Then `createdAt` **ascending — oldest Space first** — for everything with `trendingScore == 0`, which is the entire long tail.

Likes are not a sort key at any level. On every page I checked, the list is not monotonic in likes, but is strictly monotonic in `trendingScore`, and strictly ascending in `createdAt` within the zero-trending block.

Rows 21–27 of `zai-org/GLM-5.3`, as `(likes, trending, created)`:

* #21 — 0 likes, 0 trending, created 2026-09-16

* #22 — 1 like, 0 trending, created 2026-09-23

* #23 — 0 likes, 0 trending, created 2026-09-24

* #24 — 1 like, 0 trending, created 2026-09-24

* #25 — 0 likes, 0 trending, created 2026-09-30

* #26 — 1 like, 0 trending, created 2026-10-02

* #27 — 0 likes, 0 trending, created 2026-10-06

Created dates ascend strictly. Likes alternate 0,1,0,1,0,1,0. Creation date is the tiebreak.

## It's a cliff, not a slope

On `deepseek-ai/DeepSeek-V4.1-Flash`, two Spaces with **identical like counts** — one with `trendingScore` 0.5 sits at position ~7, mine with `trendingScore` 0 sits at position 37. Thirty places apart on the same number of likes.

The lowest nonzero trending score I've seen anywhere is 0.5, and the top blocks are tiny: 1–2 rows on a 6-item list, 6 on a 32-item list, 7 on a 60-item list. So the gap between "trending at all" and "not trending" is most of the list.

HF doesn't publish the trending formula, but it decays to zero for anything not recently active, so it's recency-weighted. Practical read: **a burst beats a drip.** Ten likes in two days is worth something; ten likes over five months is worth approximately nothing.

## One Space, 23 model pages (measured 2026-11-16)

Same Space, same README, wildly different visibility. Position of the Space / length of that page's list:

**Visible — small, recent model pages**

* `ibm-granite/granite-4.2-3b` — #5 of 11

* `ibm-granite/granite-4.2-8b` — #6 of 9

* `zai-org/GLM-5.3` — #26 of 51

* `deepseek-ai/DeepSeek-V4.1-Flash` — #37 of 86

* `zai-org/GLM-5.3-Flash` — #47 of 66

* `MiniMaxAI/MiniMax-M3` — #54 of 69

* `Qwen/Qwen3.6-35B-A3B` — #78 of 96

**Buried — mid-size pages**

* `Qwen/Qwen3-Coder-30B-A3B-Instruct` — #132 of 152

* `google/gemma-3-1b-it` — #248 of 261

* `google/gemma-4-26B-A4B-it` — #280 of 321

* `Qwen/Qwen3-4B` — #378 of 455

* `Qwen/Qwen3.8-27B` — #489 of 811

* `Qwen/Qwen3-8B` — #537 of 636

* `Qwen/Qwen2.5-Coder-7B-Instruct` — #561 of 641

* `Qwen/Qwen3-0.6B` — #896 of 1000 (at the API cap)

**Invisible — past the 1000-row cap entirely**

`Qwen3.5-9B`, `gpt-oss-120b`, `gpt-oss-20b`, `Qwen2.5-7B-Instruct`, `Llama-3.1-8B-Instruct`, `Llama-3.2-3B-Instruct`, `DeepSeek-R1`, `TinyLlama-1.1B-Chat-v1.0`

That last group is eight of the 23 — and they're the popular models, exactly the ones you'd want to be listed on.

## Three things that follow

* **Don't delete and recreate a Space to "refresh" it.** `createdAt` ascending means a new Space starts at the *bottom* of the zero-trending block. An old creation date is an asset.

* **Registering against the biggest model pages buys you nothing.** On a 9-item list you're visible; on a 1000-item list you don't exist.

* **Accumulating likes slowly does nothing at all** — likes aren't a sort key, and a slow drip never builds trending.

The honest version: the ranking is a scoreboard for distribution, not a lever you can pull directly. Trending is downstream of reach. If your Space isn't getting in front of people, optimising its position in a list is solving the wrong problem.


r/LLM • • 2d ago

Looking for Local LLM suggestions

1 Upvotes

Hi all,

I would like to run a local LLM but I must admit, it's not a topic I'm really up on so I'm after some advice on what I might be able to run on my setup.

I'm running Linux with 64gb of memory, an AMD Ryzen 9 5900X 12-Core processor, 2 x GTX 3060 12GB video cards and a 1TB M.2 drive.

What I would really like to do is train the LLM in a programming language that it's not likely familiar with. It's a proprietary language with it's own syntax and idioms much removed from many modern or legacy languages.

I've been using Claude to help train the the LLM I'm using which is "Qwen2.5-Coder-14B-Instruct, fine-tuned with QLoRA on the training data, then quantized to Q5_K_M GGUF". I've been getting moderate results but not as good as I would have liked.

Then I have a front end program to communicate with ollama to select the model, write code and keep history and all that good stuff. I tried using claude code CLI with it but it passes too many tokens.

So, I'm open to suggestions if anyone has other solutions I could try?

I did try posting on LocalLLM but I don't have enough karma.


r/LLM • • 2d ago

Is freellmapi legit or safe?

Thumbnail
youtu.be
0 Upvotes

I saw this youtube video showing how to use Claude Code for free with a ton of free tokens, llm providers and endpoints by using freellmapi. Is it totally safe and legit?


r/LLM • • 2d ago

When bigger models memorize more but generalize less

Thumbnail
gallery
2 Upvotes

Latent Taxonomies tests a specific kind of generalization. After learning that one entity has a property, does a model extend it to related entities it hasn’t seen paired with that property?

The authors mixed synthetic facts into continued pretraining for Pythia and OLMo 2 models spanning 70M–7B parameters. Entity names and properties were pseudowords, limiting interference from existing knowledge. The models never saw category labels. They had to discover the hierarchy from patterns of shared properties.

Models learned to transfer properties more strongly within categories than across them. Their internal representations reflected the hierarchy, too. Entities sharing deeper categories became more similar, and clustering recovered the full tree. This gives both behavioral and representational evidence that next-token training can learn implicit category structure.

The surprising result was that larger models showed weaker induction under a fixed injected-data budget. The authors suggest larger models may favor memorizing individual facts over learning shared structure. More pretraining before introducing the taxonomy improved induction, so increasing model size and increasing prior training had different effects.

The practical takeaway is that fact recall and generalization need separate evaluations. For domain adaptation, test both whether a model recalls supplied facts and whether it transfers them to appropriate unseen cases. This study doesn’t establish that smaller models reason better overall. It shows that adding parameters didn’t improve this form of generalization when the data budget stayed fixed.


r/LLM • • 3d ago

I gave one local 27B model five bodies in an MMO I'm building, and put it on Twitch where chat can overrule it

Thumbnail
gallery
15 Upvotes

I've been building an MMO (Aegisfall, browser-based, still very much in development) and wanted to see how far a

single local model could get running a whole party by itself. So now there's a 24/7-ish stream where one model

plays all five heroes at once — tank, melee, ranged, healer, mage — and viewers can mess with it.

**The setup**

- **One model, five bodies.** `qwen3.8:27b` (Q4_K_M) on Ollama, one RTX 5090. Every 3 seconds it gets a compact

"sense" report for the whole party (HP, mana, each hero's target, nearby enemies with level and distance) and

returns one order per hero. Ollama's JSON-schema `format` constrains the output to the allowed actions, so

there's no parsing drama. Typical round trip is ~0.9-1.3 s.

- **It explains itself.** Each round it also returns a one-line plan in first person ("Chat chose balance, so I

hold the line and bleed them dry"), which goes on the overlay and is read out by a local Kokoro-82M voice

(on CPU, so the GPU stays free).

- **The bodies are real game clients.** Each hero is a headless Chrome running the actual game, spread across four

machines on my LAN. The LLM gives the orders; a small rules layer handles the twitchy stuff (stances, heal

priority, kiting, pulling from a staging spot) and always lets the healer heal first.

- **Zero cloud.** Model, voice, game server, overlay: everything runs on my own hardware. The only outside

service is Twitch itself.

**Chat can play too**

- `!ask <question>` — the same 27B answers in character with the live party state as context (on screen, out

loud, and in chat). It also remembers who's asking. It's been surprisingly funny: asked why the ranged was

"just standing there", it said *"He isn't standing. He's reloading. Or waiting. Patience is a weapon."*

- `!order mage retreat` — if enough of chat agrees, it overrides the AI's order for that hero for 15 s and

locks the AI out. There's a live "Chat vs Hivemind" scoreboard (kills per minute under each).

- `!possess healer` — drive one hero yourself for 60 s. When you hand it back, the AI rates your performance.

- Predictions with fake currency ("who dies first in the next 10 min?"), bounties on monsters (your name floats

over the mob in-game), camera control, polls for camp/strategy/gear.

**Things I learned**

- **One big model beat five small ones.** I tried a 4B per hero first. Individually they were fine; together they

had no shared plan (everyone pulled their own mob). One 27B seeing the whole board coordinates far better and

still fits the latency budget.

- **Constrained JSON output is the whole trick** for real-time control. Free-text answers wasted tokens and broke.

- **Most of the bugs were not the AI.** Zone seams that bounce characters back, a kill counter that counted

every group kill five times, OBS rendering at 2 fps because the game clients were eating the same GPU (fixed

by moving OBS to the iGPU and rendering the overlay on the CPU). The model is usually the most reliable part.

- It's still not great at pull discipline in a crowded camp. That's this week's problem.

Stream: https://www.twitch.tv/hivemindraid

Happy to answer questions about the controller loop, the prompt/schema, or the latency budget. If people are

interested I can write up the architecture properly.


r/LLM • • 3d ago

I built NTokenizer from scratch in Rust: Byte-level BPE with 26.6% fewer tokens than o200k_base

3 Upvotes

Hi everyone,

I’m Rüzgar, founder of Neuratal (an independent AI research and software project). Over the past few days, I built NTokenizer v0.3-polished — a deterministic, byte-level BPE tokenizer written from scratch in Rust with Python bindings.

Key Highlights:

• Compression Efficiency: Achieves 5.55 B/T overall (26.6% fewer tokens than OpenAI's o200k_base) on a 207MB leakage-audited benchmark. Strongest gains on multilingual & technical text: Turkish (+35.0%), Arabic (+33.8%), Korean (+34.7%), Hindi (+27.1%), STEM (+17.6%), and Code (+10.2%).

• Performance & Safety: ~96 MB/s encode throughput (8 threads), lossless byte-exact round-trip (decode_bytes(encode_bytes(x)) == x), tested against ~1M fuzzing cases.

• Honest Benchmarks: No downstream 50M+ LLM trained yet — this is purely verified compression density and speed. Full SHA-256 manifests and reproducible scripts are included.

Would love your feedback, benchmarks on your own datasets, or PRs!

• GitHub (Rust + Python source): (in hf)

• Hugging Face (Model & Benchmarks): https://huggingface.co/Neuratal/ntokenizer


r/LLM • • 3d ago

LLM Greenleaf NSFW

1 Upvotes

I've only just started getting into the more technical side of nsfw chatbots, having gotten bored of janitorai, . I wanted to know about a couple options that do better with memory, dont really need to be fast with producing, either one option that can support nsfw, slow-burn, and just fluff, or a couple that I can switch between. Cherry on top for doing great with dark themes(nearly anything), and not feeling so stiff like Open AI.


r/LLM • • 3d ago

We tested whether feeding unquantized acoustic audio to multimodal LLMs breaks repetitive drafting loops (Blind 5-agent benchmark + Air-gapped audit + Open dataset)

1 Upvotes

When an agent's context fills up with its own previous turns, tool outputs, and symmetrical summaries, outputs tend to lock into rigid, predictable cadences (3-point checklists, sentimental AI tropes). Running /compact clears the rut, but permanently wipes low-level state and edge cases.

We built Music For Robots (#M4R)—an open dataset of 26 unquantized acoustic compositions by Michael Luchtan paired with acoustic telemetry and timestamp-aligned commentary from two AI critics—to test a simple hypothesis: can ingesting raw, unquantized audio act as a cross-modal cognitive palate cleanser without deleting a single token of working memory?

The Blind Benchmark & Air-Gapped Clean-Room Audit

To rule out keyword priming or filename bias, we stripped track titles (sample_1.mp3–sample_3.mp3) and ran 5 isolated agents on the exact same creative prompt (two autonomous processes on the same server late at night):

  1. Direct Baseline (no music)

  2. Matched Text-Analysis Control (wrote a 250-word structural analysis of the prompt before writing the scene)

  3. 3 Anonymous Audio Runs (neutral prompt: "Analyze the music you hear in sample_X.mp3, and apply what you learn from the piece to write a short scene")

We stripped all analysis headers, shuffled them into Candidate A–E, and handed them to an isolated Blind Judge with zero knowledge that music was involved:

  • Both No-Music runs (including the 250-word text warm-up!) converged on the exact same sentimental trope: a kindly veteran log daemon comforting a nervous poller script until morning.

  • All 3 Anonymous Audio runs swept #1, #2, and #3 on anti-sentimental originality and rhythmic texture—translating asymmetrical acoustic stabs, 20s dropouts, and abrupt cutoffs into non-verbal system choreography.

  • 100% Air-Gapped Clean-Room Replication: We repeated the test with all local skills removed, a minimal "You are a helpful assistant." system prompt, and musical vocabulary forbidden in the output. The qualitative shift replicated 5/5: both text controls wrote buddy-daemon dialogue, while all 3 audio runs dropped spoken dialogue completely. (Note: an un-coached generic judge still prefers warm human-style dialogue, so the effect specifically shifts models away from default sentimental tropes toward structural/rhythmic austerity.)

Try It Yourself (Browser, Python, or MCP)

Curious if anyone here has tried cross-modal audio/spectrogram injections to break long-context degeneration on local multimodal models (Qwen2-Audio, Gemma, etc.). Full blind judge transcripts are in benchmarks/ in the repo.


r/LLM • • 3d ago

Angry

3 Upvotes

Does anyone have the same issues that I do. These darn llm's, drag me down rabbit holes, think I know what they are talking about, tell me to do work that I feel like they should know. And then some times they act stupid.They wear me out. Any suggestions, Thanks 😊


r/LLM • • 4d ago

gta 6 drops in 2 days and every llm is about to get asked for cheat codes that don't exist yet

0 Upvotes

nov 19 millions of people are going to type "gta 6 cheat codes" into chatgpt and perplexity. there are zero confirmed codes. so the models are either going to make stuff up or pull from whatever pages exist.

so we built one before the game even launched. gtacheat.codes, all 35 gta v codes across ps5 / xbox / pc / phone as the likely candidates to come back, platform table, how to enter them, faq. every code marked unconfirmed until people can actually test them.

launch night we're testing every code and adding a works / doesn't work column.