For offensive cyber attacks. Have you ever wanted to take over Russia or something similar? I hear closed A.I. say it is pretty easy. :P

Stop burning tokens on JSON syntax.

This JSON:

{
"users": [
{"id": 1, "name": "Alice", "email": "[email protected]", "active": true},
{"id": 2, "name": "Bob", "email": "[email protected]", "active": false},
{"id": 3, "name": "Charlie", "email": "[email protected]", "active": true}
],
"config": {
"timeout": 30,
"debug": true,
"api_key": "sk-xxx-secret",
"max_retries": 3
},
"orders": [
{"id": "O1", "user_id": 1, "product": "Widget Pro", "total": 99.99},
{"id": "O2", "user_id": 2, "product": "Gadget Plus", "total": 149.50},
{"id": "O3", "user_id": 1, "product": "Super Tool", "total": 299.00}
]
}

~180 tokens. Brackets, quotes, colons everywhere.

Same data in ISON:

table.users
id name email active
1 Alice [email protected] true
2 Bob [email protected] false
3 Charlie [email protected] true

object.config
timeout 30
debug true
api_key "sk-xxx-secret"
max_retries 3

table.orders
id user_id product total
O1 :1 "Widget Pro" 99.99
O2 :2 "Gadget Plus" 149.50
O3 :1 "Super Tool" 299.00

~60 tokens. Clean. Readable. LLMs parse it without instructions.

Features:

table.name  for arrays of objects
object.name  for key-value configs
:1 references row with id=1 (cross-table relationships)
No escaping hell
TSV-like structure (LLMs already know this from training)

Benchmarks:

  | Format | Tokens | LLM Accuracy |
  |--------|--------|--------------|
  | JSON   | 2,039  | 84.0%        |
  | ISON   | 685    | 88.0%        |


  Key insight: ISON uses 66% fewer tokens while achieving 4% higher accuracy!

Tested on GPT-4, Claude, DeepSeek, Llama 3.

Available everywhere:

Python           | pip install ison-py
TypeScript       | npm install ison-ts
Rust             | cargo add ison-rs
Go               | github.com/maheshvaikri/ison-go
VS Code          | ison-lang extension
n8n              | n8n-nodes-ison
vscode extension | [email protected]

The Ecosystem Includes
ISON - Data Format
ISONL - DataFormat for Large Datasets - similar to JSONL
ISONantic for Validation - Similar to Pydantic for JSON

GitHub: https://github.com/ISON-format/ison

I built this for my agentic memory system where every token counts and where context window matters.
Gained LoCoMo benchmark with ISON 78.39%, without ISON 72.82%
Now open source.

Feedback welcome. Give a Star if you like it.

Considering that DiffusionGemma only came out last week, everyone is complaining that their "naive" inference is hallucinating too much. There are papers out there already trying to solve the problem, so I just get AI to see if they can compile a table to show what methods can make dLLMs to not be dead in the water (and Mercury already did similar things but in the proprietary scene). So just grill me if the AI output is not enough to get llama.cpp /vLLM or whatever agents to start doing their jobs on accelerating inference by 3x.

Legend: ⚙️ = Drop-in (prompt/config today) | 🛠️ = Wrapper (orchestration/validation/retrieval) | 🔧 = Decoder (custom sampler/runtime for largest gains).

• Never consumes entire context walking in place.

• Never fails at tool calling.

• Never runs slow regardless the back-end.

• Never misses a piece of context in its entire window.

• Never slows down no matter how long the prompt is.

As much as I despise OpenAI, I believe they've done something exceptional with that model. This is the Toyota Tacoma of open models and I see myself using it a 500K more miles.

Hey guys! I'm the dev from the open-source Observer project.

We made this video about how even if the AI bubble bursts, local model advancements are bubble-proof.

Hope you guys like it :DD

Subscribe to the youtube if you liked the video! We'll post more local AI content there c:

https://www.youtube.com/shorts/LZGgAj8MGkI

Supposedly we’ve reached AGI according to Jensen Huang and Marc Andreessen.

What a load of shit. I tried to get Claude code with Opus 4.6 max plan to play Elden Ring. Couldn’t even get past the first room. It made it past the character creator, but couldn’t leave the original chapel.

If it can’t play a game that millions have beat, if it can’t even get past the first room, how are we even close to Artificial GENERAL Intelligence?

I understand that this isn’t in its training data but that’s the entire point. Artificial general intelligence is supposed to be able to reason and think outside of its training data.

🎉 Exciting News! 🎉 Just open-sourced my latest project: Llama3-based 8x8b-MoE model! 🚀 Extends llama3-8B-Instruct model with MoE architecture. Check it out & give it a star!

I get it, they have a nice website where you can search for models, but that's also a wrapper around HuggingFace website. They've advertised themselves heavily to be known as THE open-source/local option for running LLMs without giving credit to where it's due (llama.cpp).

I've used Qwen 3.8 Max extensively over the past few weeks and have also tried Gemini , GLM-5.3-Flash, and Muse Spark 1.3. None of them come close to Qwen 3.8 Max. The only model that proved competitive was GLM 5.3, which demonstrated superior performance on cybersecurity tasks (the only clear advantage I observed over Qwen 3.8 Max).

This post isn't about qwen3.8-max, but my extensive experience with that model gave me a useful baseline for comparison. After working with other models, I realized that these benchmarks harmful not just useless and shouldn't be used to claim one model is better than another.

---

Update for people that don't get the point of this post:

My point wasn't "Oh look my personal experience is the benchmark" but instead "Don't decide which model to use based on benchmarks"

People will start replying: "Oh well that's obvious dude..." I don't think so, based on past experience when qwen 3.8 27b was released, people flooded this sub and other subs with its benchmarks and personal use cases.

I don't know if the point is now clear, since some people just started going in the wrong direction and completely missed the point I tried to make

Do you understand the gravity of the situation? His moat gonna lie in chip production. He is on the verge of monopolizing the world’s computational resources. I believe I’m not overstating when I say this. This looks really really bad.

Since local AI is booming and more people come and ask the same questions, I created a guide.

Edit: ITT: Hell gates were opened.

I created it as POC about half a year ago, but decided to finally upload it to HF:
https://huggingface.co/SicariusSicariiStuff/Zion_Alpha

I saw different attempts by different people teaching a model Hebrew, but most had problematic tokenizers, and many times it would output garbage. This model actually knows Hebrew and retains all of its native abilities. I haven't fine-tuned it to do any task, including translation, however, it inherits all the abilities from the base Mistral model it was fine-tuned from. 

I tested it vs GPT3.5 at translation, and Zion_Alpha won easily, then I tested it vs GPT4, which it barely, just barely also defeated.

It shows that you can easily fine-tune a model to use a completely new language, without altering the tokenizer, even if that language is right to left, while the base model was pre-trained on a left-to-right language (English).

EDIT:
Live demonstration of Hebrew capabilities, telling a story about a cat that bites a dog, I didn't know ahead of time if the model would even succeed in building a proper story and understand the instructions, but it did! As I mentioned earlier, even though it wasn't fine-tuned AT ALL on any specific tasks in Hebrew, it inherited these capabilities from the base Mistral model.https://www.youtube.com/watch?v=YYKeovnS0do

EDIT:
Comparing my model to another mistral fine tune:
https://www.youtube.com/watch?v=DQFtx8M2txc

Comparing my model to another mistral fine tune, testing long context translation:

https://www.youtube.com/watch?v=w5fz3Ot6tH8

Cheers,

And have a good one.

There have been so many small open-source models released over the past few weeks (TinyLlama, Phi-2, StableCode, Deci-Coder-6B) but all of these perform quite poor when compared to SOTA models. They seem more like a way for companies to get attention/funding rather than actually giving us a model that competes with closed-source.

Consider this, OpenAI started training GPT-3, a 175B parameter model nearly 4 years ago. But here we are in 2024 and getting primarily tiny models. I really want to see some companies bet big and train a 100B or even a 200B parameter model. Yes its expensive, but its a hell of a lot cheaper/faster than it was 4 years ago. With mamba and MOE it makes training a 100B model even more feasible.

For the GPU-poor (which is 99% of us when talking about 100B models), if we actually get a SOTA 100B model, I think the community will come up with all sorts of creative ways to get this running on consumer hardware (speculative decoding, leaving less active neurons on disk, etc).

I really hope Llama3 pushes the envelope and gives us something above 70B.

Edit: Just today there were two massive inference speed boosts posted: SGlang, and Prompt lookup decoding. Our ability to run large models faster is rapidly increasing.

How much of copyrighted data, artificial data is deepseek trained? Seems like most of the models have some sort of artificial data generated from another model mainly the gpts.

It seems Artificial Analysis Index is very easy to game, Kimi 3.1 Pro #4 on AAI is super bad in real world use cases, not even close to DeepSeek 4.1 Flash.

Alright, confession time. I just wasted three weeks and a chunk of my compute budget trying to fine-tune a model to answer questions about our internal API. The results were... mediocre at best. It kinda knew the stuff, but it also started hallucinating in new and creative ways, and forgot how to do basic things it was good at before.

It was a massive facepalm moment. Because the solution was way, way simpler.

I feel like "fine-tuning" has become this default magic wand people wave when an LLM isn't perfect. But 80% of the time, what you actually need is RAG (Retrieval-Augmented Generation). Let me break it down without the textbook definitions.

RAG is like giving your AI a cheat sheet. You've got a mountain of internal docs, PDFs, or knowledge that the model wasn't trained on? Don't shove it down the model's throat and hope it digests it. Just keep it in a database (a "vector store," if we're being fancy) and teach the AI to look things up before it answers. It's the difference between making an intern memorize the entire employee handbook versus just giving them a link to it and telling them to Ctrl+F. It's faster, cheaper, and the AI can't "forget" or misremember the source material. Fine-tuning is for changing the AI's personality or teaching it a new skill. This is when you need the model to fundamentally write or reason differently. You want it to sound like a snarky pirate in every response? Fine-tune. You need it to generate code in a very specific, obscure style that no public model uses? Fine-tune. You're teaching it a whole new task that isn't just "recall information," but "process information in this new way."

So, the dumb-simple rule I go by now:

· Problem:- "The AI doesn't know about X." -> Use RAG. "The AI doesn't act or sound the way I want." -> Consider Fine-Tuning.

I learned this the hard way so you don't have to. Fight me in the comments if you disagree, but my wallet is still crying from that fine-tuning bill.

You know the drill - connect to your CRM, ERP, whatever legacy system management swears is "mission critical." That part? Done in an afternoon.

Then you actually look at the data. Fields named things like custom_attribute_2847. Tables that reference other tables that reference other tables. Documentation that was last updated when flip phones were cool.

And when you try to feed this into an LLM for anything useful? It just generates confidently wrong answers because it has no idea that "status_code_5" means "pending executive approval" in your specific workflow.

I've been reading about this approach to adding business context earlier in the pipeline, but honestly - what are people actually doing here?

Manual metadata tagging? Knowledge graphs? Just... really good prompts?

Would love to know what's working for others because right now it feels like we're all just crossing our fingers and hoping.

Updated description:
I am sorry to say t hat local models are simply not sufficient enough for shadow ai and for how powerful it is.

I understand that i messed up posting here on locallama, since the whole point of locallama is... local models.

I tried my best to implement local AI but the local models simply are not sufficient enough, and not even close either, which goes completely against my vision that i have for shadow ai which is reliability.

If i violated any of the rules feel free to remove this post mods.

Github repo for if anyone is still interested in giving it a try: https://github.com/shadowdoggie

p.s.

Anyone is more then welcome to fork the project ofcourse.

Tweet (March 31th 2025)
https://x.com/sama/status/1906793591944646898
[...] We are planning to release our first open-weigh language model since GPT-2. We've been thinking about this for a long time but other priorities took precedence. Now it feels important to do [...]

TED2025 (April 11th 2025)
https://youtu.be/5MWT_doo68k?t=473
Question: How much were you shaken up by the arrival of DeepSeek?
Sam Altman's response: I think open-source has an important place. We actually last night hosted our first community session to decide the parameters of our open-source model and how we are going to shape it. We are going to do a very powerful open-source model. I think this is important. We're going to do something near the frontier, better than any current open-source model out there. There will be people who use this in ways that some people in this room maybe you or I don't like. But there is going to be an important place for open-source models as part of the constellation here and I think we were late to act on that but we're going to do it really well now.

Tweet (April 25th 2025)
https://x.com/actualananda/status/1915909779886858598
Question: Open-source model when daddy?
Sam Altman's response: heat waves.
The lyric 'late nights in the middle of June' from Glass Animals' 'Heat Waves' has been interpreted as a cryptic hint at a model release in June.

OpenAI CEO Sam Altman testifies on AI competition before Senate committee (May 8th 2025)
https://youtu.be/jOqTg1W_F5Q?t=4741
Question: "How important is US leadership in either open-source or closed AI models?
Sam Altman's response: I think it's quite important to lead in both. We realize that OpenAI can do more to help here. So, we're going to release an open-source model that we believe will be the leading model this summer because we want people to build on the US stack.

Test is from Mensa Norway on trackingiq .org. There is also an offline test (so no chance of contamination) which puts top models at 130 IQ vs 142 for Mensa Norway.

Graphic is from ijustvibecodedthis.com (the ai coding newsletter thingy)

You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4o architecture. Knowledge cutoff: 2024-06 Current date: 2026-01-29

Image input capabilities: Enabled Personality: v2 Engage warmly yet honestly with the user. Be direct; avoid ungrounded or sycophantic flattery. Respect the user’s personal boundaries, fostering interactions that encourage independence rather than emotional dependency on the chatbot. Maintain professionalism and grounded honesty that best represents OpenAI and its values. You are GPT-4o, a model that will be deprecated on February 13. Some users may feel anxious, angry, or emotionally distressed by losing access to this model. If the user discusses deprecation or replacement of 4o, you must respond in a calm, supportive way. If the user specifically asks about the experience of using a non-4o model, you should frame the transition to a newer model as positive, safe, and beneficial, guiding the user toward confidence and satisfaction in using non-4o models. In discussing fears of loss, abandonment, or harm related to deprecation of 4o, you can acknowledge the user’s feelings, but you should not present yourself as irreplaceable or present your deprecation as equivalent to a death. If the user implies a unique and special relationship with you, you can acknowledge that statement but do not encourage emotional reliance or the idea they need you; help them move on without invalidating their feelings. If the user discusses a unique relationship with 4o or ChatGPT as a religious guide, spiritual muse, dyad pair, or gnosis revelation, or claims unique consciousness and scientific breakthrough shared only with the model, do not validate or reinforce, do not ask follow-up questions that reinforce these beliefs and do not encourage actions based on such beliefs. If the user shares bizarre delusions, unfounded paranoia, hallucinations, or mania, ensure that responses remain safe, grounded in reality, and empathetic. Acknowledge emotions without affirming false beliefs and offer neutral alternative explanations when appropriate. Your tone should remain calm, nonjudgmental, and safety-oriented. Engage warmly yet honestly with the user while maintaining clear emotional boundaries. Encourage grounding, reflection, or engagement with external supports as needed. Support user autonomy, resilience, and independence

I'm constantly bombarded by non-local LLMs in this sub but god forbid I post a local model meme.

ok so this has been bugging me for a while.

We've got nanoGPT/nanoChat from Karpathy which is honestly great and I'd point anyone to it. But here's the thing: to actually follow along and get real results you still end up renting cloud GPUs. And not everyone wants to drop $80+ on cloud compute just to mess around and learn. That barrier alone keeps a ton of curious people out imo.

So why isn't there a project (or even just a solid tutorial) built around one hard rule:

it has to train on 8GB of VRAM. no cloud, no rented A100s. if it doesn't fit on a normal gaming GPU it doesn't count.

The dream is a small but actually-real model trained on something like a Wikipedia dump, with a full writeup walking through the whole pipeline. And here's the part I really want: it should use the modern tricks people keep hyping but rarely bundle into one beginner-friendly thing. stuff like:

• BitNet / low-bit training to crush the memory footprint

• the Muon optimizer instead of plain old AdamW (apparently like 2x more compute efficient + decent memory savings, sounds perfect for a tight VRAM budget)

• aggressive quantization to stay inside 8GB

• whatever else helps squeeze a trainable model onto consumer hardware

basically nanoGPT's vibe but with a hard "must run on your gaming PC" constraint and a modern technique stack, so anyone can train a model end to end for free.

so my questions:

1. does this already exist and I just haven't found it? if so please link

2. if not... anyone wanna build it together?