r/OpenSourceeAI • • Mar 18 '26 Bravo

I bought 200$ claude code so you don't have to :)

Post image
669 Upvotes

I open-sourced what I built:

Free Tool: https://grape-root.vercel.app
Github Repo: https://github.com/kunal12203/Codex-CLI-Compact
Discord(debugging/feedback): https://discord.gg/xe7Hr5Dx

I’ve been using Claude Code heavily for the past few months and kept hitting the usage limit way faster than expected.

At first I thought: “okay, maybe my prompts are too big”

But then I started digging into token usage.

What I noticed

Even for simple questions like: “Why is auth flow depending on this file?”

Claude would:

  • grep across the repo

  • open multiple files

  • follow dependencies

  • re-read the same files again next turn

That single flow was costing ~20k–30k tokens.

And the worst part: Every follow-up → it does the same thing again.

I tried fixing it with claude.md

Spent a full day tuning instructions.

It helped… but:

  • still re-reads a lot

  • not reusable across projects

  • resets when switching repos

So it didn’t fix the root problem.

The actual issue:

Most token usage isn’t reasoning. It’s context reconstruction.
Claude keeps rediscovering the same code every turn.

So I built an free to use MCP tool GrapeRoot

Basically a layer between your repo and Claude.

Instead of letting Claude explore every time, it:

  • builds a graph of your code (functions, imports, relationships)

  • tracks what’s already been read

  • pre-loads only relevant files into the prompt

  • avoids re-reading the same stuff again

Results (my benchmarks)

Compared:

  • normal Claude

  • MCP/tool-based graph (my earlier version)

  • pre-injected context (current)

What I saw:

  • ~45% cheaper on average

  • up to 80–85% fewer tokens on complex tasks

  • fewer turns (less back-and-forth searching)

  • better answers on harder problems

Interesting part

I expected cost savings.

But, Starting with the right context actually improves answer quality.

Less searching → more reasoning.

Curious if others are seeing this too:

  • hitting limits faster than expected?

  • sessions feeling like they keep restarting?

  • annoyed by repeated repo scanning?

Would love to hear how others are dealing with this.


r/OpenSourceeAI • • May 09 '26 WOW Heartwarming Bravo

Build a modern LLM from scratch. Every line commented. Explained like we are five.

Thumbnail
github.com
408 Upvotes

r/OpenSourceeAI • • Apr 11 '26 Popcorn

I reduced my token usage by 178x in Claude Code!!

Post image
283 Upvotes

Okay so, I took the leaked Claude Code repo, around 14.3M tokens total. Queried a knowledge graph, got back ~80K tokens for that query!

14.3M / 80K ≈ 178x.

Nice. I have officially solved AI, now you can use 20$ claude for 178 times longer!!

Wait a min, JK hahah!
This is also basically how everyone is explaining “token efficiency” on the internet right now. Take total possible context, divide it by selectively retrieved context, add a big multiplier, and ship the post, boom!! your repo has multi thousands stars and you're famous between D**bas*es!!

Except that’s not how real systems behave. Claude isn't that stupid to explore 14.8M token repo and breaks it system by itself! Not only claude code, any AI tool!

Actual token usage is not just what you retrieve once. It’s input tokens, output tokens, cache reads, cache writes, tool calls, subprocesses. All of it counts. The “177x” style math ignores most of where tokens actually go.

And honestly, retrieval isn’t even the hard problem. Memory is. That's what i understand after working on this project for so long!

What happens 10 turns later when the same file is needed again? What survives auto-compact? What gets silently dropped as the session grows? Most tools solve retrieval and quietly assume memory will just work. But It doesn’t.

I’ve been working on this problem with a tool called Graperoot.

Instead of just fetching context, it tries to manage it. There are two layers:

  • a codebase graph (structure + relationships across the repo)

  • a live in-session action graph that tracks what was retrieved, what was actually used, and what should persist based on priority

So context is not just retrieved once and forgotten. It is tracked, reused, and protected from getting dropped when the session gets large.

Some numbers from testing on real repos like Medusa, Gitea, Kubernetes:

We benchmark against real workflows, not fake baselines.

Results

RepoFilesToken ReductionQuality Improvement
Medusa (TypeScript)1,57157%~75% better output
Sentry (Python)7,76253%Turns: 16.8 to 10.3
Twenty (TypeScript)~1,90050%+Consistent improvements
Enterprise repos1M+50 to 80%Tested at scale

Across repo sizes, average reduction is around 50 percent, with peaks up to 80 percent. This includes input, output, and cached tokens. No inflated numbers.

~50–60% average token reduction

up to ~85% on focused tasks

Not 178x. Just less misleading math. Better understand this!
(178x is at https://graperoot.dev/playground)

I’m pretty sure this still breaks on messy or highly dynamic codebases. Because claude is still smarter and as we are not to harness it with our tools, better give it access to tools in a smarter way!

Honestly, i wanted to know how the community thinks about this?

Open source Tool: https://github.com/kunal12203/Codex-CLI-Compact
Better installation steps at: https://graperoot.dev/#install
Join Discord for debugging/feedback: https://discord.gg/YwKdQATY2d

If you're enterprise and looking for customized infra, fill the form at https://graperoot.dev/enterprises


r/OpenSourceeAI • • Jun 30 '26 Bravo

Onklaud 5 : a fusion model pipeline matching Fable 5 at 1/100th the cost. 57% of tasks at $0. Open source.

Post image
224 Upvotes

We've spent the last few weeks building something that changed how we think about AI assisted coding.

The problem nobody talks about

Every AI coding tool works the same way: one model does everything. It generates code. Then it reviews its own code. Same brain. Same blind spots. Same biases.

This is insane. In real engineering, you never let a developer review their own pull request. It defeats the entire purpose of code review. Yet every AI assistant does exactly that — and we've all accepted it.

Worse: ~60% of coding tasks already have a stdlib solution. "Read a JSON file" is json.load(). It's been in Python since 2.6. But your AI assistant will happily generate 20 lines of custom code and charge you tokens for the privilege.

What we built

Onklaud 5 (https://github.com/KorroAi/onklaud-5) is a fusion pipeline. Not a model. 3 AI models (Kimi K2.7 + GLM 5.2 + DeepSeek V4 Pro) working through a structured 6 stage council, surrounded by 4 cost saving infrastructure layers.

The 3 models:

Kimi K2.7 (Moonshot AI): primary code generation. HumanEval 99.0

GLM 5.2 (Z.AI / Tsinghua): architecture design, independent code review, final arbitration. 1M context. Open weights.

DeepSeek V4 Pro: direct API engine for lightweight tasks. Significantly cheaper per token than going through OpenRouter. Handles simple work so Kimi and GLM only get called when needed.

The 4 cost saving layers (all $0, all offline):

  1. Ponytail Ladder checks if stdlib, native functions, or existing deps can solve it. 57% of tasks stop here. $0. Under 100ms.

  2. Immune Memory stores every failure pattern. Scans future tasks BEFORE code is written. 19 patterns, 50% detection, growing every session.

  3. Headroom provides 60 to 95% context compression. Prevents quality degradation in 50+ message sessions. Keeps the pipeline coherent when single model systems fall apart.

  4. Quality Gate scores output across 7 dimensions on a 10/10 scale. Broken code blocked before it ships.

The pipeline:

GLM designs architecture → Kimi generates code → BOTH independently review → disagreements trigger GLM arbitration → quality gate blocks anything below 10/10.

Measured results (2026-06-22, real hardware)

57.1% tasks resolved at $0 (35 real tasks, 3 languages, 95% CI)

100% syntax pass rate (deterministic, 14 files)

67.2% context reduction (Headroom)

96.7% pipeline test pass rate (29/30 tests)

Cost: literally cents for hours of iteration. We built 4 production systems with this and spent less than a coffee.

Full research paper with methodology and statistical analysis included in the repo.

Why this matters

The AI industry is obsessed with bigger models. But the real frontier isn't model size. It's architecture. Ensemble methods have been standard in ML for 20+ years. It's time coding assistants caught up.

Model agnostic. Swap models in and out. The pipeline, verification, immune memory, and quality gate stay intact.

https://github.com/KorroAi/onklaud-5

Research paper, benchmarks, demo video. All in the repo. python test_pipeline.py to verify everything.


r/OpenSourceeAI • • May 14 '26

After months of building in vain, a stranger made a YouTube video about our project & I cried a little

Post image
180 Upvotes

A few months ago I told my co-founder I wasn't sure if anyone would ever care about what we were building.

We started Dograh as an open-source voice AI platform. Alternative to the closed players like Vapi and Retell. We thought developers would want this. But for a long time, GitHub stars trickled in slowly. Discord stayed quiet. Some days I'd refresh the analytics dashboard hoping to see something move, and nothing would.

Today everything changed. 

Our stars started climbing fast and we couldn't figure out why. Then we looked at our homepage bot, which asks every new user where they heard about us. Almost all of them said YouTube. We searched and found a tutorial from BetterStack, posted an hour ago. They'd built something with Dograh, liked it enough to record a video, and put it out into the world. We had no idea it was coming. We've never spoken to them.

We just crossed 500 stars. I keep refreshing the signup graph because part of me still doesn't believe it.

If you're building something open source and the silence is getting to you, I just want to say: someone out there might already be using your project. They might be about to tell the world. Keep shipping.


r/OpenSourceeAI • • Aug 21 '26

Built a Harness for LLMs using locally-run Qwen

175 Upvotes

Sharing my harness for running local LLMs that I built using Qwen 3.x 27B (> 90% locally built).

Its free, no telemetry, and open-source. Works on Windows, Linux (sorry, no Mac yet).

I use it for coding + mixed workflows.

  • llama.cpp + whisper Server Manager. Can run LLMs here and use with OpenCode/Claude Code etc.

  • Built-in MCP Tools - Filesystem, web fetch, code graph, To-Dos, and more. Extensible by external MCPs.

  • Use Sub-agents to split & offload your tasks, use other conversations as source of information.

  • Review all AI messages using a second adversarial AI, and avoid potential pitfalls as per your rules.

  • Voice-chat with AI - dictate with speech and get answers by TTS - annotate and comment without leaving voice mode.

  • Use work-modes to change AI behavior between planning, building, researching, or reviewing. Fully customizable.

  • Custom-compile llama.cpp backends for your system, GPU-agnostic - works with CUDA/ROCm/Vulkan.

Website: https://warpdrv.ai (Docs coming soon)
GitHub: https://github.com/mikjee/warpdrv

Appreciate your feedback, (or stars). Thanks :)
And, yes - I used the harness to build the harness :D


r/OpenSourceeAI • • Aug 20 '25

I made a website to visualize machine learning algorithms + derive math from scratch

174 Upvotes

Check out the website: https://ml-visualized.com/

  1. Visualizes Machine Learning Algorithms Learning

  2. Interactive Notebooks using marimo and Project Jupyter

  3. Math from First-Principles using Numpy and Latex

  4. Fully Open-Sourced

Feel free to star the repo or contribute by making a pull request to https://github.com/gavinkhung/machine-learning-visualized

I would love to create a community. Please leave any questions below; I will happily respond.


r/OpenSourceeAI • • 9d ago Bravo

I made a free place to share open source stuff built with AI

Post image
151 Upvotes

hey all,

i've been working on whatwemade and could use a few more open source projects on there.

it's a site for sharing stuff built with AI, and i've recently added support for github repos.

you can just paste your repo link and it'll pull in the readme, license, stack, stars, recent commits, and whatever people need to run it themselves. there's also an option to show which coding agents you used and whether you're looking for contributors.

i've added an MCP server too, so you can search existing projects directly from your agent. figured that might be useful before starting yet another project from scratch.

there aren't many open source projects on there yet, so if you've got something on github, feel free to add it. it's free and takes about a minute (the mcp can also create the project).

also curious if there's anything else you'd want to see on a listing like this.


r/OpenSourceeAI • • Apr 26 '25

I think I am going to move back to coding without AI

149 Upvotes

The problem with AI coding tools like Cursor, Windsurf, etc, is that they generate overly complex code for simple tasks. Instead of speeding you up, you waste time understanding and fixing bugs. Ask AI to fix its mess? Good luck because the hallucinations make it worse. These tools are far from reliable. Nerfed and untameable, for now.


r/OpenSourceeAI • • Mar 19 '26

Open-source models are production-ready — here's the data (5 models × 5 benchmarks vs Claude Opus 4.6 and GPT-5.4)

132 Upvotes

I've been running open-source models in production and finally sat down to do a proper side-by-side comparison. I picked 3 open-source models and 2 proprietary — the same 5 in every benchmark, no cherry-picking.

Open-source: DeepSeek V3.2, DeepSeek R1, Kimi K2.5 Proprietary: Claude Opus 4.6, GPT-5.4

Here's what the numbers say.


Code: SWE-bench Verified (% resolved)

ModelScore
Claude Opus 4.680.8%
GPT-5.4~80.0%
Kimi K2.576.8%
DeepSeek V3.273.0%
DeepSeek R157.6%

Proprietary wins. Opus and GPT-5.4 lead at ~80%. Kimi is 4 points behind. R1 is a reasoning model, not optimized for code.


Reasoning: Humanity's Last Exam (%)

ModelScore
Kimi K2.5 *50.2%
DeepSeek R150.2%
GPT-5.441.6%
Claude Opus 4.640.0%
DeepSeek V3.239.3%

Open-source wins decisively. R1 hits 50.2% with pure chain-of-thought reasoning. Kimi matches it with tool-use enabled (*without tools: 31.5%). Both beat Opus by 10+ points.


Knowledge: MMLU-Pro (%)

ModelScore
GPT-5.488.5%
Kimi K2.587.1%
DeepSeek V3.285.0%
DeepSeek R184.0%
Claude Opus 4.682.0%

GPT-5.4 leads narrowly but all three open-source models beat Opus. Total spread is only 6.5 points — this benchmark is nearly saturated.


Speed: output tokens per second

Modeltok/s
Kimi K2.5334
GPT-5.4~78
DeepSeek V3.2~60
Claude Opus 4.646
DeepSeek R1~30

Kimi at 334 tok/s is 4x faster than GPT-5.4 and 7x faster than Opus. R1 is slowest (expected — reasoning tokens).


Latency: time to first token

ModelTTFT
Kimi K2.50.31s
GPT-5.4~0.95s
DeepSeek V3.21.18s
DeepSeek R1~2.0s
Claude Opus 4.62.48s

Kimi responds 8x faster than Opus. Even V3.2 beats both proprietary models.


The scorecard

MetricWinnerBest open-sourceBest proprietaryGap
Code (SWE)Opus 4.6Kimi 76.8%Opus 80.8%-4 pts
Reasoning (HLE)R1R1 50.2%GPT-5.4 41.6%+8.6 pts
Knowledge (MMLU)GPT-5.4Kimi 87.1%GPT-5.4 88.5%-1.4 pts
SpeedKimi334 t/sGPT-5.4 78 t/s4.3x faster
LatencyKimi0.31sGPT-5.4 0.95s3x faster

Open-source wins 3 out of 5. Proprietary leads Code (by 4 pts) and Knowledge (by 1.4 pts). Open-source leads Reasoning (+8.6 pts), Speed (4.3x), and Latency (3x).

Kimi K2.5 is top-2 on every single metric.

Note: Kimi K2.5's HLE score (50.2%) uses tool-augmented mode. Without tools: 31.5%. R1's 50.2% is pure chain-of-thought without tools.


What "production-ready" means

  1. Reliable. Consistent quality across thousands of requests.

  2. Fast. 334 tok/s and 0.31s TTFT on Kimi K2.5.

  3. Capable. Within 4 points of Opus on code. Ahead on reasoning.

  4. Predictable. Versioned models that don't change without warning.

That last point is underrated. Proprietary models change under you — fine one day, different behavior the next, no changelog. Open-source models are versioned. DeepSeek V3.2 behaves the same tomorrow as today. You choose when to upgrade.

Sources: Artificial Analysis | SWE-bench | Kimi K2.5 | DeepSeek V3.2 | MMLU-Pro | HLE


r/OpenSourceeAI • • Mar 14 '26

I cut Claude Code costs by up to 80% (45% avg) and responses got better, benchmarked on 10 real engineering tasks

Thumbnail
gallery
115 Upvotes

Free tool: https://grape-root.vercel.app
Discord: https://discord.gg/rxgVVgCh (For debugging/feedback)

I’ve been building an Free tool called GrapeRoot (dual-graph context system) using claude code that sits on top of Claude Code. I just ran a benchmark on the latest version and the results honestly surprised me.

Setup:

Project used for testing:

Restaurant CRM: 278 files, 16 SQLAlchemy models, 3 frontends

10 complex prompts (security audits, debugging, migration design, performance optimization, dependency mapping)

Model: Claude Sonnet 4.6

Both modes had all Claude tools (Read, Grep, Glob, Bash, Agent).

GrapeRoot had the same tools plus pre-packed repo context (function signatures and call graphs).

Results

Normal ClaudeGrapeRoot
Total Cost$4.88$2.68
Avg Quality76.686.6
Avg Turns11.73.5

45% cheaper.
13% better quality.
10/10 prompts won.

Some highlights:

Performance optimization:
80% cheaper

20 turns → 1 turn
quality 89 → 94

Migration design:
81% cheaper

12 turns → 1 turn

Testing strategy:
76% cheaper

quality 28 → 91

Full-stack debugging:
73% cheaper

17 turns → 1 turn

Most of the savings came from eliminating exploration loops.

Normally Claude spends many turns reading files, grepping, and reconstructing repo context.

GrapeRoot instead pre-scans the repo, builds a graph of files/symbols/dependencies, and injects the relevant context before Claude starts reasoning.

So Claude starts solving the problem immediately instead of spending 10+ turns exploring.

Quality scoring:

Responses were scored 0–100 based on:
problem solved (30)
completeness (20)
actionable fixes/code (20)
specificity to files/functions (15)
depth of analysis (15)

Curious if other Claude Code users see the same issue:
Does repo exploration burn most of your tokens too?


r/OpenSourceeAI • • Apr 01 '26 Bravo

While Everyone Was Chasing Claude Code's Hidden Features, I Turned the Leak Into 4 Practical Technical Docs You Can Actually Learn From

Post image
113 Upvotes

After reading through a lot of the existing coverage, I found that most posts stopped at the architecture-summary layer: "40+ tools," "QueryEngine.ts is huge," "there is even a virtual pet." Interesting, sure, but not the kind of material that gives advanced technical readers a real understanding of how Claude Code is actually built.

That is why I took a different approach. I am not here to repeat the headline facts people already know. These writeups are for readers who want to understand the system at the implementation level: how the architecture is organized, how the security boundaries are enforced, how prompt and context construction really work, and how performance and terminal UX are engineered in practice. I only focus on the parts that become visible when you read the source closely, especially the parts that still have not been clearly explained elsewhere.

I published my 4 docs as pdfs [here](https://blog.netmind.ai/article/Claude_Code_Source_Code_Deep_Analysis_(in_pdf)), but below is a brief.

# The Full Series:

  1. **Architecture** — entry points, startup flow, agent loop, tool system, MCP integration, state management

  2. **Security** — sandbox, permissions, dangerous patterns, filesystem protection, prompt injection defense

  3. **Prompt System** — system prompt construction, [CLAUDE.md](http://CLAUDE.md) loading, context injection, token management, cache strategy

  4. **Performance & UX** — lazy loading, streaming renderer, cost tracking, Vim mode, keybinding system, voice input

# Overall

The core is a streaming agentic loop (`query.ts`) that starts executing tools while the model is still generating output. There are 40+ built-in tools, a 3-tier multi-agent orchestration system (sub-agents, coordinators, and teams), and workers can run in isolated Git worktrees so they don't step on each other.

**They built a full Vim implementation.** Not "Vim-like keybindings." An actual 11-state finite state machine with operators, motions, text objects, dot-repeat, and a persistent register. In a CLI tool. We did not see that coming.

**The terminal UI is a custom React 19 renderer.** It's built on Ink but heavily modified with double-buffered rendering, a patch optimizer, and per-frame performance telemetry that tracks yoga layout time, cache hits, and flicker detection. Over 200 components total. They also have a startup profiler that samples 100% of internal users and 0.5% of external users.

**Prompt caching is a first-class engineering problem here.** Built-in tools are deliberately sorted as a contiguous prefix before MCP tools, so adding or removing MCP tools doesn't blow up the prompt cache. The system prompt is split at a static/dynamic boundary marker for the same reason. And there are three separate context compression strategies: auto-compact, reactive compact, and history snipping.

**"Undercover Mode" accidentally leaks the next model versions.** Anthropic employees use Claude Code to contribute to public open-source repos, and there's a system called Undercover Mode that injects a prompt telling the model to hide its identity. The exact words: "Do not blow your cover." The prompt itself lists exactly what to hide, including unreleased model version numbers `opus-4-7` and `sonnet-4-8`. It also reveals the internal codename system: Tengu (Claude Code itself), Fennec (Opus 4.6), and Numbat (still in testing). The feature designed to prevent leaks ended up being the leak.

Still, listing a bunch of unreleased features are hidden in feature flags:

* **KAIROS** — an always-on daemon mode. Claude watches, logs, and proactively acts without waiting for input. 15-second blocking budget so it doesn't get in your way.

* **autoDream** — a background "dreaming" process that consolidates memory while you're idle. Merges observations, removes contradictions, turns vague notes into verified facts. Yes, it's literally Claude dreaming.

* **ULTRAPLAN** — offloads complex planning to a remote cloud container running Opus 4.6, gives it up to 30 minutes to think, then "teleports" the result back to your local terminal.

* **Buddy** — a full Tamagotchi pet system. 18 species, rarity tiers up to 1% legendary, shiny variants, hats, and five stats including CHAOS and SNARK. Claude writes its personality on first hatch. Planned rollout was April 1-7 as a teaser, going live in May.


r/OpenSourceeAI • • Mar 20 '26

Built an open source tool to find precise coordinates of any street image

97 Upvotes

Hey Guys,

I'm a college student and the developer of Netryx, after a lot of thought and discussion with other people I have decided to open source Netryx, a tool designed to find exact coordinates from a street level photo using visual clues and a custom ML pipeline and Al. I really hope you guys have fun using it! Also would love to connect with developers and companies in this space!

Link to source code: https://github.com/sparkyniner

Netryx-OpenSource-Next-Gen-Street-Level-Geolocation.git

Attaching the video to an example geolocating the Qatar strikes, it looks different because it's a custom web version but pipeline is same.


r/OpenSourceeAI • • Apr 26 '25

Deepseek R2 is almost here

Post image
96 Upvotes

▪︎ R2 is rumored to be a 1.2 trillion parameter model, double the size of R1

▪︎ Training costs are still a fraction of GPT-4o

▪︎ Trained on 5.2 PB of data, expected to surpass most SOTA models

▪︎ Built without Nvidia chips, using FP16 precision on a Huawei cluster

▪︎ R2 is close to release

This is a major step forward for open-source AI


r/OpenSourceeAI • • Sep 15 '26

Evolving neural networks to beat Mario ROM hacks — a Bowser's Crown 4-1 clear

95 Upvotes

This is a recorded Bowser's Crown 4-1 clear from NEATEvolve, a continuing community Mario AI project.

The bot uses NEAT: neural-network controllers are evaluated, selected and mutated across generations. They receive nearby game-state information and produce button presses. This video is a replay of a successful controller, rather than live learning during the clip.

The interesting part for me is how far this approach can get on SMB1 ROM hacks. A clear is encouraging, but backtracking and difficult nonlinear stages remain challenges, and one winning replay does not establish reliability on unfamiliar levels. The neural nets will have to learn the next level independantly of the previous one.

This builds on SethBling's MarI/O and work by Akisame and Electra. My continuing development is heavily assisted by AI tools.


r/OpenSourceeAI • • Mar 21 '26

I hate file formats that aren't Markdown, so I built md-anything

82 Upvotes

PDFs, ePubs, random web articles, and YouTube videos are a nightmare for AI agents. Claude and Cursor are great, but they only provide value if the context you feed them is clean.I got tired of wrestling with these "dead" formats. I just want my data in Markdown so I can actually work with it. So, I built md-anything. It’s a local-first CLI and MCP server that takes any file or URL (PDF, YouTube, images, epub, HTML) and converts it into honest, agent-ready Markdown + JSON metadata in one command.

• Agent-Native: It outputs structured Markdown that agents actually understand. It runs entirely on your machine.

• MCP Support: Wire it to Claude Desktop, Cursor, or VSCode and you have document ingestion built directly into your IDE.

It’s open-source (MIT). If you’re tired of messy document ingestion or want a cleaner way to feed context to your agents, give it a spin.

GitHub: https://github.com/ojspace/md-anything

Would love to hear your feedback. If you find it useful, a star on GitHub would mean the world to an indie project just starting out!


r/OpenSourceeAI • • Feb 20 '26

We open-sourced a local voice assistant where the entire stack - ASR, intent routing, TTS - runs on your machine. No API keys, no cloud calls, ~315ms latency.

Post image
82 Upvotes

VoiceTeller is a fully local banking voice assistant built to show that you don't need cloud LLMs for voice workflows with defined intents. The whole pipeline runs offline:

  • ASR: Qwen3-ASR-0.6B (open source, local)

  • Brain: Fine-tuned Qwen3-0.6B via llama.cpp (open source, GGUF, local)

  • TTS: Qwen3-TTS-0.6B with voice cloning (open source, local)

Total pipeline latency: ~315ms. The cloud LLM equivalent runs 680-1300ms.

The fine-tuned brain model hits 90.9% single-turn tool call accuracy on a 14-intent banking benchmark, beating the 120B teacher model it was distilled from (87.5%). The base Qwen3-0.6B without fine-tuning sits at 48.7% -- essentially unusable for multi-turn conversations.

Everything is included in the repo: source code, training data, fine-tuning configuration, and the pre-trained GGUF model on HuggingFace. The ASR and TTS modules use a Protocol-based interface so you can swap in Whisper, Piper, ElevenLabs, or any other backend.

Quick start is under 10 minutes if you have llama.cpp installed.

GitHub: https://github.com/distil-labs/distil-voice-assistant-banking

HuggingFace (GGUF model): https://huggingface.co/distil-labs/distil-qwen3-0.6b-voice-assistant-banking

The training data and job description format are generic across intent taxonomies not specific to banking. If you have a different domain, the slm-finetuning/ directory shows exactly how to set it up.


r/OpenSourceeAI • • Sep 18 '25

IBM AI Releases Granite-Docling-258M: An Open-Source, Enterprise-Ready Document AI Model

Thumbnail
marktechpost.com
80 Upvotes

IBM’s Granite-Docling-258M is an open-source (Apache-2.0) compact vision-language model for document conversion, succeeding SmolDocling with a Granite 165M backbone and SigLIP2 vision encoder. It outputs structured DocTags to preserve layout, tables, code, and equations with measurable accuracy gains across OCR, equations, and tables, plus improved stability. The model includes experimental multilingual support (Japanese, Arabic, Chinese), integrates with the Docling pipeline, and is available on Hugging Face in Transformers, ONNX, vLLM, and MLX formats for enterprise-ready, structure-preserving document AI....

full analysis: https://www.marktechpost.com/2025/09/17/ibm-ai-releases-granite-docling-258m-an-open-source-enterprise-ready-document-ai-model/

models on hugging face: https://huggingface.co/collections/ibm-granite/granite-docling-682b8c766a565487bcb3ca00

demo: https://huggingface.co/spaces/ibm-granite/granite-docling-258m-demo


r/OpenSourceeAI • • Mar 16 '26

Claude code can become 50-70% cheaper if you use it correctly! Benchmark result - GrapeRoot vs CodeGraphContext

Thumbnail
gallery
76 Upvotes

Free tool: https://grape-root.vercel.app/#install
Github: https://discord.gg/rxgVVgCh (For debugging/feedback)

Someone asked in my previous post how my setup compares to CodeGraphContext (CGC).

So I ran a small benchmark on mid-sized repo.

Same repo
Same model (Claude Sonnet 4.6)
Same prompts

20 tasks across different complexity levels:

  • symbol lookup

  • endpoint tracing

  • login / order flows

  • dependency analysis

  • architecture reasoning

  • adversarial prompts

I scored results using:

  • regex verification

  • LLM judge scoring

Results

MetricVanilla ClaudeGrapeRootCGC
Avg cost / prompt$0.25$0.17$0.27
Cost wins3/2016/201/20
Quality (regex)66.073.866.2
Quality (LLM judge)86.287.987.2
Avg turns10.68.911.7

Overall GrapeRoot ended up ~31% (average) went upto 90% cheaper per prompt and solved tasks in fewer turns and quality was similar to high than vanilla Claude code

Why the difference

CodeGraphContext exposes the code graph through MCP tools.

So Claude has to:

  1. decide what to query

  2. make the tool call

  3. read results

  4. repeat

That loop adds extra turns and token overhead.

GrapeRoot does the graph lookup before the model starts and injects relevant files into the Model.

So the model starts reasoning immediately.

One architectural difference

Most tools build a code graph.

GrapeRoot builds two graphs:

• Code graph : files, symbols, dependencies
• Session graph : what the model has already read, edited, and reasoned about

That second graph lets the system route context automatically across turns instead of rediscovering the same files repeatedly.

Full benchmark

All prompts, scoring scripts, and raw data:

https://github.com/kunal12203/Codex-CLI-Compact

Install

https://grape-root.vercel.app

Works on macOS / Linux / Windows

dgc /path/to/project

If people are interested I can also run:

  • Cursor comparison

  • Serena comparison

  • larger repos (100k+ LOC)

Suggest me what should i test now?

Curious to see how other context systems perform.


r/OpenSourceeAI • • Aug 13 '25

[UPDATE] DocStrange - Structured data extraction from images/pdfs/docs using AI models

76 Upvotes

I previously shared the open‑source library DocStrange. Now I have hosted it as a free to use web app to upload pdfs/images/docs to get clean structured data in Markdown/CSV/JSON/Specific-fields and other formats.

Live Demo: https://docstrange.nanonets.com

Would love to hear feedbacks!

Original Post - https://www.reddit.com/r/OpenSourceeAI/comments/1mh8i1s/built_a_free_document_to_structured_data/


r/OpenSourceeAI • • Aug 04 '25

Built a free document to structured data extractor — processes PDFs, images, scanned docs with free cloud processing

Thumbnail
gallery
73 Upvotes

Hey folks,

I recently built DocStrange, an open-source tool that converts PDFs, scanned documents, and images into structured Markdown — with support for tables, fields, OCR fallback, etc.

It runs either locally or in the cloud (we offer 10k documents/month for free). Might be useful if you're building document automation, archiving, or data extraction workflows.

Would love any feedback, suggestions, or ideas for edge cases you think I should support next!
GitHub: https://github.com/NanoNets/docstrange


r/OpenSourceeAI • • Aug 20 '26

I Made OpenCode Way Better

Post image
67 Upvotes

Hey everyone,

I have been using OpenCode for a while now. It's pretty great, but there was this one thing that kept bugging me: I couldn't easily create custom reliable workflows and pipelines. For a while, Opencode was one prompt and one model at a time. So, I created OpenFlow, a very minimalist open-sourced project that allows you to orchestrate a pipeline of agents while still connecting your own API keys. I forked it from OpenCode, so you still have the harness, skills, and overall ability. I published it recently and am still often improving it. I would love to hear some feedback of what you guys thought of it and how I can improve it. Thanks!

Link: https://github.com/SeeRay11/OpenFlow