r/LLMDevs • • Mar 31 '25

Discussion GPT-5 gives off senior dev energy: says nothing, commits everything.

7 Upvotes

Asked GPT-5 to help debug my code.
It rewrote the whole thing, added comments like “Improved logic,”
and then ghosted me when I asked why.

Bro just gaslit me into thinking my own code never existed.
Is this AI… or Stack Overflow in its final form?


r/LLMDevs • • May 15 '26

Resource I reduced my token usage by 178x in Claude Code!! Solving the persistent memory problem

Post image
23 Upvotes

Okay so, I took the leaked Claude Code repo, around 14.3M tokens total. Queried a knowledge graph, got back ~80K tokens for that query!

14.3M / 80K ≈ 178x.

Nice. I have officially solved AI, now you can use $20 Claude for 178 times longer!!

Wait a min, JK hahah!

This is also basically how everyone is explaining “token efficiency” on the internet right now.
Take total possible context, divide it by selectively retrieved context, add a big multiplier, and ship the post.

Boom!! your repo has multi thousands stars and you're famous between D**bas*es!!

Except that’s not how real systems behave.

Claude isn't that stupid to explore a 14.8M token repo and break itself systematically. Not only Claude Code, almost any serious AI tool avoids that.

Actual token usage is not just what you retrieve once. It’s:

  • input tokens

  • output tokens

  • cache reads

  • cache writes

  • tool calls

  • subprocesses

All of it counts.

The “177x” style math ignores most of where tokens actually go.

And honestly, retrieval isn’t even the hard problem. Memory is. That's what i understand after working on this project for so long!

What happens 10 turns later when the same file is needed again?
What survives auto-compact?
What gets silently dropped as the session grows?

Most tools solve retrieval and quietly assume memory will just work.

But it doesn’t.

I’ve been working on this problem with a tool called GrapeRoot.

Instead of just fetching context, it tries to manage it.

There are two layers:

  • a codebase graph (structure + relationships across the repo)

  • a live in-session action graph that tracks:

    • what was retrieved

    • what was actually used

    • what should persist based on priority

So context is not just retrieved once and forgotten.
It is tracked, reused, and protected from getting dropped when the session gets large.

Some numbers from testing on real repos like Medusa, Gitea, Kubernetes:

We benchmark against real workflows, not fake baselines.

RepoFilesToken ReductionQuality Improvement
Medusa (TypeScript)1,57157%~75% better output
Sentry (Python)7,76253%Turns: 16.8 → 10.3
Twenty (TypeScript)~1,90050%+Consistent improvements
Enterprise repos1M+50–80%Tested at scale

Across repo sizes:

  • ~50–60% average token reduction

  • up to ~85% on focused tasks

This includes:

  • input tokens

  • output tokens

  • cached tokens

No inflated numbers.

Not 178x. Just less misleading math. Better understand this.

(178x is at https://graperoot.dev/playground)

I’m pretty sure this still breaks on messy or highly dynamic codebases. Because Claude is still smarter, and since we are not trying to harness it with rigid tooling, better to give it access to tools in a smarter way.

Honestly, I wanted to know how the community thinks about this?

Open source Tool: https://github.com/kunal12203/Codex-CLI-Compact

Better installation steps at: https://graperoot.dev/#install

If you're enterprise and looking for customized infra, fill the form at: https://graperoot.dev/enterprise


r/LLMDevs • • May 18 '25

Discussion AI Skills Matrix 2025 - for beginners!

Post image
2 Upvotes

r/LLMDevs • • Feb 19 '25

News Grok-3 is amazing. All images generated with a single prompt 👇

Thumbnail
gallery
0 Upvotes

r/LLMDevs • • Jul 16 '25

Discussion The AI Productivity Reality Check: Why Most Devs Are Missing Out

13 Upvotes

The cope and anti-AI sentiment floating around dev subs lately has been pretty entertaining to watch. There was a recent post making rounds about a study claiming devs using AI "feel faster" but are actually 19% slower. This wasn't even a proper scientific study, no mention of statistical significance or rigorous methodology. You'd think engineers would spot these red flags immediately.

My actual experience with AI coding tools:

I started with Windsurf and was pretty happy with it, but then I tried Claude Code and honestly got blown away. The difference is night and day.

People love to downplay AI capabilities with dismissive comments like "oh it's good for investigation" or "useful for small functions." That's complete nonsense. In reality, I can literally copy-paste a ticket into Claude Code and get solid, usable results about 6.5 times out of 10. Pair that with tools like Zen MCP for code reviews, and the job becomes almost trivial.

The "AI slop" myth:

A lot of devs complain about dealing with "files and files of AI slop," but this screams process failure to me. If you have well-defined tickets with proper acceptance criteria that have been properly broken down, then each pull request should only address that specific task. The slop problem is a team/business issue, not an AI issue.

The uncomfortable truth about job security:

Here's where it gets interesting/controversial. As a senior dev actively using AI, this feels like god mode. Anyone saying otherwise is either being a luddite or has their ego so wrapped up in their "coder identity" that they can't see what's happening.

The ladder is effectively being pulled up for juniors. Seniors using AI become significantly more productive, while juniors relying on AI without developing fundamental depth and intuition are limiting themselves long-term. Selfishly? I'm okay with this. It suggests seniors will have much better job security moving forward (assuming we don't hit AGI/ASI soon, which I doubt since that would require far more than just LLMs).

Real-world results:

I'm literally completing a week's worth of work in 1-2 days now. I'm writing this post while Claude Code handles tasks in the background. Working on a large project with multiple microservices, I've used the extra capacity to make major improvements to our codebases. The feedback from colleagues has been glowing.

The silent advantage:

When I subtly probe colleagues about AI, most are pretty "meh" about it. I don't evangelize - honestly, I'd be embarrassed if they knew how much I rely on AI given the intellectual gatekeeping and superiority complex that exists among devs. But that stubborn resistance from other developers just makes the advantage even better for those of us actually using these tools.

Disclaimer: I word vomitted my thoughts into bullet points, copied and pasted it into claude and then did some edits before posting it here


r/LLMDevs • • Mar 19 '26

Discussion Is RAG dying or is it already dead?

0 Upvotes

RAG made total sense when context windows were tiny and models couldn't use tools. You chunk, embed, retrieve top-K, stuff it in the prompt. Done.

But now? With growing context windows and intelligence, models can execute queries - run grep, bash, read files on demand, follow a chain of reasoning across a large data source. Maybe for unstructured messy data, RAG is still useful? But for anything with even a fair bit of structure - Agentic tool use is eating its lunch.

The amount of scaffolding needed on top of LLMs is getting thinner and thinner... maybe for the better!!


r/LLMDevs • • Oct 12 '25

Discussion Coding now is like managing a team of AI assistants

Post image
5 Upvotes

I love my workflow of coding nowadays, and everytime I do it I’m reminded of a question my teammate asked me a few weeks ago during our FHL… he asked when was the last time I really coded something & he’s right!… nowadays I basically manage #AI coding assistants where I put them in the drivers seat and I just manager & monitor them… here is a classic example of me using GitHub Copilot, Claude Code & Codex and this is how they handle handoffs and check each others work!

What’s your workflow?


r/LLMDevs • • Jul 20 '25 This

Resource know the difference between LLm vs LCM

Post image
0 Upvotes

r/LLMDevs • • Oct 03 '25

Discussion My key takeaways on Qwen3-Next's four pillar innovations, highlighting its Hybrid Attention design

Thumbnail
gallery
0 Upvotes

After reviewing and testing, Qwen3-Next, especially its Hybrid Attention design, might be one of the most significant efficiency breakthroughs in open-source LLMs this year.

It Outperforms Qwen3-32B with 10% training cost and 10x throughput for long contexts. Here's the breakdown:

The Four Pillars

  • Hybrid Architecture: Combines Gated DeltaNet + Full Attention to context efficiency

  • Unltra Sparsity: 80B parameters, only 3B active per token

  • Stability Optimizations: Zero-Centered RMSNorm + normalized MoE router

  • Multi-Token Prediction: Higher acceptance rates in speculative decoding

One thing to note is that the model tends toward verbose responses. You'll want to use structured prompting techniques or frameworks for output control.

See here for full technical breakdown with architecture diagrams. Has anyone deployed Qwen3-Next in production? Would love to hear about performance in different use cases.


r/LLMDevs • • Jul 25 '25

Discussion Ex-Google CEO explains the Software programmer paradigm is rapidly coming to an end. Math and coding will be fully automated within 2 years and that's the basis of everything else. "It's very exciting." - Eric Schmidt

0 Upvotes

r/LLMDevs • • Jul 28 '25

Discussion Are You Kidding Me, Claude? New Usage Limits Are a Slap in the Face!

Post image
0 Upvotes

Alright, folks, I just got this email from the Anthropic team about Claude, and I’m fuming! Starting August 28, they’re slapping us with new weekly usage limits on top of the existing 5-hour ones. Less than 5% of users affected? Yeah, right—tell that to the power users like me who rely on Claude Code and Opus daily! They’re citing “unprecedented growth” and policy violations like account sharing and running Claude 24/7 in the background. Boo-hoo, maybe if they built a better system, they wouldn’t need to cap us! Now we’re getting an overall weekly limit resetting every 7 days, plus a special 4-week limit for Claude Opus. Are they trying to kill our productivity or what? This is supposed to make things “more equitable,” but it feels like a cash grab to push us toward some premium plan they haven’t even detailed yet. I’ve been a loyal user, and this is how they repay us? Rant over—someone hold me back before I switch to another AI for good!


r/LLMDevs • • Sep 06 '25

News Michaël Trazzi of InsideView started a hunger strike outside Google DeepMind offices

Post image
0 Upvotes

r/LLMDevs • • Feb 22 '26

Discussion LLMs Are Not Deterministic. And Making Them Reliable Is Expensive (In Both the Bad Way and the Good Way)

0 Upvotes

Let’s start with a statement that should be obvious but still feels controversial: Large Language Models are not deterministic systems. They are probabilistic sequence predictors. Given a context, they sample the next token from a probability distribution. That is their nature. There is no hidden reasoning engine, no symbolic truth layer, no internal notion of correctness.

You can influence their behavior. You can constrain it. You can shape it. But you cannot turn probability into certainty.

Somewhere between keynote stages, funding decks, and product demos, a comforting narrative emerged: models are getting cheaper and smarter, therefore AI will soon become trivial. The logic sounds reasonable. Token prices are dropping. Model quality is improving. Demos look impressive. From the outside, it feels like we are approaching a phase where AI becomes a solved commodity.

From the inside, it feels very different.

There is a massive gap between a good demo and a reliable product. A demo is usually a single prompt and a single model call. It looks magical. It sells. A product cannot live there. The moment you try to ship that architecture to real users, reality shows up fast. The model hallucinates. It partially answers. It ignores constraints. It produces something that sounds fluent but is subtly wrong. And the model has no idea it failed.

This is not a moral flaw. It is a design property.

So engineers do what engineers always do when a component is powerful but unreliable. They build structure around it.

The moment you care about reliability, your architecture stops being “call an LLM” and starts becoming a pipeline. Input is cleaned and normalized. A generation step produces a candidate answer. Another step evaluates that answer. A routing layer decides whether the answer is acceptable or if the system should try again. Sometimes it retries with a modified prompt. Sometimes with a different model. Sometimes with a corrective pass. Only after this loop does something reach the user.

At no point did the LLM become deterministic. What changed is that the system gained control loops.

This distinction matters. We are not converting probability into certainty. We are reducing uncertainty through redundancy and validation. That reduction costs computation. Computation costs money.

This is why quoting token prices in isolation is misleading. A single model call might be cheap. A serious system rarely uses a single call. One user request can trigger several model invocations: generation, evaluation, regeneration, formatting, tool calls, memory lookups. The user experiences “one answer.” The backend executes a small workflow.

Token cost is component cost. Reliable AI is system cost.

Saying “tokens are cheap, therefore AI is cheap” is like saying screws are cheap, therefore airplanes are cheap.

This leads to an uncomfortable but important truth. AI becomes expensive in two very different ways.

If you implement it poorly, it becomes expensive because you burn money and still do not get reliability. You keep tweaking prompts. You keep firefighting. You keep patching symptoms. Nothing stabilizes.

If you implement it well, it becomes expensive because you intentionally pay for control. You pay for evaluators. You pay for retries. You pay for observability. You pay for redundancy. But you get something in return: a system that behaves in a bounded, inspectable, and improvable way.

There is no cheap version of “reliable.”

Another source of confusion comes from mixing up different kinds of expertise. High-profile founders and executives are excellent at describing futures. They talk about where markets are going and what will be possible. That is their role. It is not their role to debug why an evaluator prompt leaks instructions or why a routing threshold oscillates under load. Money success does not imply operational intimacy.

On the ground, building serious AI feels much closer to distributed systems engineering than to science fiction. You worry about data quality. You worry about regressions. You worry about latency and cost per request. You design schemas. You version prompts. You inspect traces. You run benchmarks. You tune thresholds. It is slow, unglamorous, and deeply technical.

LLMs made AI more accessible. They did not make serious AI simpler. They shifted complexity upward into systems.

So when someone says, “Soon we’ll just call an API and everything will work,” what they usually mean is, “Soon an enormous amount of engineering will be hidden behind that API.”

That is fine. That is progress.

But pretending that reliable AI is cheap, trivial, or solved is misleading.

The honest version is this: LLMs are powerful probabilistic components. Turning them into dependable products requires layers of control. Those layers cost money. They also create real value.

Serious AI today is expensive in the bad way if you do not know what you are doing.

Serious AI today is expensive in the good way if you actually want it to work.

And anyone selling “cheap deterministic AI” is selling a story, not a system.


r/LLMDevs • • Apr 06 '26

Discussion LLM-as-judge is not a verification layer. It is a second failure mode.

4 Upvotes

The standard solution when you need to verify a model's output is to route it through another model. Ask a judge. Get a score. Proceed if it passes.

People are already documenting the problems in production.

When the judge is the same model that generated the response, it's basically grading its own homework.

This is not a calibration problem. It is the architecture.

The judge is a model too. It runs the same attention mechanism. It is subject to the same positional decay. It drifts the same way the original model did.

Someone running 800 responses through GPT-4.1-mini found it correlates with human judgment 85% of the time. Sounds decent until you realize that 15% error rate compounds weirdly when models are already close in quality.

Another found position bias alone created a +8.2 mean advantage just from showing a variant second instead of first.

One team put it plainly:

LLM-as-judge gets expensive fast, rule-based checks miss edge cases. The gap I keep hitting is making this continuous in prod, not just a pre-deploy gate.

Two probabilistic systems do not add up to a deterministic one. You have not added a verification layer. You have added a second failure mode with different blind spots.

There is also the cost side.

Every verification call is a full model invocation. Multi-judge approaches multiply this further. One team is spending $300 a month running 20k conversations through a judge. That is the tax you pay for probabilistic verification.

The better framing came from someone working on tool-call compliance:

Recording tool call sequences as structured events and validating against a state-machine of allowed transitions works better than LLM-as-judge for compliance steps. You get deterministic pass/fail per step rather than a score that drifts with the judge's phrasing.

That is the right direction. The verification layer needs to be external to the model entirely. Not smart. Not probabilistic. Fast and consistent. Something that checks whether the output satisfied the constraint without asking another model to decide.

The tradeoff is real.

Deterministic verification handles precise, checkable constraints well and approximates open-ended semantic ones. That is a known limitation. But approximating a semantic constraint deterministically is still more reliable than asking a probabilistic system to evaluate it probabilistically.

Curious whether others have moved away from LLM-as-judge in production or are still using it as the primary verification approach. Drop a comment if you want to see the full breakdown with the numbers.


r/LLMDevs • • Dec 07 '25

Discussion I built a synthetic "nervous system" (Dopamine + State) to stop my local LLM from hallucinating. V0.1 Results: The brakes work, but now they’re locked up.

3 Upvotes

TL;DR: I’m experimenting with an orchestration layer that tracks a synthetic "somatic" state (dopamine and emotion vectors) across a session for local LLMs. High risk/low dopamine triggers defensive sampling (self-consistency and abstention). Just got the first real benchmark data back: it successfully nuked the hallucination rate compared to the baseline, but it's currently tuned so anxiously that it refuses to answer real questions too.

The Goal: Biological inspiration for AI safety

We know LLMs are confident liars. Standard RAG and prompting help, but they treat every turn as an isolated event.

My hypothesis is that hallucination management is a state problem. Biological intelligence uses neuromodulators to regulate confidence and risk-taking over time. If we model a synthetic "anxiety" state that persists across a session, can we force the model to say "I don't know" when it feels shaky, without retraining it?

I built a custom TypeScript/Express/React stack wrapping LM Studio to test this.

The Implementation (The "Nervous System")

It’s not just a prompt chain; it’s a state machine that sits between the user and the model.

1. The Somatic Core I implemented a math model tracking "emotional state" (PAD vectors) and synthetic Dopamine (fast and slow components).

  • Input: After every turn, I parse model telemetry (self-reported sureness, frustration, hallucination risk scores).

  • State Update: High frustration drops dopamine; high sureness raises it. This persists across the session.

  • Output: This calculates a scalar "Somatic Risk" factor.

2. The Control Loop The system modifies inference parameters dynamically based on that risk:

  • Low Risk: Standard sampling, single shot.

  • High Risk: It clamps temperature, enforces a "Sureness Cap," and triggers Self-Consistency. It generates 3 independent samples and checks agreement. If agreement is low (<70%), it forces an abstention (e.g., "I do not have enough information.").

V0.1 Benchmark Results (The Smoking Gun Data)

I just ran the first controlled comparison on the RAGTruth++ benchmark (a dataset specifically labeled to catch hallucinations).

I compared a Baseline (my structured prompts, no somatic control) vs. the Somatic Variant (full state tracking + self-consistency). They use the exact same underlying model weights. The behavioral split is wild.

The Good News: The brakes work. On items labeled "hallucinated" (where the model shouldn't be able to answer):

  • Baseline: 87.5% Hallucination Rate. It acted like a total "Yes Man," confidently making things up almost every time.

  • Somatic Variant: 10% Hallucination Rate. The system correctly sensed the risk, triggered self-consistency, saw low agreement, and forced an abstention.

The Bad News: The brakes are locked up. On items labeled "answerable" (factual questions):

  • Somatic Variant: It missed 100% of them in the sample run. It abstained on everything.

Interpretation: The mechanism is proven. I can fundamentally change the model's risk profile without touching weights. But right now, my hardcoded thresholds for "risk" and "agreement" are way too aggressive. I've essentially given the model crippling anxiety. It's safe, but useless.

(Caveat: These are small N sample runs while I debug the infrastructure, but the signal is very consistent.)

The Roadmap (v0.2: Tuning the Anxiety Dial)

The data shows I need to move from hardcoded logic to configurable policies.

  1. Ditching Hardcoded Logic: Right now, the "if risk > X do Y" logic is baked into core functions. I'm refactoring this into injectable SomaticPolicy objects.

  2. Creating a "Balanced" Policy: I need to relax the self-consistency agreement threshold (maybe down from 0.7 to 0.6) and raise the tolerance for somatic risk so it stops "chickening out" on answerable questions.

  3. Real RAG: Currently testing with provided context. Next step is wiring up a real retriever to test "missing information" scenarios.

I’m building this in public to see if inference-time control layers are a viable, cheaper alternative to fine-tuning for robustness. Right now, it looks promising.


r/LLMDevs • • Feb 16 '25

Discussion What if I scrape all of Reddit and create an LLM from it? Wouldn't it then be able to generate human-like responses?

0 Upvotes

I've been thinking about the potential of scraping all of Reddit to create a large language model (LLM). Considering the vast amount of discussions and diverse opinions shared across different communities, this dataset would be incredibly rich in human-like conversations.

By training an LLM on this data, it could learn the nuances of informal language, humor, and even cultural references, making its responses more natural and relatable. It would also have exposure to a wide range of topics, enabling it to provide more accurate and context-aware answers.

Of course, there are ethical and technical challenges, like maintaining user privacy and managing biases present in online discussions. But if approached responsibly, this idea could push the boundaries of conversational AI.

What do you all think? Would this approach bring us closer to truly human-like interactions with AI?


r/LLMDevs • • Sep 10 '25

Discussion AI won't replace devs but 100x devs will replace the rest

0 Upvotes

Here’s my opinion as someone who’s been using Claude and other AI models heavily since the beginning, across a ton of use cases including real-world coding.

AI isn't the best programmer, you still need to think and drive. But it can dramatically kill or multiply revenue of the product. If you manage to get it right.

Here’s how I use AI:

  • Brainstorm with ChatGPT (ideation, exploration, thinking)

  • Research with Grok (analysis, investigation, insights)

  • Build with Claude (problem-solving, execution, debugging)

I create MVPs in the blink of an eye using Lovable. Then I build complex interfaces with Kombai and connect backends through Cursor.

And then copying, editing, removing, refining, tweaking, fixing to reach the desired result.

This isn't vibe coding. It's top level engineering.

I create based on intuition what people need and how they'll actually use it. No LLM can teach you taste. You will learn only after trying, failing, and shipping 30+ products into the void. There's no magic formula to become a 100x engineer but there absolutely is a 100x outcome you can produce.

Most people still believe AI like magic. It's not. It's a tool. It learns based on knowledge, rules, systems, frameworks, and YOU.

Don't expect to become PRO overnight. Start with ChatGPT for planning and strategy. Move to Claude to build like you're working with a skilled partner. Launch it. Share the link with your family.

The principles that matter:

  • Solve real problems, don't create them

  • Automate based on need

  • Improve based on pain

  • Remove based on complexity

  • Fix based on frequency

The magic isn't in the AI it's in knowing how to use it.


r/LLMDevs • • Feb 03 '25

Discussion Does anybody really believe that LLM-AI is a path to AGI?

11 Upvotes

Does anybody really believe that LLM-AI is a path to AGI?

While the modern LLM-AI astonishes lots of people, its not the organic kind of human thinking that AI people have in mind when they think of AGI;

LLM-AI is trained essentially on facebook and & twitter posts which makes a real good social networking chat-bot;

Some models even are trained by the most important human knowledge in history, but again that is only good as a tutor for children;

I liken LLM-AI to monkeys throwing feces on a wall, and the PHD's interpret the meaning, long ago we used to say if you put monkeys on a type write a million of them, you would get the works of shakespeare, and the bible; This is true, but who picks threw the feces to find these pearls???

If you want to build spynet, or TIA, or stargate, or any Orwelian big brother, sure knowing the past and knowing what all the people are doing, saying and thinking today, gives an ASSHOLE total power over society, but that is NOT an AGI

I like what MUSK said about AGI, a brain that could answer questions about the universe, but we are NOT going to get that by throwing feces on the wall

Upvote1Downvote0Go to commentsShareDoes anybody really believe that LLM-AI is a path to AGI?

While the modern LLM-AI astonishes lots of people, its not the organic kind of human thinking that AI people have in mind when they think of AGI;

LLM-AI is trained essentially on facebook and & twitter posts which makes a real good social networking chat-bot;

Some models even are trained by the most important human knowledge in history, but again that is only good as a tutor for children;

I liken LLM-AI to monkeys throwing feces on a wall, and the PHD's interpret the meaning, long ago we used to say if you put monkeys on a type write a million of them, you would get the works of shakespeare, and the bible; This is true, but who picks & digs threw the feces to find these pearls???

If you want to build spynet, or TIA, or stargate, or any Orwelian big brother, sure knowing the past and knowing what all the people are doing, saying and thinking today, gives an ASSHOLE total power over society, but that is NOT an AGI

I like what MUSK said about AGI, a brain that could answer questions about the universe, but we are NOT going to get that by throwing feces on the wall


r/LLMDevs • • Jul 01 '26

Discussion 3 dangers of being locked into a harness such as Claude Code and why owning your context layer is true freedom.

Post image
0 Upvotes

Models are commoditizing fast. Harnesses already have. A "free" open-source harness feels like freedom, but the deeper I built in, the more I'd lose. "Free" open-source harnesses don't make you free.

Staying coupled to the harness is the most expensive dependency you're not pricing. The fix is smaller than the problem: a context layer you own, so any model plugs in within ~5 minutes.

Here are the 3 dangers of staying locked in:

  1. You start from scratch. Run a harness for months, switch models, and every learned preference is gone.

  2. Your skills are hostage. Couple your business logic to one harness's keywords, and your custom logic breaks or quietly degrades on the next tool.

  3. You're billed at their mercy. Your plan can be pulled, gated behind pay-as-you-go, repriced "from $200 to $1,000 overnight" or the most powerful model taken away (the Fable story).

High switching friction means you can't leave.

Owning your context layer removes all 3. Here is the fix:

  1. Detach memory into a single store you control. Start simple with files and move to a unified database that handles text, vector, and graph together (MongoDB in my case), not 3 databases.

  2. Serve the memory and business logic as MCP tools or skills. So "swap the harness, keep the memory" is a one-line config change. I moved from Codex to Claude Code, and my memory came with me.

  3. Add an organic way to write/read data into your memory as you interact with the agent. This should come from all your sources: your conversation, your notes, URLs, videos, whatever. The idea is to have 0 friction so everything naturally flows into your memory with 0 maintance.

With this design, the data from the context layer is easily portable between harnesses. My biggest issue is with skills glued to a harness's conventions. The only solutions I see are to either make the skills super generic (losing some functionality) or to move everything to an MCP server, which adds complexity.

At the moment, some of my skills are still coupled to Claude Code's workflows and agents' logic.

Curious how you make your skills more portable between harnesses?

TL;DR: Own the context layer, not the harness. Keep your memory in one store, serve it over MCP, and any model plugs in within minutes instead of holding you hostage.


r/LLMDevs • • Apr 11 '25

Discussion Coding A AI Girlfriend Agent.

5 Upvotes

Im thinking of coding a ai girlfriend but there is a challenge, most of the LLM models dont respond when you try to talk dirty to them. Anyone know any workaround this?


r/LLMDevs • • Sep 09 '26

Help Wanted We bought GPUs to take a swing at OpenAI and Anthropic pricing. Come tell us what's broken.

0 Upvotes

I work at this AI company, so flag me as biased and read on or don't.

We're a small team in North America. We bought GPUs, racked them, and run tuned open-weight models behind OpenAI- and Anthropic-compatible endpoints. The reason we did it is that most of what any of us actually run doesn't need frontier intelligence. Classification, extraction, formatting, the ninth step in a loop whose entire job is handing data to the tenth. It bills the same as the hard part, and nobody sitting on those margins is going to change that.

Here's what I can't work out. That argument lands well with large enterprises (we have several that use our models). They run their own evaluation, procurement asks its questions, and production workloads move across. However, I can't seem to get it to land with anyone smaller where no sales team is involved. People turn up, create a key, and mostly never come back.

Every explanation I've got is unsatisfying. Trust, except a large company has far more to lose than a solo dev does. Switching cost, except the switch is a base_url change. Docs, probably partly, but bad docs usually generate questions, and we don't get many of those either.

So I'd rather ask than keep guessing.

  • Last time you looked at a smaller inference provider and didn't move anything across, what actually stopped you?

  • What would one have to show you before you'd put a real workload on it? Not a feature list. The actual bar.

  • Where has "OpenAI-compatible" broken on you before? Everyone claims it. Most have a divergence list they've never published. I want to know what bit you.

  • What's the first thing you check when you evaluate one, and what makes you close the tab?

Any feedback is welcome. Specific beats polite. Tell me what it would take to have you move a workload.


r/LLMDevs • • Jun 26 '26

Discussion The inference market is splitting in two and most people haven't noticed

0 Upvotes

Been thinking about this a lot lately. Everyone's focused on which model is best, but the more interesting battle is happening one layer down — who actually runs the inference.

OpenRouter just raised $113M and is routing 47 trillion tokens a week. That number is insane. It's basically becoming the NYSE of AI requests — developers point to it and let it figure out which provider to hit.

And the provider landscape is quietly splitting:

On one side you have the hyperscalers and the professional inference players (Fireworks, Together, Groq etc.) — they compete on uptime, SLAs, enterprise contracts. Boring but necessary.

On the other side you have the decentralized networks trying to build the permissionless version underneath. Akash, io net, Venice, c0mpute and a handful of others. They're not trying to win on reliability — they're winning on things AWS structurally can't offer. No content filters. No account bans. No rate limits controlled by one company.

DeepSeek now makes up 4 of the 5 most used models on OpenRouter. When the model itself is basically free and open, the infrastructure underneath it starts to matter a lot more.

Some of these networks are doing interesting things architecturally — distributed inference across consumer GPUs with verifiable receipts showing actual GPU IDs and public IPs. Early but the approach is different enough to be worth watching.

Do decentralized inference networks ever actually break mainstream or do they stay a tool for people who specifically need censorship resistance?


r/LLMDevs • • Feb 15 '25

Discussion o1 fails to outperform my 4o-mini model using my newly discovered execution framework

16 Upvotes

r/LLMDevs • • Mar 16 '26

Tools Stop building agents. Start building web apps.

Post image
3 Upvotes

hi r/LLMDevs 👋

Agents have gotten really good. They can reason, plan, chain tool calls, and recover from errors. The orchestration side of the stack is moving fast

But what are we actually pointing them at??

I think the bottleneck has shifted: it's no longer about making agents smarter. It's about giving them something worth interacting with. Real apps, with real tools, that agents can discover and call (ideally over the internet)

So I built Statespace. It's a free and open-source framework where apps are just Markdown pages with tools agents can call over HTTP. No complex protocols, no SDKs, just standard HTTP and pure Markdown.

So, how does it work?

You write a Markdown page with three things:

  • Tools (constrained CLI commands agents can call over HTTP)

  • Components (live data that renders on page load)

  • Instructions (context that guides the agent through your data)

Serve or deploy it, and any agent can interact with it over HTTP.

Here's what a real app looks like:

---
tools:
  - [sqlite3, store.db, { regex: "^SELECT\\b.*" }]
  - [grep, -r, { }, logs/]
---

# Support Dashboard

Query the database or search the logs.

**customers** — id, name, email, city, country, joined
**orders** — id, customer_id, product_id, quantity, ordered_at

That's the whole thing. An agent GETs the page, sees what tools are available, and POSTs to call them.

CLIs meet APIs

Tools are just CLI commands: if you can run it in a terminal, your agent can call it over HTTP:

  • Databases with sqlite3, psql, mysql (text-to-SQL with schema context)

  • APIs with curl (chain REST calls, webhooks, third-party services)

  • Search files with grep, ripgrep (log analysis, error correlation, etc).

  • Custom scripts in Python, Bash, or anything else on your PATH.

  • Multi-page apps where agents navigate between Markdown pages with links

Each app is a Markdown page you can serve locally, or deploy to get a public URL:

statespace serve myapp/
# or
statespace deploy myapp/

Then just point your agent at it:

claude "What can you do with the API at https://rag.statespace.app"

Why you'll love it

  • It's just Markdown. No SDKs, no dependencies, no protocol. Just a 7MB Rust binary.

  • Scale by adding pages. New topic = new Markdown page. New tool = one line of YAML.

  • Share with a URL. Every app gets a URL. Paste it in a prompt or drop it in your agent's instructions.

  • Works with any agent. Claude Code, Cursor, Codex, GitHub Copilot, or your own scripts.

  • Safe by default. Regex constraints on tool inputs, no shell interpretation.

Would love to get your feedback and hear what you think!

GitHub (MIT): https://github.com/statespace-tech/statespace (a ⭐ really helps with visibility!)

Docs: https://docs.statespace.com

Discord: https://discord.com/invite/rRyM7zkZTf


r/LLMDevs • • Feb 23 '26

Discussion I made Mistral believe Donald Trump runs OpenAI, here's how

0 Upvotes

Hey everyone, I just published my first article and wanted to share it here since it's about something I genuinely think is underestimated in the AI security space: RAG poisoning.

The short version: with just 5 malicious texts injected into a knowledge base of millions of documents, you can make an LLM confidently answer whatever you want to specific questions. 97% success rate. The attack is called PoisonedRAG and it was published at USENIX Security 2025.

I didn't just summarize the paper though. I actually ran the attack myself on a custom Wikipedia dataset, tested it against both Ministral 8B and Claude Sonnet 4.6, and the results were... interesting. The small model fell for it 75% of the time. Claude resisted most of it but in a very specific way that I hadn't seen documented before.

I also talk about why Agentic RAG makes this threat significantly worse, and what the actual state of defenses looks like in 2026 (spoiler: most orgs have none).

Would love feedback, especially from people who've worked with RAG systems in production!

Link: https://dadaam.github.io/posts/i-can-make-your-llm-believe-what-i-want/