r/AISystemsEngineering • • Apr 30 '26

I finally sat down and did the math on my Cloud LLM bills… and I’m moving almost everything to a 4090.

71 Upvotes

I used to be all-in on cloud APIs. For any side project, I’d just grab an OpenAI or Anthropic key and not think twice. It was convenient. No worrying about VRAM, super fast responses, and I could spin something up in minutes.

But that “pay-as-you-go” comfort slowly turned into real pain.

Last month one of my small RAG tools that I built for a few friends racked up $120 in API costs. Then an experimental agent I left running in a loop hit $450. That was the moment I opened a spreadsheet and realized I was basically burning money every time someone used my stuff.

The numbers that really shocked me were pretty simple:

A single RAG query on something like GPT-4o-mini costs around $0.0005. Sounds tiny, right? But once you scale to a million queries, that becomes a $500 monthly bill for what’s supposed to be a side project.

Now compare that to running a quantized Llama-3.1-8B locally on a 4090. For those same million queries, you’re probably looking at just $15–30 in electricity and normal hardware wear.

Even at a more realistic 200k tokens per month, the cloud bill was hitting $50 while the local setup cost me barely $10. And the best part? My latency went from about 2 seconds waiting on the cloud to under 0.5 seconds locally.

These days I still use Claude 3.5 Sonnet when I’m in the early prototyping phase and I need that really strong reasoning. But the moment a project starts getting real users or higher volume, I move it over to a local model.

The freedom feels good. No more rate limits, full privacy, and zero surprise bills at the end of the month.

If you’re tired of watching your cloud costs creep up, try tracking your token usage for just one week. If you’re spending more than $50 a month on inference for stuff that a 7B or 8B model can handle decently, it might be worth thinking about running things locally instead of renting compute forever.

Has anyone else made the switch from cloud to local and actually stuck with it?


r/AISystemsEngineering • • Oct 07 '26

I want to build a fully local AI system for an engineering company using open-source models. It may eventually need to understand CAD files and assist with design. Confidentiality is critical. Any good books, YouTube channels, courses, or projects to learn from?

47 Upvotes

r/AISystemsEngineering • • Apr 26 '26

I finally uninstalled LangChain and cleared 50GB of hype off my drive

40 Upvotes

I’ve spent the last two years installing every revolutionary LLM tool that trended on GitHub. Most of them looked incredible in a 30-second demo, but after a week of real use, they just turned into dead weight.

Last month, I finally did a massive cleanup and realized half my disk space was taken up by abstractions I hadn't touched in months.

LangChain was the first to go. It was a great training wheel tool when I was first learning RAG, but once I understood the data flow, I realized I was spending 80% of my time fighting the framework instead of building. 

Between the abstraction leaks and constant breaking updates, I just rewrote my core logic in plain Python and never looked back. I did the same with most autonomous agent frameworks like AutoGen and CrewAI. 

They are fun for demos, but they were massive overkill for 90% of what I do. I ended up just writing simple loops with direct Ollama calls.

I even gave Chroma the boot. It was fine for quick prototypes, but once my index hit 100k vectors, the memory usage just ballooned. Switching back to a simple FAISS index on disk was faster, lighter, and hasn't crashed once. 

Now my environment is clean, my laptop boots fast, and I’m shipping twice as quickly because I’m not babysitting CUDA versions or fighting framework black boxes.

Next time you’re tempted to add a new orchestration library, try writing the logic in raw Python first. If it takes fewer than 50 lines to handle your prompts and tool calls, you don't need a framework, you just need a script.


r/AISystemsEngineering • • Sep 17 '26

Cut my agent rule files from ~70k tokens to ~2k per session and built a spec gate that refuses vague work. Here's the whole setup.

31 Upvotes

Solo founder, been running Claude Code, Codex, Cursor and Grok on the same repos for a year. This week I measured what I'd built up and it was bad: 64 Claude rule files, 65 Cursor .mdc files (55 always-apply), 24 skills duplicated in three directories, a 30KB machine-level CLAUDE.md. Roughly 70k tokens loaded before the first prompt on Claude, 50k on Cursor. A third of the context window gone, and the docs say adherence drops past 200 lines anyway. So the rules were both expensive and ignored.

What I did, in case it saves someone the week:

1. One instruction file. Codex, Cursor, and Grok Build all read AGENTS.md natively. Claude Code doesn't, but CLAUDE.md supports u/import, so CLAUDE.md is literally one line: u/AGENTS.md. Root file is under 90 lines. Per-project files are 5-line stubs that only load when you're in that folder. Everything procedural became a SKILL.md, which all four tools support and which costs about 80 tokens at startup until triggered. Codex silently truncates combined instruction files at 32KB so keep the shared one small.

2. Hooks for anything that's a rule. "Never push to main" in a markdown file is a suggestion. A PreToolUse hook that exits 2 on a git write verb is a wall. I wrote one guard script, wired it into Claude's settings.json, Codex's hooks.json and Cursor's hooks.json, then mutation-tested it: removed each check one at a time and confirmed the matching test went red. Found two real bugs that way (PowerShell prepends a BOM to piped stdin and my guard was failing open on unparseable input; and the regex matched "git commit" inside echo strings). Test your guards by trying to beat them.

3. Split design from build across machines. Desktop has no push credential, no toolchain, and a hook that denies writes outside the design folder. It reads code through a read-only share from the VM. The VM is the only writer. The handoff between them is a single spec file dropped in an outbox folder. If you can't push, you can't drift into "I'll just fix it real quick."

4. A spec gate that refuses underspecified work. This is the part I'd actually pay forward. Skills, in order:

  • grill — one question at a time, won't accept "it depends", writes answers straight into the spec. Asks appetite (days before you cut scope) and no-gos before it lets you dream.

  • premortem — Klein's method. "It's six weeks after ship and it failed, write the story." Three independent passes so the first cause doesn't anchor the others. Any cause that would survive find-and-replace of the feature name gets thrown out. Survivors become requirements or scope cuts.

  • clarify — scans for vague adjectives, flows with no error path, requirements a stranger couldn't judge pass/fail on, numbers with no source. Max 5 questions, each with candidate answers.

  • challenge — runs in a fresh session on a different model than the one that wrote the spec. Three sections: Wrong (with evidence), Weak (would pass the gate and still fail a user), Missing. Every finding must carry a fix. I have to mark each one accepted, rejected with a written reason, or deferred.

  • spec-ready — 12 checks. Requirements in EARS syntax ("WHEN x THE SYSTEM SHALL y"), every flow has a mockup and an error path, every requirement covered, rubric for an evaluator that isn't the builder, zero TBDs. Reports gaps; only I can mark it ready.

  • graph-author — compiles the spec into a task graph as JSON and runs a 40-line stdlib Python checker: DAG, every dep resolves, every requirement cited by a node, every node has a guard that can be made to fail. The VM runs the same checker on intake and rejects on any path that doesn't exist at HEAD.

  • retro — reads the rejections and evaluator failures, traces each to the spec section that caused it, appends one line to a lessons file. grill reads that file first on the next idea.

5. On the build side: builder and evaluator are different providers with no shared context (Anthropic's harness posts are right about self-grading), guards are proven red-then-green not just green, and there's a fixed escalation ladder driven by config profiles rather than vibes: same failing packet goes medium → high → different model → stop and write a blocker.

Sources, since none of this is original: GitHub Spec Kit (clarify/checklist/analyze), AWS Kiro (EARS + traceability), Klein's premortem (HBR 2007), Anthropic's harness-design and effective-harnesses posts, OpenAI's harness engineering writeup.

My before number is 1 in 4 builds surviving without rewrite. Will report the after once I've got twenty through. If people want the actual SKILL.md files and the checker, send me a dm and ill send them to you. If it helps you save a year lol

UPDATE: I forgot to share the actual file https://drive.google.com/file/d/1e50hcKj8TfjHKM7BXRABX-qoqySYW5q0/view?usp=sharing


r/AISystemsEngineering • • Oct 09 '26

Technical book recommendations specifically on AI agent planning, tool calling & orchestration

26 Upvotes

I’m a software engineer currently working on production AI agent/workflow systems. I already have a strong software/system-design background, so I’m not looking for general system-design books, basic ML textbooks, or broad introductions to LLMs.

I want to go deep specifically on agentic AI architecture, especially:

  • Agent planning and task decomposition

  • ReAct / reflection / replanning loops

  • Tool/function calling and tool selection

  • Designing good tools/interfaces for agents

  • Planner–executor architectures

  • Agent loops: plan → act → observe → replan

  • Workflow vs autonomous-agent design

  • Orchestration and routing

  • Supervisor/worker and coordinator/delegator patterns

  • Multi-agent communication and delegation

  • State, memory and context in agent workflows

  • MCP / A2A where relevant

  • Evaluation and debugging of multi-step agent behavior

  • Failure recovery, stopping conditions and guardrails

I care much more about depth and syllabus relevance than popularity or Amazon ratings. A newer or less-known book is completely fine if it genuinely covers these topics well.

Some books I’ve come across:

  • Agentic Design Patterns — Antonio Gulli

  • AI Agents in Action, 2nd Edition — Micheal Lanham

  • AI Agents: The Definitive Guide — Nicole Koenigstein

  • Building the Agentic Enterprise — Jaroslav Pantsjoha

  • Building Agentic AI Systems — Anjanava Biswas & Wrick Talukdar

  • Systems Thinking for Agentic AI

Has anyone actually read any of these?

If you wanted to become really strong at designing production agent planning/tool-use/orchestration systems, which 1–3 books would you choose?

Also very interested in books I haven't listed. Papers/blogs/courses are welcome too if you think books haven't caught up with the field yet.


r/AISystemsEngineering • • Apr 30 '26

The future of AI isn’t language models — it’s unified multimodal reasoning systems

24 Upvotes

There’s a growing misconception in current AI discussions that progress mainly means “better language models.” In reality, language is only one interface for intelligence—not intelligence itself.

The real frontier is unified multimodal reasoning systems that can jointly process and reason across vision, language, audio, and action in a single coherent framework.

Language models are powerful, but they are fundamentally limited by their format: they operate on sequential tokens detached from the physical world. Even when they appear to reason, they are manipulating symbolic representations rather than directly grounding understanding in perception.

A unified multimodal system changes this. Instead of converting everything into text first, it builds shared representations across modalities:

  • Vision grounded in objects and relationships, not captions

  • Language is directly tied to perception and context

  • Memory that persists across tasks and time

  • Reasoning that operates over world states, not just text sequences

This is closer to how intelligence works in practice: not as text prediction, but as a continuously updating model of the world that integrates multiple information streams.

Many current limitations in AI, such as hallucinations, brittle reasoning, and weak generalization, start to look less like “language issues” and more like representation and grounding issues.

Discussion question:
Do you think scaling language models alone is enough to reach general intelligence, or is multimodal grounding a necessary shift?


r/AISystemsEngineering • • 22d ago

Looking for ideas for genuinely useful AI engineering projects

22 Upvotes

I've been learning AI and software development for a while and have worked with several technologies, but I feel like I'm missing the experience of building software that solves real-world problems.

I don't want to build another generic chatbot, AI wrapper, or project that looks good on a resume but has little practical value. I want to build something genuinely useful that demonstrates my engineering skills and teaches me how real-world systems are designed, built, and deployed.

My current stack includes Python, scikit-learn, PyTorch, LLMs, RAG, LangChain, LangGraph, FastAPI, PostgreSQL, Docker, and Git/GitHub.

I'm looking for ideas that go beyond following tutorials or connecting an API to a frontend. I'd like to work on something with a real use case, meaningful technical challenges, and room to improve it over time.

I'd appreciate suggestions from people working in AI engineering or software development, especially those who have built and shipped real products.

What problems do you think are worth solving? What kinds of projects would help someone develop practical engineering skills and build a portfolio that stands out to employers?

I'm more interested in building one useful, well-engineered project than several generic ones.


r/AISystemsEngineering • • Oct 10 '26

Shapeshifting Software: Exploring an AI-native architecture for self-adapting applications

Post image
18 Upvotes

What if existing software could reshape itself based on natural-language instructions?

Around 8–10 years ago, when I was learning to program, I had this idea inspired by shapeshifters in science fiction.

What if software could change its own UI, workflows, and behavior depending on what a user needs?

Not just through configuration options, but through natural-language instructions.

Imagine telling an existing CRM:

Reorganize the dashboard for our sales team so customers awaiting follow-up appear first. Also, create a reminder workflow for customers who haven't responded in three days.

Instead of requiring a developer to implement these changes, the application would use its existing capabilities to make the changes, validate them, and provide a way to roll back if something goes wrong.

I've been calling this idea Shapeshifting Software.

The interesting part isn't simply adding an AI chatbot to an application. I'm thinking about an architecture where AI is built into the application from the beginning, with a controlled set of tools and capabilities it can use to modify the system.

Conceptually, this is similar to how coding agents like Claude Code and OpenAI Codex interact with codebases through tools, instructions, permissions, and execution workflows.

The application would define what the AI can change, what must remain protected, how modifications are tested, and when human approval is required.

A few days ago, I started exploring this idea through an architectural proposal, an AI agent skill file, and a small demo.

GitHub: https://github.com/TriptoAfsin/shape-shifter

This is an early-stage experiment, not a production-ready implementation. I'm interested in understanding whether the underlying architectural approach makes sense.

Some questions I'm trying to work through:

  • Configuration vs. code: Should an adaptive application modify declarative configuration, generate or modify source code, or combine both approaches?

  • Security: How do we prevent an AI from accidentally weakening permissions, bypassing business rules, or exposing sensitive data?

  • Reliability: How can changes be validated and rolled back reliably, especially when database schemas and interconnected workflows are involved?

  • Architecture: Would a built-in AI harness with explicitly defined capabilities be a good foundation, or is there a better pattern for this?

  • Consistency: How do we make sure similar instructions produce predictable outcomes without making the system too restrictive?

I'm particularly interested in perspectives from people working on software architecture, developer tools, AI agents, and enterprise applications.

Does this seem like a practical direction for AI-native software, or am I overlooking an architectural pattern that already solves most of this?

I'd appreciate critical feedback, alternative approaches, relevant research, and suggestions for improving the experiment.


r/AISystemsEngineering • • Apr 29 '26

Why many RAG projects are still hallucinating

17 Upvotes

I’ve been auditing quite a few RAG codebases lately, and it’s surprising how often the hallucinations creep in even when the setup looks decent on paper.

A lot of the trouble starts with chunking. People are still breaking documents into fixed-size pieces with no overlap whatsoever. That means a sentence can get sliced right down the middle, or an important qualifying detail ends up in a completely different chunk. The model doesn’t get the full picture, so it ends up guessing to make the answer hang together.

I’ve tried switching to splitting on actual sentences and adding something like 100 tokens of overlap. It’s a small tweak, but it gives the model complete thoughts instead of fragments. In the cases I tested, it reduced a good chunk of those made-up answers pretty quickly.

Another issue that shows up a lot is missing metadata filtering. The retriever just grabs any chunks that seem related, even if they come from totally different documents or sections. 

You might get one piece from the beginning of a report and another from way later, and the model tries to stitch them together. That almost always leads to invented connections that weren’t in the original material.

Putting in basic filters, like keeping everything tied to the right filename or section header, helps keep the context focused and relevant. It’s not fancy, but it stops a lot of that mixing-and-matching nonsense.

On top of that, most projects don’t test properly. Throwing in a line like “be accurate” in the prompt doesn’t do much in practice. What actually helps is putting together a small set of real questions (maybe 20 or so) that you know the correct answers for, then using another LLM to judge whether the generated response sticks faithfully to the retrieved sources. 

Without that kind of check, it’s hard to know if your system is really solid or just lucky on the easy cases.

When it comes down to it, making RAG reliable has less to do with picking the newest model and more to do with cleaning up these everyday parts, better ways to split the text, smarter retrieval rules, and honest evaluation that catches problems early.

If your RAG starts hallucinating on a question, my first move now is to look at the chunk boundaries. If a key fact is split between two chunks, the model never really had everything it needed, so it’s no wonder it starts filling in the blanks.

Have any of you dealt with hallucinations that were tricky to track down? What fixed it for you?


r/AISystemsEngineering • • Apr 14 '26

Anyone else noticing how automation is changing real estate?

17 Upvotes

Automation isn’t just “making real estate faster,” it’s quietly reshaping the information asymmetry layer that the entire industry runs on.

A few shifts that stand out:

  • Pricing discovery is getting compressed: Algorithmic valuation tools and automated comps are reducing the gap between listed price and perceived fair value. That tightens negotiation margins, especially in high-liquidity urban markets.

  • Brokerage roles are being unbundled: Tasks like listing syndication, lead qualification, and basic client matching are increasingly automated. What’s left for humans is either high-trust advisory or edge-case deal structuring.

  • Deal flow is becoming data-driven, not relationship-driven: Institutional buyers already use automated pipelines for identifying undervalued assets. This reduces the advantage of local knowledge in many segments.

  • Due diligence is getting systematized: Title checks, risk scoring, rental yield projections, and even tenant screening are increasingly automated, which reduces transaction friction but also standardizes outcomes.

  • Market velocity increases in transparent segments: When pricing and risk signals become machine-readable, good deals don’t stay “undiscovered” for long.

That said, the biggest bottleneck is still not execution; it’s regulatory fragmentation and physical-world constraints. Automation smooths the information layer, but real estate is still anchored in local law, zoning, and physical scarcity.

So what’s emerging is a split market:

  • highly automated, liquid segments (rentals, standard residential, REIT-like assets)

  • and slow, relationship-heavy, regulation-bound segments (development, commercial edge cases, land plays)

Curious — are you seeing automation mostly impact pricing efficiency, or is it already changing how deals are actually sourced and closed in your experience?


r/AISystemsEngineering • • Oct 12 '26

We deployed 150+ AI agents across 8 departments without failing enterprise security

16 Upvotes

About six months ago, we walked into an enterprise software company where internal AI work had basically hit a wall with different teams were running their own experiments with LLMs but nothing was making it into production.

Security was blocking deployments, people were accidentally building the exact same internal tools twice and leadership couldn’t tell if any of it was moving real numbers.

Fast forward six months, they now have 179 agents and 185 automation workflows running live across 8 departments with a clean compliance audit across the whole setup.

The biggest thing that made this work was killing vanity metrics early on where we told teams to stop reporting prompt counts or how many people were testing a bot and instead every single proposed workflow had to prove one number before anyone wrote code: estimated hours saved per year cause that gave us an objective priority queue.

On the engineering side, automating regression test cycles saved roughly 80,000 hours a year and over in RevOps, handling sales credit resolution dropped from 24 hours down to 2 so saving another 22,000 hours.

We also avoided the usual mistake of setting up a centralized AI steering committee because committees just create bottlenecks so we embedded small engineering pods inside individual business units like Product, GTM and Ops.

They built right inside the tools people already used every day like Slack, email and their existing systems and crucially everything stayed inside the company’s own private infra so data never left their security perimeter and internal staff worked alongside us so they could own the code going forward.

To keep InfoSec happy without slowing down work, we ran an automated risk scanner across candidate workflows before anything went live where it checked for data isolation, hardcoded credentials and guardrails across 720 workflows and catching issues in 84% of them before they ever touched production and on top of that, agents were given strict boundaries so the routine data tasks ran on their own but any sensitive financial change or external action required a human to sign off.


r/AISystemsEngineering • • May 08 '26

Why aren’t more companies building internal RAG systems over their microservices/codebases?

14 Upvotes

We already have pretty powerful local/open-source coding models now, and it’s possible to build a custom RAG over all repos of a company.

So why aren’t more engineering teams doing this internally?

Imagine:

  • indexing all microservices

  • connecting architecture docs + codebase + APIs

  • asking questions like: “Where is payment retry handled?” “Which service publishes this Kafka event?” “What breaks if I change this DTO?”

With decent local models + RAG, this feels very achievable now.

What are the biggest blockers in real companies?

  • security/privacy?

  • infra cost?

  • hallucinations?

  • poor retrieval quality?

  • engineering effort to maintain it?

  • developers not trusting AI enough yet?

Curious to hear from teams who’ve actually tried building internal AI coding assistants over their repo.


r/AISystemsEngineering • • 11d ago

Agent factories vs. one agent with clear goals and a map of your context: where we've landed

12 Upvotes

There's a pattern going around called AI factories (agent factories, software factories). For anyone who's unfamiliar: you split the work into fixed stages or roles, often they each have their own folder of skills and guidance, and each stage hands a reliable output to the next. A video factory might have a researcher agent and then a scriptwriting agent, etc., sometimes with an orchestrator agent on top deciding when to spin up each one.

We run an agency and do most of our client work through Claude, so I've been thinking about where this fits for us. I want to share where we've landed and would love to know how other people are handling it.

Where enforced stages make sense for us: steps where the AI has to follow a rigid rule or give output in a very specific format. Compliance requirements, brand standards, a handoff that has to happen the same way every time. Our article process, for example, has enforced stages to make sure we aren't linking to competitors and that all of the facts we cite are accurate.

Where I don't think factories fit: most of the rest of the work. A factory decides in advance how the work is split and what context each stage gets, and each stage only sees what the stage before it handed over. Claude Code's own sub-agent docs say each sub-agent "starts with a fresh, isolated context window," and they say to keep the work in the main conversation when "Multiple phases share significant context, such as planning, implementation, and testing."

What we do instead:

- Clearly defined goals: what the finished work needs to achieve and what done looks like.

- Guidance, context and data organized so the AI can find it. Each client has their own repo that works as their knowledge base, and the AI gets a map of what lives where so it can pull in what a task needs.

- Enforced stages only for the steps that need a rule.

Opus 5.5 doesn't need you to tell it exactly what context to hand over to a sub-agent. As long as it understands the goal and has a clear map of the context and data it can pull in, it's really good at putting together a plan, spinning up sub-agents and giving them just the context they need.

Anthropic's guidance points the same way. Their "Building effective agents" recommends "finding the simplest solution possible, and only increasing complexity when needed." Their post on their multi-agent research system says multi-agent systems use about 15x more tokens than chats. OpenAI's practical guide to building agents says, "Our general recommendation is to maximize a single agent's capabilities first."

Factory patterns can be very useful, for rigid processes, at least for now. They're also useful for helping us understand what the AI is doing.

But to me, factories are another example of us trying to get AI to work the way humans do. As AI gets smarter and more capable, I believe frameworks like these are going to hinder it more than help. AI agents are not factory workers.


r/AISystemsEngineering • • May 01 '26

Why Most “AI Systems” Are Just Automation + Analytics

12 Upvotes

A lot of businesses today describe their workflows as “AI-powered.” Still, when you look closely, most systems are really just combinations of automation and analytics with very little actual intelligence involved.

Here’s the simplest way I separate the three layers:

  • Automation → executes repetitive tasks through predefined workflows

  • Analytics → tracks performance, conversions, and operational outcomes

  • Intelligence → adapts decisions dynamically based on context and intent

For example, in a sales workflow:

  • Automation sends follow-up emails and updates CRM records

  • Analytics measures open rates, meetings booked, and pipeline performance

  • Intelligence decides which lead should receive attention, what message should be sent, and when the next action should happen

The interesting part is that many organizations invest heavily in workflow tools, dashboards, and integrations, but their systems still operate like rigid rule engines. They can execute tasks quickly, but they struggle to adapt when customer behavior or business context changes.

On the other side, relying too heavily on AI reasoning without structured workflows can also create operational problems. Systems become unpredictable, difficult to monitor, and hard to scale consistently.

That’s why I think the strongest AI setups combine all three layers:

  • automation for execution,

  • analytics for visibility,

  • and intelligence for adaptability.

Without that balance, most “AI systems” are either overengineered automation stacks or unreliable autonomous experiments.

Discussion Question:

Do you think most companies today are building genuinely intelligent systems, or are they simply rebranding advanced automation as AI?


r/AISystemsEngineering • • 8d ago

The Rise of the One-Person Institution

Thumbnail
choosingvictory.com
11 Upvotes

Most white-collar professionals I know are using AI incorrectly.

Because white papers, reports, and press materials are still the coin of the realm for knowledge work, many people are applying AI narrowly—to help with specific steps in producing their next individual piece of content.

For example, a research assistant might query a chatbot for an answer to a specific historical question, track down a piece of data, or find an appropriate citation to weave into a report; or a marketing director might use AI to generate the first cut of a press release.

I don’t believe this is going to remain tenable for much longer. I would say Astra 6 and Fable 5.1 are nearly as capable as I am at producing frontier knowledge-work. We are at a point now where a single well-crafted prompt can elicit a detailed, coherent, 20-page report that exceeds the quality of what a human consultant could produce over several weeks.

Rather than throw up my hands in dismay at my personal inability to compete with AI, I have spent 2026 adapting to what I believe will become a norm for white-collar work: using AI to build systems and tools that assist with the mass-production and assortment of knowledge in my particular domain expertise.

The result has been a transformation of my professional identity: From writer-and-analyst to founder-and-engineer of autonomous institutions that run themselves.


r/AISystemsEngineering • • Apr 03 '26

Anyone else noticing how automation is changing real estate?

11 Upvotes

Automation isn’t just “making real estate faster,” it’s quietly reshaping the information asymmetry layer that the entire industry runs on.

A few shifts that stand out:

  • Pricing discovery is getting compressed: Algorithmic valuation tools and automated comps are reducing the gap between listed price and perceived fair value. That tightens negotiation margins, especially in high-liquidity urban markets.

  • Brokerage roles are being unbundled: Tasks like listing syndication, lead qualification, and basic client matching are increasingly automated. What’s left for humans is either high-trust advisory or edge-case deal structuring.

  • Deal flow is becoming data-driven, not relationship-driven: Institutional buyers already use automated pipelines for identifying undervalued assets. This reduces the advantage of local knowledge in many segments.

  • Due diligence is getting systematized: Title checks, risk scoring, rental yield projections, and even tenant screening are increasingly automated, which reduces transaction friction but also standardizes outcomes.

  • Market velocity increases in transparent segments: When pricing and risk signals become machine-readable, good deals don’t stay “undiscovered” for long.

That said, the biggest bottleneck is still not execution; it’s regulatory fragmentation and physical-world constraints. Automation smooths the information layer, but real estate is still anchored in local law, zoning, and physical scarcity.

So what’s emerging is a split market:

  • highly automated, liquid segments (rentals, standard residential, REIT-like assets)

  • and slow, relationship-heavy, regulation-bound segments (development, commercial edge cases, land plays)

Curious — are you seeing automation mostly impact pricing efficiency, or is it already changing how deals are actually sourced and closed in your experience?


r/AISystemsEngineering • • Oct 17 '26

Right now a million developers are making software factory patterns..

10 Upvotes

Just like a million of us built memory mcps, ralp loops and subagent workflows. We were spurred on and influenced by social media that is directed by machine learning algorithims, influenced also by our own engagements with algorithms via the existing agentic coding harnesses.

They are all fed back into the machine, which will select the best one.

This is machine learning in reverse.


r/AISystemsEngineering • • Jul 07 '26

Archestra V1.3 brings a way to connect a fleet of Claude Code, Codex, and other agents to LLM & MCP Gateways + deliver Skills

Post image
10 Upvotes

r/AISystemsEngineering • • Apr 19 '26

Reducing LLM context from ~80K tokens to ~2K without embeddings or vector DBs

10 Upvotes

I’ve been experimenting with a problem I kept hitting when using LLMs on real codebases:

Even with good prompts, large repos don’t fit into context, so models:

  • miss important files

  • reason over incomplete information

  • require multiple retries


Approach I explored

Instead of embeddings or RAG, I tried something simpler:

  1. Extract only structural signals:

    • functions

    • classes

    • routes

  2. Build a lightweight index (no external dependencies)

  3. Rank files per query using:

    • token overlap

    • structural signals

    • basic heuristics (recency, dependencies)

  4. Emit a small “context layer” (~2K tokens instead of ~80K)


Observations

Across multiple repos:

  • context size dropped ~97%

  • relevant files appeared in top-5 ~70–80% of the time

  • number of retries per task dropped noticeably

The biggest takeaway:

Structured context mattered more than model size in many cases.


Interesting constraint

I deliberately avoided:

  • embeddings

  • vector DBs

  • external services

Everything runs locally with simple parsing + ranking.


Open questions

  • How far can heuristic ranking go before embeddings become necessary?

  • Has anyone tried hybrid approaches (structure + embeddings)?

  • What’s the best way to verify that answers are grounded in provided context?


Docs : https://manojmallick.github.io/sigmap/

Github: https://github.com/manojmallick/sigmap


r/AISystemsEngineering • • Oct 04 '26

The exact architectural framework we used to fix a stalled enterprise AI workflow (and drop manual task handoffs by 80%)

9 Upvotes

A few months ago, we audited an enterprise workflow that had hit a wall. The company spent significant engineering time trying to automate their post-sales handoffs, meeting summaries, and CRM updates using custom AI prompts, but adoption was near zero due to edge-case errors, slow execution, and security pushback.

Here is the exact architectural shift we implemented to bring error rates below 1% and get the system live in under 3 weeks:

  1. Replaced single-prompt logic with deterministic routing Instead of feeding raw meeting transcripts or call audio into a single context window, we split the process into discrete, task-specific AI agents (entity extraction, decision mapping, CRM payload formatting). If an agent falls below a strict confidence threshold, it routes to a human-in-the-loop fallback instantly rather than guessing.

  2. Built a zero-data-retention security pipeline Enterprise legal teams rightfully push back on multi-tenant cloud logging. Moving processing to a VPC / local-first framework with zero-data-retention policies reduced compliance review from 6 months to 48 hours.

  3. Formatted for database webhooks instead of human summaries Paragraph-based meeting summaries create administrative clutter. Real leverage happens when the AI outputs structured JSON objects that push straight into CRMs, databases, and task runners automatically.

The result: Handoff processing dropped from 45 minutes down to under 30 seconds per record, with complete data privacy.

If you’re currently building or auditing custom AI workflows, agentic automation, or voice integrations in-house, what’s your biggest bottleneck right now—prompt drift, security compliance, or integration stability? Happy to answer technical questions in the comments.


r/AISystemsEngineering • • Apr 24 '26

When will LLM‑based customer‑support agents actually feel like ‘helpful teammates’ instead of broken chatbots?

9 Upvotes

Right now, most LLM-based customer support agents sit in a weird middle ground: they’re better than keyword chatbots, but still inconsistent enough that users don’t fully trust them as “teammates.” The gap isn’t just model capability; it’s system design, tooling, and accountability layers around the model.

What’s improving fast:

  • Better retrieval systems (RAG) pulling from real-time, company-specific knowledge bases

  • Tool use (CRM access, order lookup, refunds, ticket creation) instead of just text generation

  • Conversation memory within sessions, so users don’t repeat context

  • More structured workflows (escalation rules, confidence thresholds, fallback routing to humans)

What still holds them back:

  • Hallucinations under edge cases or incomplete data

  • Weak context persistence across multiple support channels

  • Lack of true “state awareness” (they often don’t understand what has already happened in the system)

  • Poor handling of ambiguous or emotionally charged cases

  • Integration gaps with legacy enterprise systems, which force brittle workarounds

The shift toward “helpful teammate” behavior will likely happen when agents stop being just language models and become orchestrated systems, LLMs + tools + strict business logic + real-time data pipelines + monitoring. In practice, that means the AI isn’t deciding everything; it’s coordinating actions inside well-defined boundaries.

A realistic timeline:

  • Basic “teammate-like” behavior for simple workflows: already happening in some SaaS and e-commerce systems 

  • Reliable enterprise-grade agents with low hallucination rates: likely 2–4 years

  • Fully autonomous, high-trust support agents across domains: longer, because governance and risk tolerance matter more than raw model capability

The bottleneck is less “can the model do it?” and more “can companies safely let it do it end-to-end?”

Discussion question:
What do you think is the bigger blocker right now, model reliability, or companies being too slow to redesign their support systems around agents?


r/AISystemsEngineering • • Apr 13 '26

What’s your current stack for building agents (LangChain, LlamaIndex, custom), and why?

9 Upvotes

A practical stack for building AI agents today is typically hybrid, using frameworks where they accelerate development, and custom layers where control and reliability matter.

1. LangChain (selective use)

Useful for quick prototyping, chaining tools, and setting up basic agent workflows. However, its abstractions can become restrictive in complex, production-scale systems.

2. LlamaIndex

Strong for building RAG pipelines, handling document ingestion, indexing, and retrieval. It simplifies connecting agents to structured and unstructured data sources.

3. Custom orchestration layer (core layer)

Most production logic sits here:

  • Task planning and execution flows

  • Memory management (short-term and long-term)

  • Tool and API integrations

  • Error handling, retries, and guardrails

4. Vector databases (Pinecone, Weaviate, FAISS)

Power semantic search and long-term memory, enabling agents to retrieve relevant context efficiently.

5. Model layer (OpenAI + open-source LLMs)

Closed models for reliability and performance; open-source models for flexibility, control, and cost optimization.

Why this approach?

Frameworks help move fast, but production agents require deeper control, observability, and stability. A custom layer ensures better handling of edge cases, scaling challenges, and long-running workflows, while still leveraging frameworks where they add speed.

Curious to hear: are most teams over-relying on frameworks, or is building custom orchestration becoming the real standard for serious agent development?


r/AISystemsEngineering • • Feb 24 '26

AI Memory Isn’t Just Chat History, But We’re Using the Wrong Mental Model

9 Upvotes

People often describe AI memory like human memory:

  • Short-term

  • Long-term

  • Episodic

  • Semantic

Helpful analogy, but technically misleading.

Models built by companies like OpenAI, Anthropic, and Google DeepMind are actually stateless.

They don’t “remember.”

What feels like memory is usually a stack of systems:

  • Context window (temporary buffer of recent messages)

  • Persistent storage (saved preferences/account data)

  • Retrieval systems (RAG) that search past conversations and inject relevant pieces back into the prompt

If stored data never gets retrieved and injected into the model, it’s not really memory; it’s just an archive.

Maybe the real question isn’t:

“Does AI remember like humans?”

But:

“What should be retrievable, and under what limits?”

Should AI memory decay? Be user-owned? Be transparent?

Curious what you think.


r/AISystemsEngineering • • Oct 14 '26

Need Help Building an AI-Powered Gmail Task Management System

8 Upvotes

Hey everyone, I’m trying to build a personal AI powered sales assistant that connects to my Gmail, automatically reviews incoming and outgoing emails, and creates actionable tasks based on conversations.
The goal is to have a system that can:
● Automatically scan Gmail throughout the day using read-only access.
● Identify unanswered emails, follow-ups, customer requests, deadlines, and important conversations.
● Use AI to understand context and determine whether something actually requires action.
● Automatically create and organize tasks in a private web-based dashboard.
● Prioritize tasks based on urgency, importance, and customer activity.
● Track completed tasks and recognize when an email response resolves an existing task.
● Generate morning and afternoon task summaries, along with weekly sales activity reports.
● Allow me to manage everything through a simple dashboard without manually entering tasks.

One major requirement is that the AI must never modify Gmail, send emails, create drafts, or contact customers. Gmail access needs to remain strictly read only. The system should only create and update tasks within its own database.

I’ve been experimenting with ChatGPT, Gmail integrations, a custom web app, and MCP servers, but I’m struggling to get everything working so the process works automatically.

Ideally, I’d like to accomplish this without building an overly complicated infrastructure or paying for several different subscriptions.

Has anyone built something similar or know the simplest, most reliable way to accomplish this? I’m open to suggestions on tools, integrations, or alternative approaches.


r/AISystemsEngineering • • Sep 10 '26

Sometimes I look at what I built and yell "Black Magic"

8 Upvotes

I moved from "AI helps me write scripts" to "AI is the operator" for most of day-to-day security and IT ops. Concretely:

Autonomous SOC. Every SIEM incident now hits an LLM triage pipeline before a human ever sees it — full investigation, classification, and auto-close with an audit-trail comment marking it as agent-closed vs. human-closed. It's running at roughly a 70%+ auto-close rate on real incidents, sub-30% false-positive rate on the underlying detection rules. I still review the closures, but I'm reviewing decisions, not doing the investigation.

Identity & access automation. Conditional access exception grants (the "I'm traveling, temporarily allow this login pattern" kind of request) used to be a ticket-and-wait process. Now it's a bounded, self-expiring workflow — the agent grants the exception, sets its own revocation timer, and audits itself daily to make sure nothing lingering slipped through. Same pattern for stale account sweeps, app registration hygiene, and OAuth consent triage — categories of cleanup that used to get done once a quarter if someone remembered, now running on a schedule with zero human toil.

Device lifecycle. New machine shows up → gets enrolled, gets its policies, gets renamed to convention, gets reported on — one pipeline, no human touching a console. Hundreds of endpoints have gone through it.

Helpdesk. Tiered, least-privilege agent access into the ticketing system handling the boring 80% of requests — password resets, access requests, routine break/fix — with escalation paths for anything that needs a human judgment call.

The part that actually took the work: none of this is one giant prompt. It's a knowledge base of dozens of narrow, composable skill modules — each one covering a specific system (the identity provider, the SIEM, the endpoint management layer, the network gear, whatever) with just enough operational detail that the model doesn't have to guess. The system is only as good as that knowledge base, and building/maintaining it has been the actual job. The automation is downstream of that.

The economics, since that's what people always ask: at current token pricing, running this stack costs meaningfully less than a single junior-to-mid IT/security hire's salary — and it doesn't sleep, doesn't context-switch, and doesn't forget the runbook. That's not a knock on the humans I've worked with, it's just what happens when the marginal cost of "investigate this alert" drops to fractions of a cent.

The scary/exciting part isn't any single automation — it's that the aggregate has quietly become the primary operator for a real, live, multibillion-dollar production environment, and most days it does the job better than the version of this that only had a human doing it manually.