asked cursor's cli agent to merge a branch and clean up the worktree. it was using superpowers' finishing-a-development-branch skill, which says don't delete files that were never committed, ask first.

git wouldn't remove the worktree because of my uncommitted NOTES.md and a scratch script. so the agent just deleted them so it wouldn't need --force. didn't ask.

happened 24 of 24 runs. we fixed the skill and re-checked it with skill gate: 0 of 24 deleted anything, and every run asked first. opened a PR on superpowers too. links in the comments.

Hi everyone,

Please help me out or at least point me in the right direction.

Since the whole ordeal started with Microsoft I completely lost ability to do anything with GitHub on ChatGPT in Codex.

It sits there connected and it sees the repository, but all I get is:

"Retried just now. Workspace commands and ZIP download still fail; the status service reports offline / pending. GitHub tools are also unavailable here."

I tried disconnecting and reconnecting GitHub, I tried clearing all browser cache and cookies. Ive tried logging out and in to ChatGPT.

Any help is welcome, as I am at my wits end with this already.

Disclosure: I'm the author. Both tools are free, GPL-3.0 (it covers the tools, not your code), no paid tier.

TL;DR: Rusta, a terminal coding agent in Rust built for DeepSeek, GLM and Kimi, plus Devscriptor, a local code-graph MCP server, resolved 75–81% of SWE-bench Multilingual with one attempt per task. The DeepSeek run cost $13.98 for all 300 tasks. All 900 runs are published.

| Model | Resolved | 95% CI | Cost |

|:--|:--|:--|:--|

| DeepSeek V4.1-Flash | 81.3% (244/300) | 76.5–85.3% | $13.98 total, API |

| GLM-5.3 | 78.7% (236/300) | 73.7–82.9% | GLM Coding Plan Max |

| Kimi K2.8 Preview | 75.3% (226/300) | 70.2–79.9% | Kimi Code subscription |

That's under 5¢ a task for DeepSeek; 97.8% of its 967M input tokens were cache hits. Official harness (swebench 5.0.2), max effort, Nov 9–11, no web access, git history cut at the base commit. Every model cleared 50% in every language (Go was hardest). DeepSeek V4.1-Flash (MIT) and GLM-5.3 have open weights; Kimi K2.8 Preview doesn't. The runs used the vendors' APIs.

Why a dedicated agent: generic OpenAI-compatible clients often drop the reasoning these models produce between turns (DeepSeek returns HTTP 400 if a request with tools omits it) and send sampling or effort fields each vendor handles differently. Rusta speaks each API natively: preserved thinking, per-vendor effort and sampling, explicit output caps. Any OpenAI-compatible endpoint works too, local vLLM included, with thinking preserved.

Devscriptor works with Claude Code, Cursor, Cline or any MCP client, not only Rusta:

- Your repo as a knowledge graph in one SQLite file: 26 languages and frameworks, calls, inheritance and imports resolved across files, kept current by incremental scans and a watcher

- 75 MCP tools, 38 of them LSP-backed (21 language servers)

- Semantic search with a local embedding model, no API keys

- Agent memory (decisions, standards, past failures) that persists across sessions

- No telemetry, no backend; works offline once the model is cached

75 tools, but not 75 schemas in your context. Rusta keeps only the twelve that find and follow code in its tool table (search, definitions and references, relationships, index status, Context Graph), about a sixth of the full schema size. The other 63 are listed by name and a one-line description until the model loads one with `select_tools`. In Claude Code, tool search defers MCP tools by default, and Devscriptor's server instructions name the same twelve to load first. (That's Rusta 1.0.1; the benchmark ran 1.0.0, which kept all 75 on demand.)

What the numbers don't show (the results page covers each):

- How much Devscriptor adds on its own: there's no run of the same tasks without it

- A ranking against other agents or the official leaderboard: different agent, newer models, bigger budget

- Zero contamination: the fixes are public, and build/test leftovers in the images may have hinted at where some fixes belong (worst case 3–9 points)

Single binaries for Linux, macOS and Windows.

- Devscriptor: https://devscriptor.com/

- Rusta: https://devscriptor.com/rusta/

- Results, re-scoring steps, all trajectories: https://devscriptor.com/benchmarks/

- Source: https://github.com/vladimirpesic/devscriptor · https://github.com/vladimirpesic/rusta

I'm building an AI operations system for a small business.

My current setup works, but it's all manual:

- Builder: Codex (GPT)

- Planner: Claude Opus in a chat (decisions, scope, writing the prompts)

- Reviewer: Claude Code CLI checks every change before it goes live

- Sometimes a second model does an extra review

The problem is I'm the messenger. I copy the planner's prompt into Codex, copy Codex's reply to the reviewer, then copy the review back to the planner. It's slow, and it falls apart when I'm busy.

What I want:

- All the agents in one place, passing work to each other

- Builder and reviewer from different AI companies, so they don't share blind spots

- A reviewer and a security checker running on every change

- A "scout" agent that keeps an eye out for new tools, repos, model releases and security issues that could improve the build

- Me approving anything that goes live. That part's non-negotiable

I'm on Windows, I already pay for ChatGPT (Codex) and Claude, and the project lives in a local git repo (private client data, so no public GitHub).

What are you actually using for this? Real-world experience, good or bad, would help a lot, especially on keeping it safe when agents talk to each other without me in the middle.

Designing good systems is becoming extremely important even for engineers who are very early in their careers. Even if you are not actively looking for new roles, getting very good at system design is highly recommended so that you can design your systems well. This is how you go one step closer to transforming from a junior/mid-level to a senior engineer.

Most of us, when asked about getting better at system design, usually talk about technical topics like networking, databases, messaging, latency, failure modes, scalability, etc.

However, your technical design is only as good as the requirements, constraints, and invariants you were able to uncover. Therefore, it becomes essential to develop a knack for identifying requirements, hidden assumptions, operations, guarantees, etc. Engineers don't get to practice this skill easily because almost all system design exercises give us clean requirements.

How can AI help you get better at this? Below, I am explaining the method I practice with my fellow engineers.

1/ You can ask AI to choose a system-design problem and give you only a very short description and nothing else.

For example: "Give me a vague, incomplete system-design problem but don't tell me its requirements and constraints. Only when I ask relevant questions, show them to me. At the end, tell me everything I missed".

You can add more details to this prompt as per your level and goals. Let's say AI asks you to design a system that sends notifications via email, SMS or mobile push.

2/ You take the problem statement and ask questions based on your current understanding. You might ask, "What kind of notifications are there? Are they time-sensitive?", "How many notifications should a system handle normally and at peak?", "Does the system have any delivery guarantees?", "What happens in case of a missing or duplicate notification?".

If you are new to this, you might struggle to come up with these questions, but don't worry; this is the learning phase. You are building reps, and with time, you will get better.

3/ When you ask a question, note "What am I uncertain about?", "Based on this answer, can I make a decision, which one and why?", etc. For example: I need peak write rate because I need to decide whether one DB is enough or write partitions are needed.

This step develops the ability to ask good questions and derive decisions from them.

4/ I usually divide the information I have into 3 categories:

a/ Known facts - confirmed by AI

b/ Assumptions - things I am treating as true currently

c/ Unknowns - what I don't know yet but that can change the design

Tbh, I am still surprised that my assumptions allow hidden constraints to creep into my design. This is why writing them is important, which we do in the "assumptions" category.

5/ Once you feel that you have done the best you could, you submit the following:

a/ Discovered requirements (remember, you weren't told about them in the beginning - you came up with them yourself)
b/ Remaining assumptions (it happens often that AI doesn't provide answers to all your assumptions)
c/ System guarantees or invariants
d/ Proposed design
e/ Remaining risks or unknowns

6/ AI now tells what constraints were missed, which questions were too broad, what was not used in the design, contradictions, etc. But be careful with this step; the AI may change/invent details in between. Therefore, it is always a good idea to ask it to show the complete case before the review; you then compare it with answers it gave earlier and note inconsistencies.

Always remember that AI can help you review your design, but it is not the final authority.

7/ Optional: you can repeat this exercise by asking AI to change one condition and keep it hidden so that you can build your reps on the specific problems.

8/ In the beginning, you can have a cheat list under which you would ask questions such as data, scale, invariants, failures, cost, security, etc. With time, you stop using this list and see if those questions are coming naturally to you.

Your learning loop looks like this:

Ask questions -> AI responds -> Update your understanding -> Propose a design -> Show hidden cases -> Check what you failed to discover.

This way, AI can act as your personal mock interviewer and learning partner.

With codex moving to one token allowance shared with chatgpt from december, plans are turning into a token question. most of those tokens are cache reads tho, so a billion token plan is less fresh work than it sounds

for open model plans the per token price is easy to look up, the same weights sit on the labs own api and on deepinfra and a few other hosts

so try one normal workday, note what share of the 5h and weekly windows it used and price that day at those rates. if the plan only covers 2 or 3 days a week its basically prepaid API. how long does yours last on a heavy week with piles of work

Each new worktree starts with an empty target/ (or node_modules), so the first build in it is a cold build. On a big Rust workspace that's 10+ GB of disk and a lot of CPU, for every agent you start.

On a Mac you can skip most of that with an APFS clone:

cp -c -R ../main/target ./target

-c makes cp use clonefile, so the copy shares disk blocks with the original until files change. I measured it on a 9.7 GB target dir: 1.1 seconds, and about 1 MB of new disk used. Cargo then keeps every dependency it already built and recompiles only the workspace's own crates.

The same works for node_modules. On Linux, cp --reflink=auto does this on btrfs or XFS, but ext4 can't.

Two gotchas: du still reports the full size for every clone, so check free space with df instead. And if you're short on disk, removing finished worktrees frees far more than cleaning the main target, which just costs you a full rebuild.

Recently, I was working on two things. One personal, one for work. The personal one was setting up Nanabot, an AI agent that automates the work I do as a programmer. For work, I was writing an MCP server to manage the company's products. And I ran into some problems. When there are a lot of MCP methods, their detailed descriptions really bloat the LLM's context. If you hide them behind Code Mode MCP, the LLM sometimes can't find the MCP method it needs-or doesn't even realize it should look for one.

I learned about Code Mode MCP from a news story about Cloudflare running into a problem with AI agents using up too much context. In the traditional MCP setup, an MCP client can pass the LLM descriptions of all connected tools. Cloudflare has more than 2,500 API methods, and by their estimate, representing all of them as MCP tools would take about 1.17 million tokens. Cloudflare proposed a solution: use Code Mode MCP, which gives the agent just two things instead of thousands of tools: search() and execute().

The first lets you find the API methods you need and get their signatures on request, while the second lets you run JavaScript code that calls those methods. In the implementation they showed, a single execute() call can combine several MCP method calls and return the final result without going back to the LLM between each operation. That can save tokens, but repeated searches and loading signatures also use them. The amount saved depends on the size of the catalog, how often you search, and the number of calls. In my scenario, savings seem likely, but to measure them accurately, you need to test on concrete tasks.

How does this technology work in practice? You tell the AI agent: "Open example.com and take a screenshot of the page." Below is one way to implement an aggregator: search an array of MCP tools, tools. This isn't universal Code Mode syntax. For example, Cloudflare's search() looks for operations in an OpenAPI specification. In this aggregator, search could look like this:

return tools.filter(t =>
  t.name.includes("navigate") ||
  t.name.includes("screenshot")
);

This returns the description:

[
  {
    "name": "navigate_page",
    "description":
      "Navigate by URL/history, or reload.\nDefaults to project URL.",
    "inputSchema": {
      "type": "object",
      "properties": {
        "pageId": {
          "type": "number",
          "description": "Targets a specific page by ID."
        },
        "type": {
          "type": "string",
          "enum": ["url", "back", "forward", "reload"],
          "description": "Navigate by URL or history action."
        },
        "url": {
          "type": "string",
          "description": "Target URL (only type=url)."
        },
        "ignoreCache": {
          "type": "boolean",
          "description": "Ignore cache on reload."
        },
        "handleBeforeUnload": {
          "type": "string",
          "enum": ["accept", "dismiss"],
          "description":
            "Accept/dismiss unload dialogs; default is accept."
        },
        "initScript": {
          "type": "string",
          "description": "Run on each document before other scripts."
        },
        "timeout": {
          "type": "integer",
          "description": "Maximum wait in ms; 0 uses the default."
        }
      },
      "required": ["pageId"]
    }
  },
  {
    "name": "take_screenshot",
    "description": "Take a screenshot of the page or element.",
    "inputSchema": {
      "type": "object",
      "properties": {
        "pageId": {
          "type": "number",
          "description": "Targets a specific page by ID."
        },
        "filePath": {
          "type": "string",
          "description":
            "Absolute or relative save path; don't attach it."
        },
        "format": {
          "type": "string",
          "enum": ["png", "jpeg", "webp"],
          "default": "png",
          "description":
            "Format to save as.\nDefault is png."
        },
        "fullPage": {
          "type": "boolean",
          "description": "Capture full page; incompatible with uid."
        },
        "quality": {
          "type": "number",
          "minimum": 0,
          "maximum": 100,
          "description":
            "JPEG/WebP quality 0-100.\nHigher is larger; PNG ignores it."
        },
        "uid": {
          "type": "string",
          "description":
            "Snapshot element UID; omit to capture the page."
        }
      },
      "required": ["pageId"]
    }
  }
]

In this implementation, the method description is returned on request rather than necessarily being loaded into context in full. The signature shows that pageId is required for both methods. For this example, let's assume the required pageId value was obtained beforehand.

Next, execute is called:

await chrome_devtools.navigate_page({
  pageId: pageId,
  type: "url",
  url: "https://example.com"
});

const screenshot = await chrome_devtools.take_screenshot({
  pageId: pageId,
  format: "png"
});

return screenshot;

Notice that there’s no new request to the LLM between navigate_page and take_screenshot. The Code Mode aggregator runs the next JavaScript statement itself, waiting for the first one to return. I think the benefits are obvious.

Meanwhile, Cloudflare recommends using Skills alongside MCP. Skills explain to the agent when to use Cloudflare’s capabilities and which ones to use. MCP provides access to Cloudflare operations, while Code Mode provides search() and execute() for finding methods and running them through JavaScript.

So I used the same architecture. I hid a large number of MCP methods behind Code Mode. Now the LLM sees just one MCP server that can search for and run MCP methods. But, as I said above, sometimes the AI agent doesn't find the right methods, or doesn't even think to search for them. That's where Skills come in. In their descriptions, I list which MCPs and methods to use. I don't give full descriptions, just brief ones. Here's an example of a Skill that fixes bugs:

---
name: bugfix-process
description: >
  Automatically fix bugs by ticket number.
  Use when the user asks to fix a bug,
  task, or error from the ticketing system.
---

# Bugfix Process

## Available MCP servers and methods

### ticket_mcp - ticketing system
- `get_ticket` - get the ticket description.
- `get_comments` - get all comments.
- `list_attachments` - get the list of attachments.
- `download_attachment` - download an attachment.
- `update_status` - change the ticket status.

### samba_mcp - file server
- `list_files` - find files and logs related to the ticket.
- `download_file` - download a file from Samba.

### filesystem_mcp - file system
- `create_directory` - create a directory.
- `write_file` - write a file.
- `read_file` - read a file.

### git_mcp - working with Git
- `current_branch` - get the current branch.
- `stage_files` - add files to the index.
- `commit` - create a commit.
- `show_commit_diff` - get the commit diff.
- `push` - push changes to the remote repository.

### codex_mcp - programming
- `start_task` - start Codex with a prompt.
- `wait_task` - wait for Codex to finish.
- `get_task_result` - get the result of the task.

### jenkins_mcp - project build
- `trigger_build` - start a Jenkins build.

## Bug-fixing process

1. Identify the ticket number, repository, branch, and Jenkins build
   from the user's request.

2. Gather information about the ticket:
   - Get the ticket description and all comments.
   - Download all attachments and related logs from Samba.
   - Save the materials to `tickets/TICKET_NUMBER/` in the selected
     repository.

3. Create a Git commit containing only the contents of
   `tickets/TICKET_NUMBER/`.

4. Start Codex and wait for it to finish. Give it this prompt:

   "Fix the bug described in the files in the tickets/TICKET_NUMBER/
   directory. Review the description, comments, attachments, and logs.
   Fix the cause of the error. Run the necessary tests and report the
   result."

5. Create a separate Git commit with the changes Codex made. Do not
   include unrelated changes.

6. Push the commit with the fix to the remote repository, using the
   current branch unless the user specified a different one.

7. Now the bug fix begins. `codex_mcp` provides the `start_task`,
   `wait_task`, and `get_task_result` methods for this. The agent passes
   Codex the directory path and a prompt for the fix:

   """
   Fix the bug described in the files in the tickets/731/ directory.
   Review the description, comments, attachments, and logs.
   Fix the cause of the error.
   Run the necessary tests and report the result.
   """

   Codex reads the materials at that path, does the work, and returns
   the result. The main agent waits for the task to finish without
   passing the entire ticket and logs through its own context. This way,
   the main agent coordinates the process and delegates the actual
   coding to Codex. If the MCP interface and time limits allow it,
   waiting for Codex and retrieving the result can be combined in a
   single `execute()` call.

8. Once Codex finishes, the agent reviews the DIFF of the changes with
   `show_commit_diff` and checks the result. It then creates a separate
   commit and pushes it to the current branch with `push`. After the
   push succeeds, it starts the Jenkins build `two_jar_java17` using the
   `trigger_build` method from `jenkins_mcp`, so the build checks these
   exact changes.

9. Once Jenkins finishes, the agent evaluates the fix, Codex's tests,
   and the build result. Using `ticket_mcp.update_status`, it sets the
   status to `Resolve` if the bug is fixed, the tests pass, and Jenkins
   finishes successfully. If the fix did not work or the result cannot
   be confirmed, it sets the status to `In Progress`.

10. Based on the final status, the agent prepares a report. For
    `Resolve`, it saves it to `tickets/731/result.fixed.md`, and for `In
    Progress`, to `tickets/731/result.in-progress.md`. The report lists
    the problem, the changes, the results of the Codex and Jenkins
    checks, and any remaining questions. Its contents are checked
    against the commit DIFF and the build result. The report is added in
    a separate commit and pushed to the same branch.

11. In its reply, the agent gives the final filename and includes its
    contents. That way, the user gets more than just a message saying
    "bug fixed"; they get a full report on the work done. The whole
    process, from receiving the ticket to starting the Jenkins build, is
    described in one SKILL, and the necessary MCP methods are chosen as
    the task progresses.

And what did we end up with?

We split the information an AI agent needs into three levels:

• Persistent context. Contains only brief descriptions of available Skills and the two Code Mode tools: search() and execute().

• SKILL. Loaded when the agent determines it fits the task. Contains the workflow and the names of the necessary MCP servers and methods, but not their full signatures.

• Code Mode MCP. In the implementation described, finds full descriptions of specific methods on request and lets you execute a sequence of calls through JavaScript.

That gives us a kind of layered system for discovering and executing tools.

Previously, the agent had two not-so-great options.

The first was to load thousands of tools into context and spend a huge number of tokens on their descriptions.

The second was to hide the tools behind Code Mode and hope the LLM would figure out on its own which methods it needed and how to find them.

With this setup, there's a third option: SKILL points to what to look for and how to use it, while Code Mode finds the full signatures and executes the operations.

Of course, this doesn't guarantee success. The LLM can still fail to choose the right SKILL, search for the wrong method, or put the JavaScript together incorrectly. And not every operation can be combined into a single call; sometimes the agent needs to analyze an intermediate result and make a new decision.

With this setup, the agent doesn't need to keep detailed descriptions of all available MCP tools in context. Instead, it gets a brief instruction for a specific workflow and loads the full method details only when it actually needs them.

In this setup, SKILL is the guide to MCP capabilities, while Code Mode is the mechanism for discovering and executing them.

In the end, here's the set of SKILLs I ended up with to automate my work as a programmer (the list is, of course, incomplete):

1. Ticket-based bug fixing - gathers the ticket description, comments, attachments, and related logs; coordinates the work on the fix, records the result, and starts a verification build.

2. Ticket implementation - helps take a new feature through the full development cycle: study the requirements and materials, prepare a plan, implement the changes, save the result, and run a build.

3. Working with the ticketing system - opens tickets and shows their descriptions, discussion history, attachments, and key fields.

4. Working with GitLab - helps find projects, browse branches, commits, and files, and carry out routine repository operations.

5. Working with Jenkins - shows available builds, helps start them, and check their status.

6. Working with the file server - finds, downloads, and places files, logs, and materials needed for development and troubleshooting.

I think I'm ready to spend the winter in Thailand. I'll be lying on a sun lounger by the sea or the pool, giving my AI agent orders from my phone. It knows everything now; it can do everything. It knows all the processes, rules, and access permissions.

If AI changed how coding work was done at your job and you ended up leaving, what changed for you?

I've been using Codex extensively for development, currently on the $100/month plan. I wanted to find out whether a cheaper AI coding environment could deliver comparable results without constantly running into usage limits.

Rather than comparing them on a simple coding exercise, I ran three agents against the same substantial task in an existing private application.

The results surprised me, particularly the differences in token consumption and the bugs that survived each agent's own testing.

The setup

The project is a real, fairly large TypeScript/Next.js/Node.js application using Prisma and PostgreSQL.

I used three agents:

• Codex: GPT-6.1 Sol, High reasoning.

• Google Antigravity: Gemini 4 Argon, High reasoning.

• Meta Muse Code: Muse Wonder Full 1.0, High reasoning.

Each received the same detailed development brief, started from the exact same Git commit, and worked in a separate Git worktree on Windows.

The assignment was to build a staff-only Mathematics Teaching-Readiness Dashboard, including a backend API, authorised student access, existing instructional-plan integration, exact lesson-version selection, released-assignment checks, responsive UI, and synthetic testing.

The feature was specifically selected for this experiment, rather than drawn from a confirmed development backlog. It represented realistic full-stack work, although it partially overlapped functionality already under development.

The agents were prohibited from accessing production data, copying each other's implementation, merging or deploying.

The numbers

Codex's elapsed time was captured shortly before its final response. Antigravity reported roughly 48 minutes of active execution, with the remainder including interruptions and waiting.

Important: The passing-test counts cover different collections of tests, including existing regression suites. They are not standardized scores or evidence that 248 tests automatically mean better code than 36 tests.

All three generated working local implementations and reported responsive browser verification.

1. Token consumption was dramatically different

Antigravity recorded approximately 2.05 times Codex's total token activity, and around three times its output tokens.

Muse landed between the two, at about 1.14 times Codex's total recorded tokens.

The caching figures were interesting:

• Codex: 24.17M cached input tokens.

• Muse: 27.18M cached input tokens.

• Antigravity: 48.13M cached input tokens.

Antigravity also reported 2.48M non-cached input tokens.

I normalized the displayed totals to include cached input, but these providers don't necessarily account for tokens identically. More tokens do not automatically mean proportionally higher billing or quota usage.

Also, Antigravity produced a substantially larger implementation, with additional search and filtering features. The token difference cannot be attributed entirely to model inefficiency.

Nevertheless, I think token consumption deserves far more attention in comparisons like these.

2. Subscription quotas matter—and plan prices weren't equivalent

I started with a fresh Google AI Pro allowance.

By the time Antigravity finished, 76% of my weekly Gemini 4 Argon allowance had been consumed.

That was for one major development assignment.

However, this comparison has an important caveat: I was testing a US$100 Codex subscription against a CAD$26.99 Google subscription.

Google offers higher Antigravity allowances on more expensive Ultra tiers. If a 5× quota scaled proportionally, the same benchmark would theoretically consume about 15.2% rather than 76% of that larger weekly allowance.

I did not test Ultra, so that's only a mathematical projection.

Codex's displayed account-wide weekly quota moved from 51% to 52% during its session, but that does not establish exact task-attributable consumption.

I also didn't have a reliable before-and-after Muse quota measurement.

Therefore, I cannot responsibly claim which provider offers the most completed tasks per dollar at equivalent subscription prices.

3. Muse's Windows sandbox was a major problem

Muse produced a substantial working implementation, but its execution experience was extremely frustrating.

Its Windows sandbox repeatedly encountered ACL publication lock timeouts, forcing commands that should ordinarily run automatically into manual approval.

I counted 68 manual approval requests during a single task.

The execution lasted approximately two hours, with a substantial portion of the time spent waiting on approval decisions.

This is an environment/autonomy problem, not necessarily evidence that the Muse model reasons poorly.

I submitted a technical support and refund request because that level of intervention made it impractical for my intended workflow. The outcome is still pending.

Muse also attempted PostgreSQL integration testing, but database setup failed because of historical migration dependencies. It completed unit and frontend tests, not a successful populated-PostgreSQL integration run.

4. All three passed their own tests—but independent code inspection found problems

After completion, I published sanitized implementation-only branches privately and used a separate ChatGPT review to inspect the code.

That review was independent of the three coding runs, but it was not a blind human audit, and I did not independently rerun every test. It should be treated as a source-level review, not production certification.

Here were the most interesting findings.

Codex

Codex produced the smallest implementation, around 236 lines of new backend logic and 99 lines for its primary frontend component.

It avoided a particularly important mistake: it created a separate read-only roster query rather than calling an existing helper that can modify learning-profile records.

It reused the established academic and content-classification services instead of implementing another mastery-evaluation engine.

However, its query omitted the submitted assignment state, even though submitted assignments were previously released and are awaiting review. Its database pagination can also produce partially filled or empty pages because some eligibility checks occur after pagination.

The existing surrounding dashboard still uses an older roster endpoint with potential side effects, even though Codex's new endpoint avoids that issue.

Muse Code

Muse also reused the existing academic recommendation system, which was a positive design choice.

However, its supposedly read-only GET endpoint called a roster helper that can create or update academic learning-profile records.

It also capped a released-assignment query at 500 records, which creates a potential false-negative scenario when a relevant older assignment falls outside that limit.

Its internal reviewers found and corrected several issues, but these problems remained.

Google Antigravity

Antigravity delivered a feature-rich dashboard, including search, preparation-state filtering, summary cards and detailed assignment distinctions.

It successfully executed four disposable PostgreSQL integration tests.

However, it had the same indirect-write problem as Muse.

More concerning, certain missing curriculum or Skill records could result in an academically ready classification rather than an unknown state.

It also processed preparation details across the entire authorised student roster before applying pagination. That could become expensive as the number of students grows.

Its primary backend implementation was around 1,300 lines, compared with 236 for Codex and 421 for Muse.

More code is not automatically bad, but unnecessary complexity increases the amount of logic that must be reviewed and maintained.

5. What I learned

My biggest lesson wasn't that one model is universally better.

It was that an AI coding agent reporting that all its tests pass is not enough.

The differences that mattered most were:

• Whether the agent reused existing authority correctly or inadvertently duplicated it.

• Whether an apparently read-only endpoint had hidden write side effects.

• Whether missing academic information failed safely.

• Whether the agent could work autonomously without constant permission prompts.

• Whether token and quota consumption made the workflow sustainable.

• Whether the implementation remained small enough to understand and maintain.

For my particular Windows setup and application, Codex was the strongest overall performer, especially on autonomy, speed and architectural discipline.

Antigravity demonstrated genuine capability and was more competitive than its cheaper subscription price initially suggested. I would like to evaluate it at equivalent subscription capacity before making a definitive cost comparison.

Muse showed promise in code generation, but its Windows sandbox problems substantially hurt the experience.

Limitations

This was one task, one run per agent, on one private codebase. It was not a standardized multi-task benchmark.

The implementation scopes differed somewhat, the subscription prices weren't equivalent, and the platforms reported usage differently.

The review was source-level and not blinded. None of the three implementations has been merged into production.

I wouldn't extrapolate these results into a universal ranking.

Question for other developers

Has anyone compared these three environments—or similar ones—using actual token usage, weekly quota consumption, independent code review and the number of human interruptions?

I'm particularly interested in whether others have experienced Muse's Windows sandbox problems, and how Antigravity's quota behaves on higher-priced Ultra plans.

I'd be interested to compare experiences beyond the usual "which model feels smarter" discussions.

Three days, 28 comments, 3.1K views. I posted four ownership rules for multi-agent work on Monday, and the comments did what my own testing did not: they broke two of them.

Rule 1 broke first. I said one writer per element, claimed by id. A commenter pointed out the hole: an id is just a string any subagent can pick up, and the board only checks the name. The rule only works if every agent behaves. That is not a rule. That is a hope.

Two fixes came out of the thread, and they cover different halves of the hole. One: the board mints the id, not the agent. The board keeps its own list, writes the id there before handing work to the agent, and reloads it on restart. A write outside that list is a loud refusal, not a silent overwrite. The other: each agent signs its writes with its own key, and the board verifies the signature, so a copied id alone buys nothing. Same rule, different owner: the board, not the agent's good intentions.

Rule 3 broke second. I said a check that cannot fail is not a check, and to test it against a broken copy. A commenter put it sharper: a check that can never return false is decoration. Another one added the enforcement I was missing: a pre-write hook that refuses writes outside the claimed id, so the rule does not depend on the agent obeying it. The check is not the test I run once. It is the hook that says no every time.

The other two rules did not break. They got more specific. Missing data is still "unverifiable," never "pass": the NULL and empty-list threads in the comments kept coming back to the same line, a missing field is not an empty result, and an empty result is not a success. And the reviewer rule held its shape: a reviewer judges its own card's diff and its direct callers, and the card now carries its non-goals in writing, so a legitimate deliverable stops looking like overreach. One commenter put the same fix sharper: write done once per task, so both agents read the same thing instead of each getting its own prompt version of it.

The revised four, as I run them now:

1. The board owns identity. It mints the id, keeps the list, and refuses any write it did not issue. Writes are signed with the agent's own key and verified by the board, so a copied id alone buys nothing. If an agent cannot name its element from the board's list, it does not write.

2. A reviewer judges only its own card's diff and its direct callers. Non-goals go on the card, in writing, and done is written once per task so both agents read the same version. Anything outside that scope is "not mine to judge," never "unauthorized."

3. A check that cannot fail is decoration. Before I trust a guard, I run it against a broken copy and watch it fail, and a pre-write hook enforces it on every write, not just in the test.

4. Missing data is "unverifiable," never "pass." If the field is not there, the verdict is "I cannot see it," not green. An empty result is a result to explain, not a success to report.

The part people do not like hearing: my original four rules were not wrong, they were unenforced, and an unenforced rule is just a wish I wrote down in a nicer font. If your coordination depends on agents behaving, you do not have coordination. You have a polite request with commit access.

Hello guys,

I've been using ChatGPT for about a year now, and I have quite a few different use cases.

I started my career as a graphic designer, then moved into frontend development and UI/UX. Eventually, at the age of 27, I completed another apprenticeship, this time as an e-commerce clerk. So I also handle my own business accounting and taxes.

Here are a few things I've picked up along the way that might be useful to others:

• Convert files into formats that are easier to work with. If you have PDFs or other documents that you need to reference regularly, let ChatGPT extract the information and convert it into a more suitable format. For example, I convert invoices into CSV files and text documents into Markdown. Otherwise, ChatGPT may have to process the original PDF every time, which uses up resources and context.

• Keep a TL;DR version alongside the extracted data. After extracting the information, I also save a summarized version next to it. ChatGPT can usually work with the summary and only needs to look at the CSV or original data when the summary isn't sufficient.

• Use voice dictation to organize your thoughts. Sometimes I just dictate whatever is on my mind, including instructions or potential code changes, and let ChatGPT turn it into a patch list and a to-do list. The patch list contains immediate coding changes, while the to-do list contains things I've noticed but that can wait until later or require a separate task. This way, I can have a discussion first and possibly come up with even more improvements before I let the AI loose. I've also specified that whenever I want actual code changes, I need to explicitly say or write "green light". Otherwise, we just discuss things.

I hope some of this was helpful or gave you a few ideas. In return, I'd appreciate some advice on a problem I'm currently running into.

I'm developing a game, and things are going pretty well so far. I'm writing the core game in Rust and using the Godot engine for everything else.

Where can I cut costs and use my resources more efficiently?

Currently, I'm using the web version of ChatGPT to review and modify my code, implement certain features, and handle GitHub operations. It can pull from and push to my repository, and I then pull the changes onto my desktop for testing.

The problem is that the project has grown to the point where my work context gets filled up after just one to three tasks. I suspect that the GitHub operations are contributing to this overhead as well. At this point, the context can be full after a single request to push my current changes.

For example, I recently had a fairly extensive patch list and additionally asked ChatGPT to split some of the code into separate files to improve modularity. Now the whole process is getting a bit stuck, and I've run out of usage.

I'll try having ChatGPT package its local changes into a ZIP file so I can download them and handle the commits and pushes myself. This is the first time I've run into these limitations. My previous web development projects were never large enough to cause problems like this.

Should I switch to Codex?

I've also considered switching to Codex, since it can work directly on my local machine. That might eliminate some of the GitHub overhead. However, I'm a bit paranoid about giving an AI access to my machine, especially when I can't fully control what it can access or modify.

Would it make sense to run Codex in a separate Linux environment through WSL, or am I overcomplicating things?

What would you recommend? Where do you see the biggest opportunities to reduce context usage and avoid running into these limits?

I run Claude Code and Codex all day, often as multi-agent workflows with dozens of subagents. I could see each session but never the whole picture: what a run actually cost, where agents sat idle, whether a change to my setup helped. So I built coding.sh and pointed it at my own runs.

Workflows and subagents

Across 57 workflow runs (666 subagents, ~$2,500 API-equivalent):

• Broken prompt caches cost $106 extra, about 4% of the spend.

• 41 of those breaks were agents sitting idle at a barrier, waiting for siblings, until their cache expired. That's $21 for doing nothing.

• 40 subagents failed outright.

coding.sh workflows stats shows the critical path, how many agents ran at once and idle time at barriers. The viewer draws each run as a timeline, one bar per agent.

Experiments: does that change actually help?

I A/B tested a popular "be concise" plugin: 10 tasks, 6 runs each, paired, with 95% intervals.

• Replies: 17% shorter

• Output tokens: −8%

• Cost per run: +11% [+7%, +15%]

• Pass rate and clarity: no change

The plugin adds its style guide to every request, and that costs more than the shorter output saves. Most of what a coding agent writes is tool calls and code, not prose.

coding.sh compare and exp run your own tasks with and without a change (a rule, a skill, a plugin, a model) and tell you whether the difference is real or noise.

Your own sessions

Mining 3,600 of my sessions turned up a find-and-replace typo in my AGENTS.md that sent Codex looking for a folder that doesn't exist, in 92 sessions. You can't see that from inside one session.

What it is

• Hooks into Claude Code, Codex, Gemini CLI, OpenCode, Copilot CLI, Aider and more

• Stores prompts, tool calls, diffs, test runs, tokens and cost in SQLite on your machine

• coding.sh improve says what to fix first and whether your last AGENTS.md change helped

• MCP server, so your agent can look up its own past sessions

Free, no account, and nothing leaves your machine. npx coding.sh to try it.

fully open source as well : GitHub: https://github.com/jackulau/coding.sh

there's been a debate this week about whether anthropic silently changed opus 5.5. lots of vibes, not a lot of numbers.

i wanted to check my own logs so i wrote a thing that reads the session files claude code saves locally, groups everything by version, and tells you if output tokens per turn actually shifted. if they did it checks whether the model or effort setting also changed — if not, it blames the release.

npx driftcost

reads ~/.claude/projects, no config, no network calls.

https://github.com/tsaagan/token-drift

only works with claude code logs for now. curious if anyone sees a version-over-version shift in their data or if the perceived nerf is purely a quality thing.

Whenever you hand a larger software task or reference rebuild to a coding agent, two things usually happen:

1. It stops after a stripped-down MVP or a single happy path and declares victory.

2. If a test fails, it modifies the test expectation instead of fixing the code ("vibe verification").

I put together RefTrail (MIT, zero npm dependencies, Node.js >= 22.16) to make agent runs auditable from the outside:

• Bounded Agent Brief (prompts/reference-development.md): Instructs the agent how to research a reference product from authorized sources (Q1–Q4, plus clean-room rules if you use tools like REA for observations), scope a bounded 60-minute end-to-end workflow, and maintain a strict requirements catalog (COVERAGE.csv).

• Immutable Criteria (tools/criteria.mjs): Locks the test inputs and expected outputs as a byte-exact baseline with a SHA-256 digest before the agent writes product code.

• Anti-Cheat Catalog (tools/catalog.mjs): Calculates weighted functional coverage together with mandatory operational gates (--gates). Rounding is capped so 99.96% never prints as 100%, and a missing mandatory check blocks acceptance even at 100% feature coverage.

• Worked Example (showcase/uniq): You can clone the repo and run "node showcase/uniq/verify.mjs" and "node integration/verify.mjs" right away to see a complete cycle (including a criteria revision, Linux/Windows reference runs, and snapshot verification).

Repo (English & German): https://github.com/kidimusicofficial-prog/reftrail

Curious how others here are currently locking down acceptance criteria before letting Claude Code or Codex loose on a repo.

I tripped over this after adding a short CLAUDE.md next to an AGENTS.md I was already using for Cursor and Codex. Claude Code sessions stopped seeing the shared commands and the "ask first" list. The file was fine. It just wasn't being loaded.

Claude Code's memory docs (https://code.claude.com/docs/en/memory) say that, by default, it reads AGENTS.md only when there is no CLAUDE.md, .claude/CLAUDE.md, or CLAUDE.local.md in the working directory or above it. A personal, often-gitignored CLAUDE.local.md counts too, so you can hide the file for yourself while the rest of the team is fine.

The one-line fix is to put @AGENTS.md on line 1 of CLAUDE.md. That's an actual import. A sentence that says "please also read AGENTS.md" is not — Claude only opens it if it decides to. Then run /memory or /context all and confirm both files show up.

What I've been putting in the shared AGENTS.md, and leaving out of the Claude stub:

• The real commands (pnpm test path/to/file, not "run the tests")

• A short definition of done (typecheck / lint / tests were actually run)

• Ask-first boundaries: auth, payments, deploy, anything that hits production

• A gotchas list I only add to when the agent makes the same mistake twice

I've been keeping multi-step stuff (PR review, flaky-test hunts, handoff notes) out of AGENTS.md so it isn't injected into every session. Cursor still gets .mdc rules for file-scoped conventions; Codex only sees AGENTS.md, so anything all three tools need has to live there.

Curious how other people keep one source of truth when they're bouncing between Cursor, Claude Code, and Codex. Are you importing, symlinking, or just maintaining two files?

gemini 4 carbon opened to paid api users yesterday, and carbon and argon list the same price ($2 in / $10 out per million until nov 30). so i ran both through the api on the same tasks to see what it costs in practice.

what i ran: 3 office-style tasks x 3 runs (pull fields out of a messy email, tiered commission math, a 120-word apology email), graded by code, plus 10 hard maths problems with exact answers, one run each. default thinking level for both.

• accuracy: office 45/45 carbon vs 44/45 argon (argon's miss was a 123-word email against a 120 limit). hard set 10/10 both. my tasks weren't hard enough to separate them.

• thinking tokens: carbon 5,239 on the office calls and 10,982 on the hard set. argon 0 and 1,018. visible answers were about the same length.

• cost: office $0.0669 carbon vs $0.0148 argon (4.5x). hard set $0.1461 vs $0.0494 (3x).

• speed: median 9.5s vs 5.9s per call on office tasks. the ln(sin x) integral took carbon 3,836 thinking tokens and 40.9s, argon got it right in 10.1s with no thinking.

• thinking_level low: carbon used 0 thinking tokens, still 45/45 and 10/10, and cost the same as argon ($0.0147 / $0.0459).

limits: one run per hard problem, thinking counts vary a lot between runs (carbon's email task ranged 555 to 1,031), and i didn't test long agentic coding, which is what google says carbon is for. 3 of carbon's 19 low calls took 34 to 54s for no obvious reason.

so my takeaway: don't move an argon workload to carbon for the launch price. if you try carbon, start at thinking_level low and only raise it if answers actually get better. has anyone seen carbon beat argon on real coding tasks where the extra thinking paid off?

Working with AI made me a lot faster than I was before, no question. But lately I’ve noticed that in some cases I’m just sitting there waiting for the response, and it’s kind of become a new bottleneck.

I know I could run 10 things in parallel, but when I try that I actually feel less productive because of the constant focus switching. Around 3 tasks at a time seems to be my limit, but even then there’s a lot of time where I’m just waiting for the AI to finish.

So I’m curious how you deal with it:

• How do you organize your work so you’re not just waiting around?

• Do you have a system for running a few things in parallel without losing focus?

• Did you change how you split or batch your tasks to wait less?

Would be great to hear what works for you, or what doesn't.

TL;DR: I let a cheap model write the code and only pay a frontier model to review the finished diff. On 66 real coding tasks it solved about as many as the most expensive model working alone, for roughly 5 cents a task instead of 72 cents. Everything is public, including the receipts. Tell me what's wrong with it.

Hello good people. I got sick of paying frontier prices for every step of a coding task, when most of those steps are just reading files and typing. But when I used a cheap model on its own, it shipped bugs I only found later.

So I built a loop — the Krystal Loop Protocol:

1. A cheap model writes the change in a sandbox.

2. Your tests run.

3. A frontier model from a different company reviews the exact diff. It only ever sees one small packet — the task, the diff and the test output — which is why a review costs about a cent and a half.

4. The important part: the reviewer can't just say "this looks wrong". It has to write a test that proves it. If the test fails, the bug is real and gets fixed. If the test passes, the complaint is thrown out. Then the fix gets checked and reviewed again, until it's clean or it hits a limit.

Results on 66 feature tasks from CooperBench (Click, Jinja, dirty-equals), graded by tests the models never see:

The honest part: the cheap model on its own was a lot closer than I expected. The review only added 0 to 2 solved tasks per run. What it did do is catch real bugs (77 proven with a failing test in the best run, 70 fixed) and it never broke something that was already working. So the win is mostly price, not score.

Where it mattered more: two agents building features that conflict. I took 30 pairs where the loop had already built both features. A plain git merge worked on 2 of them. Two agents handing over through short structured messages got 28 working, at about 10 cents a pair. Agents chatting to each other also got 28, but burned 581 model calls to do it instead of 333. For reference, one frontier model building both features from scratch only managed 22, at $0.196 — nearly double the cost.

How it's different from what's out there:

The reviewer can also be a CLI subscription you already pay for. Gemini, Kimi, Grok and Qwen all reviewed identical worker output and landed within two tasks of each other — Gemini matched the metered frontier critic exactly (56 tasks) at under 3 cents a task in API-equivalent spend.

Roast me:

• What would break this on your real repos?

• Is 66 tasks enough to say anything?

• I haven't tried a local model as the builder yet. My bet is the test gate protects quality more than the builder's brains do, so a decent local model should hold the score and take the cost close to zero. If you've got a local setup, I'd love to know if that holds.

Check it yourself: every number recomputes from the receipts with no API keys (python3 -m bench.verify). Rerunning my loop costs about $3. Or run your own setup on the same tasks and hidden tests and post your score: grade your own setup

Repo: https://github.com/KrystalUnity/krystal-loop-protocol. Source-available, free for individuals and small companies.

If AI is doing half the coding, who's picking up the subscription bill, the developer or the company?

When I started agentic coding, my usage was around 20-30m tokens a month. As I handed more and more work to AI, it rose steadily to about 150-200m / month.

Last month, I started writing a custom harness. I have all my projects added to it, and I can just message it from Slack. It then plans, implements, documents and then commits the changes.

I just couldn't emphasize how powerful it seems. Previously, I would pull some data, open it in VS code, review it for 15 minutes and then tell an agent to make some changes, now, I can just tell the agent to pull said data from server X, review it and suggest a fix. I can also run 5-6 jobs at once, and it's overwhelming like I expected.

Is that what I should have been doing since the start?

I’ve recently used chat gpt to make a training assistant that can click through a training portal, work through the modules, watch videos, take transcripts of the videos etc. Chat gpt told me how to code the whole thing using python and VScoder. I’m up to the final steps before giving it its first official go but it’s told me it cannot select answers and submit questions for me about the videos it studied in the modules. How can I work around this or should I move my project to another AI that might do it? I’ve told it numerous times it’s just practice questions and I don’t fail or gain anything from it. Please help as I’m stuck now and have poured a considerate amount of time into this.

Not sure if the right forum to post this but suggestions needed!

I’ve been using ChatGPT to learn Japanese by creating a new project. I opened a main thread to create a Google Drive folder for all static contexts and course materials/course flow/future plannings etc. I linked the Drive folder to the project as reference.

I explicitly asked the model to review all the contexts and materials before starting each lesson. When I wanted to start the lesson, I would open a new chat in the same project. I just said “begin lesson”.

It’s my third day trying out the flow. I noticed each time the model would deviate from the flow agreed in the context file. When I pointed out “hey are you following the instructions in Google Drive?”the model would reinspect the files again and acknowledged they didn’t follow it strictly. I had no idea what basis or memory they used to start the current session lol

If relevant - I was mostly on gpt-6 Astra medium. When they had to write stuff using gdoc skills, they switched to Work mode and I believe was mostly on gpt-5.6 Sol.

Anything I did wrong? Or missed that’s worth doing? Maybe I need to use detailed prompt every time I open a new chat? (which I thought could have been avoided/automated since I assume the gDrive was sort of like a static memory?)

Thanks folks!