Hey guys!
https://github.com/Leeroo-AI/kapso
We built Kapso, an engine that runs experiments toward an objective, deploys the best solution on your stack and learns from every run into a bank of lessons with a credence score and a boundary.
You can view a video walkthrough here: https://www.youtube.com/watch?v=W4FHlza1fRs
Here’s how it works…
Kapso has a bank of lessons that are induced or deduced from the past experiments, and each carries a credence score, conditions on where it applies, and the evidence behind it. For each new experiment, an ideation module proposes ideas considering the earlier lessons, re-ranks them by expected return against the budget each would use, and sends the top-ranked ones to the implementor. Because the bank keeps learning from the past experiments in that environment, the proposed ideas are better suited to it than generic hypotheses from an LLM or RAG. Also, results of each experiment are verified by a verifier module that no experiment can access and cheat. As we’ve found when building such systems, sometimes the success criteria and evaluation tests need to be evolved. The experimentation modules can propose updates to the metrics and criteria for the verifier to approve or reject.
After each run, learn() updates the bank of lessons’ scores and boundaries using the evidence from that run. Changes to the bank can also be A/B tested by running otherwise identical builds with the previous and updated banks. The bank is built on git, so you can share your experience with others, list the banks available to you, and update them (`kapso experience share/list/update` in CLI). The system is model-agnostic and coding-agent-agnostic.
kapso.deploy(solution, target_stack) follows the same workflow, runs a shorter experimentation loop to adapt the best solution to the target stack (e.g. Docker, Modal, BentoCloud, LangGraph). Every domain has a harness to start from, you can write your own domain harness with /kapso:harness in the plugin or through CLI (kapso harness new <name>), and compound over that by running experimentations.
We ran it on some public benchmarks and live competitions, with the following results…
- Predictive modelling on relational databases: first on RelBench, Kumo AI (now Nvidia) and Stanford's benchmark, ahead of the best tabular foundation model (https://star-project.stanford.edu/relbench/leaderboard/). This shows that keeping the learning in the harness-level can beat foundational table models. Updating a bank of lessons costs far less than retraining a tabular foundation model.
- ML Engineering: First on OpenAI's MLE-bench at submission, still among the top open-source systems (https://github.com/openai/mle-bench/pull/107).
- AI competitions: ahead of the best agent the ALE-Bench authors at Sakana AI reported, and running autonomously at the live International Olympiad in AI 2026, above all 471 human contestants and in the top three AI systems (https://ioai-official.org/ai-models-track/ioai%c2%b2-2026-report/).
We have a public bank for ML and Data at Leeroopedia (https://leeroopedia.com/): around 100k pages extracted by experimenting on papers and repos, about 6M requests a month, about 28% from AI agents.
We would like to get your feedbacks on how to improve our learning!