r/golang • u/Charming_Skin_8549 • 1h ago
Linux kernel modules in Go
r/golang • u/AutoModerator • 4d ago
This is the weekly thread for Small Projects.
The point of this thread is to have looser posting standards than the main board. As such, projects are pretty much only removed from here by the mods for being completely unrelated to Go. However, Reddit often labels posts full of links as being spam, even when they are perfectly sensible things like links to projects, godocs, and an example. r/golang mods are not the ones removing things from this thread and we will allow them as we see the removals.
Please also avoid posts like "why", "we've got a dozen of those", "that looks like AI slop", etc. This the place to put any project people feel like sharing without worrying about those criteria.
This is a monthly recurring post. Clicking the flair will allow you to see all previous posts.
Please adhere to the following rules when posting:
Rules for individuals:
Don't create top-level comments; those are for employers.
Feel free to reply to top-level comments with on-topic questions.
Meta-discussion should be reserved for the distinguished mod comment.
Rules for employers:
To make a top-level comment you must be hiring directly, or a focused third party recruiter with specific jobs with named companies in hand. No recruiter fishing for contacts please.
The job must be currently open. It is permitted to post in multiple months if the position is still open, especially if you posted towards the end of the previous month.
The job must involve working with Go on a regular basis, even if not 100% of the time.
One top-level comment per employer. If you have multiple job openings, please consolidate their descriptions or mention them in replies to your own top-level comment.
Please base your comment on the following template:
COMPANY: [Company name; ideally link to your company's website or careers page.]
TYPE: [Full time, part time, internship, contract, etc.]
DESCRIPTION: [What does your team/company do, and what are you using Go for? How much experience are you seeking and what seniority levels are you hiring for? The more details the better.]
LOCATION: [Where are your office or offices located? If your workplace language isn't English-speaking, please specify it.]
ESTIMATED COMPENSATION: [Please attempt to provide at least a rough expectation of wages/salary.If you can't state a number for compensation, omit this field. Do not just say "competitive". Everyone says their compensation is "competitive".If you are listing several positions in the "Description" field above, then feel free to include this information inline above, and put "See above" in this field.If compensation is expected to be offset by other benefits, then please include that information here as well.]
REMOTE: [Do you offer the option of working remotely? If so, do you require employees to live in certain areas or time zones?]
VISA: [Does your company sponsor visas?]
CONTACT: [How can someone get in touch with you?]
r/golang • u/Gloomy-Ebb7311 • 9h ago
Hello everyone, wanted to share something I’ve been working on for the past year: betterauth-go.
It lets you use Better Auth’s frontend clients React, Vue, Svelte, etc. with a Go backend. Mount the handler in your Go server and point your client at it.
I’ve also been testing it in my clients’ products, and I’d love to get it into more developers’ hands.
It supports email/password auth, social login, sessions, and plugins like 2FA, organizations, and magic links. Storage options include Postgres, SQLite, MySQL, or your own adapter.
It’s MIT licensed, and there’s a Go + React example in the repo to help you get started.
https://github.com/patrickkabwe/betterauth-go
If you try it out, I’d love to hear how it goes. Feedback, issues, and contributions are welcome!
r/golang • u/BrilliantUseful2960 • 1h ago
Sharing a project that has run in production for us for a few weeks. The Go side may interest you:
CGO_ENABLED=0, a distroless image of ~28 MB, ~20 MB idle RSS.
HTTP is stdlib net/http with 1.22 route patterns. The database is pgx/v5 with hand-written SQL and goose migrations, so no ORM.
Sessions and API keys use small in-repo code (internal/auth, internal/ids) instead of JWT or UUID libraries.
Every feature is its own package under internal/ with a Mount(a *app.App) registered from init(), so adding one is one package plus a blank import.
Plugins are plain HTTP services (OpenAPI/MCP) rather than loaded code, which keeps the core small and crash-isolated.
The React/Vite app is built into web/dist and embedded with embed.
Postgres is the only required service. Vectors (pgvector), full text and advisory locks do the jobs a queue or vector DB usually would.
Tests run with -race against a real Postgres per test (make testdb), and CI also runs govulncheck and gitleaks.
Code review of the architecture is welcome: https://github.com/cdmx-in/kera
This is an open source project that I have been working on for a few years, It's available on GitHub and also has been deployed live on my Raspberry Pi
GitHub: https://github.com/Anga205/AngaDriveV3
Live deployment: https://drive.anga.codes
Since the primary focus of this project was to achieve production-level performance from the limited hardware that I hosted on (that is, a Raspberry Pi), There are a few architectural and resource related optimizations that I've implemented that I think anyone with a similar project would find interesting.
WebSocket-based communication: The frontend and backend communicate over a persistent WebSocket connection rather than making separate HTTP requests for each operation. This avoids repeatedly paying the HTTP request/connection setup overhead and allows requests and responses to flow over an already-established connection.
Real-time updates without polling: The same WebSocket connections are used to push updates to connected users. For example, when a file or collection changes, the backend can broadcast the relevant update to subscribed clients instead of having every client repeatedly poll for changes. This reduces unnecessary requests and backend load.
In-memory indexing instead of Redis: Relationships between users, files, and collections are indexed using Go maps and sets protected by sync.RWMutex. This gives average O(1) membership checks and key lookups without a separate Redis process or network/protocol communication overhead. The trade-off is that cache consistency and data management become more complex, since the application has to maintain these indexes itself.
There are also other several micro-optimizations like personal cache initialization and bounded background workers that I think anyone with dreams of making a big project in home lab infrastructure would find pretty interesting to go through in my codebase.
The general idea is to keep frequently accessed data and communication paths in-process or on persistent connections wherever practical. Many of the fundamental index operations are average-case O(1), although operations involving iteration or database work naturally have different complexity.
Tech stack: Go, Gin, GORM, SQLite, and Gorilla WebSocket. (SolidJS for frontend)
TL;DR
Default MaxIdleConnsPerHost is 2. Which is too low, causes client to open a lot more connections and causes performance drop when using http/1.1 (up to 8x drop in my benchmarks). This is because of running out of ports and Kernel having to search for one repeatedly
Fix : MaxIdleConnsPerHost should be configured for your situation
Full write up (benchmarks and stuff) : https://newsletter.esc.sh/p/performance-penalty-in-gos-default
If you use Go's default http client without changing MaxIdleConnsPerHost and you happen to use HTTP/1.1 (which is pretty common in service to service communications -- like within a Kubernetes cluster), you might be leaving a lot more throughput on the table and causing increased client side CPU usage.
So MaxIdleConnsPerHost has a default value of 2. That means Go will only keep 2 idle connections per host. So, if you have a high throughput client, it will open a lot lot more connections causing huge performance drop and CPU usage. In my benchmark (probably a worst case scenario), I saw a 8x drop in throughput compared to using MaxIdleConnsPerHost = number of goroutines (requests in flight)
The most interesting finding for me was why it happens - It turns out to be due to the client running out of ports and the Kernel having to do a very long search for free ports.
Fix is of course to set MaxIdleConnsPerHost to whatever max requests in flight you expect. But be mindful and don’t set it too high!
I have a blog post with a lot more details, benchmarks and some kernel level investigations, for those interested - https://newsletter.esc.sh/p/performance-penalty-in-gos-default
Edit: added TLDR
Edit 2: updated the max recommendation
r/golang • u/ncruces • 18h ago
Approximate aggregate window functions for SQLite, using ncruces/go-sqlite3.
r/golang • u/Metal-3d • 1d ago
I know... “yet another GTK binding”—that’s what you’re probably thinking... But let me tell you about my project first.
There are a bunch of bindings for GTK4/Adwaita, either as dynamically linked libraries or as “purego” bindings. But:
Either it takes too long to compile, which is a bit of a pain… plus I’m potentially locked into a specific version
Or the “purego” binding—which I really like, since it uses dlopen() plus a function map—clearly falls short of my expectations (lots of classes are missing, no `cairo` or `webview`)
In short, I’ve had enough. For several months now, I’ve been putting something together locally to have everything I need for the various apps I’m developing. And today, after a major cleanup, I’m releasing it as open source.
What makes it different? Not much in terms of magic, but I’ve worked hard to make sure it covers almost everything:
GTK, Adwaita, GDK, Glib, WebKit (WebView), and Cairo
generated from GIR, with “helpers” for special cases
easy to maintain, fix, and update
I’ve included examples, and I use it daily for development (so I’ll keep maintaining it)
ALL classes are (theoretically) included; so far, I haven’t found anything missing (there might be some, and if so, please let me know)
In short, if you want to test it and report any potential issues to me...
I'll be releasing (in the next day or two) a program I developed using this “Cellule” library—a tool for Matrix that, in my opinion, is much better suited to the GNOME desktop than Element.
And there are more to come :)
Hello there,
I'm working on a project which which should provide at least 2 binaries - a TUI app and an REST API and keep everything under single repository, obviously the dependencies for each are different (bubbletea and friends for TUI and gin for API), thus using separate modules makes sense to make them installable separately, but I have some common packages (for now it's database) which are required for both binaries. With chat gpt help I've made something like this:
. (modname: github.com/<username>/<projectname>)
├── Dockerfile
├── cmd
│ └── tui (modname: github.com/<username>/<projectname>/cmd/tui)
│ ├── accounts.go
│ ├── go.mod
│ ├── go.sum
│ └── main.go
├── database
│ ├── db.go
│ └── migrations
│ └── 001_initial.sql
├── go.mod
├── go.sum
├── go.work
├── go.work.sum
└── tmuxp.yaml
// go.work
go 1.27.3
use (
.
./cmd/tui
)
// cmd/tui/go.mod
module github.com/<username>/<projectname>/cmd/tui
go 1.27.3
require (
charm.land/bubbles/v2 v2.2.1
charm.land/bubbletea/v2 v2.1.2
charm.land/lipgloss/v2 v2.1.1
github.com/<username>/<projectname> v0.0.0
)Unfortunately it does not allow me to run tui/main.go, it tries reaching github for <projectname> instead of using the module in local filesystem. Ideally I would like to avoid pushing the code, tagging it and adding/removing replace. As for now nothing was pushed to github, I'd like to work locally until there's actually something to push. How can I make this work?
Hi r/golang,
Earlier this year I wrote GOKe, an Entity Component System for Go. Since then it has been the core of several of my own projects, including https://github.com/kjkrol/gram It has held up well, so I’m sharing it in case it’s useful to you too.
What it offers:
archetype-based storage, so iteration is cache-friendly
chunk-based SoA layout with direct pointer arithmetic - no GC pressure during iteration or component access
fully generic - no reflection, no interface boxing, no runtime type assertions
go get github.com/kjkrol/gokeRepo: https://github.com/kjkrol/goke
Wiki: https://github.com/kjkrol/goke/wiki
I’d love to hear feedback, especially from anyone who has used other Go ECS libraries.
I've been working on Collie since February: a Go implementation of Dynamic Consistency Boundaries (DCB) event sourcing, with a Postgres and an in-memory store.
In DCB, instead of aggregates and streams, there is one global log of tagged events. A decision reads the events it depends on with a query, and the append is refused if a matching event arrived since. The consistency boundary is exactly what the decision read. Check dcb.events for the full spec and to learn more about DCB.
I run it in production on a side project (with tens of millions of events), but that doesn’t automatically make it production ready. I’m quite confident it works well, but let’s talk again in a year...
In the meantime I’ll keep improving it. That’s also why I don’t promise a stable interface yet: I’m pretty sure I’ll introduce breaking changes whenever I find a way to make things better. The optional packages (reactor, envelope, encryption and attachments) are younger than the core, so they’re the most likely to change.
AI use: I didn’t want to vibe code it. I went old school: I designed and tested the interfaces myself. I did use Claude to quickly visualize the ideas I had in my head, but Collie is handcrafted as much as possible. It’s not one of those projects where nobody has read the code the AI wrote...
Looking forward to feedback from people!
Github: https://github.com/DenBeke/collie
My blogpost about why I actually spent time on this: https://denbeke.be/blog/programming/introducing-collie-nine-months-of-dcb-event-sourcing-in-go-postgres-rounded-up-into-a-library/
r/golang • u/Emotional-Ask-9788 • 2d ago
How do you handle OpenAPI in go?
I'm coming from C# .Net and Laravel, normally these auto generate the specs, but in go I haven't found the standard way of doing this, the golang way.
I understand there are packages you can use to do this by writing comments/annotations, which is very tedious.
Is there a proper way to do this, or do even go developers do this at all?
Your response is very welcome, especially from those working in companies with somewhat large code base.
Thank you!
r/golang • u/Fabulous-Cookie-6457 • 2d ago
I maintain Go-Gen-Ecosystem/go-migrate. I compared v1.0.1 with golang-migrate/migrate v4.20.1 and pressly/goose v3.30.1. Each library applied the same 100 SQL migrations to a fresh SQLite database. Each migration created a table and inserted one row.
On Linux/amd64 with Go 1.26.11, the median allocated per batch across six samples was 727.8 KiB for go-migrate, 10.66 MiB for golang-migrate, and 1.14 MiB for goose. Goose made the fewest allocations at default settings, with 8,547 per batch compared with 9,420 for golang-migrate and 10,873.5 for go-migrate.
I repeated the golang-migrate benchmark with one change. Setting gm.DefaultBufferSize = 4096 at the start of its benchmark, down from the 100,000-byte default, brought its median to 892.2 KiB on these small files. All three libraries used in-memory migration sources, the same pure-Go SQLite driver, WAL, and synchronous=OFF.
To reproduce the defaults from a disposable v1.0.1 checkout, run the following. The tagged benchmark module needs -mod=mod to refresh its dependency metadata.
cd benchmarks/compare
GOWORK=off CGO_ENABLED=0 GOTOOLCHAIN=go1.26.11 go test -mod=mod -run '^$' -bench '^BenchmarkUp100_' -benchmem -count=6 .r/golang • u/Horror_Farmer_2875 • 3d ago
Posting because I’d never seen this written down in one place and it cost us a few days.
Go service, shells out to az and kubectl a lot, mostly through bash -c. One morning a pod started failing everything at once. DB connection reset. fork/exec /usr/bin/bash: resource temporarily unavailable. A fetcher saying its config was missing. A cloud integration marked INVALID with “az login failed” and 500+ retries behind it. Four different errors in the same minute, none of them pointing at each other.
Turns out resource temporarily unavailable is EAGAIN, and on fork that means no more PIDs. k8s caps pids per pod and we’d hit the cap. So everything that needed to spawn a process was failing, just with different wording.
Where the PIDs went:
Three call sites used context.Background() for the exec. No timeout, nothing ever cancelled them. Our fault, boring.
The interesting one was the call site that did have a 90s timeout. exec.CommandContext(ctx, "bash", "-c", "az ... | kubectl ..."). Context expires, Go SIGKILLs bash. az and kubectl don’t get touched, they just get reparented and keep going. I’d assumed CommandContext killed the tree. It doesn’t, it kills one process. And kubectl was hanging forever on auth for a bunch of clusters where our SP was missing a role, so each scan left a few of those behind. Scan every hour, for weeks, and eventually the pod can’t fork a health check.
The fix is small once you know:
cmd := exec.CommandContext(ctx, "bash", "-c", script)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} // own process group
cmd.Cancel = func() error {
return syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) // negative pid = whole group
}
cmd.WaitDelay = 5 * time.Second // don't hang Wait if a grandchild still has the pipe
(Cancel and WaitDelay are 1.20+, before that you do it by hand with a goroutine on ctx.Done().)
Also put a pids limit on the pod so next time one pod dies instead of the node.
The thing I’d actually tell people: if you ever wrap a command in bash -c or sh -c, you have grandchildren, and CommandContext won’t clean them up for you.
Do people set Setpgid in their exec helper by default? Or is everyone finding this out the same way I did.
I wanted to understand what happens when an AI app calls a tool over MCP, so I built a small MCP server in Go and wrote up how it works.
The article follows the server side of the interaction: how the client discovers a tool, how it calls it, and what the Go handler returns. If you’ve used MCP from an AI app and wondered what the server is doing, this is the walkthrough I would have wanted.
If you’ve built an MCP server in Go, I’d be curious to hear what surprised you the first time around.
I posted this before.
Back when I started switching to Go in 2013, from PHP, what do you do, it's a mind easing and also expanding experience, and I'd wager pretty much everyone has written their own router.
And so did I. It started out as a static route trie router. As far as static routes go, it beat julienschmidt/httprouter. But of course not parameterized routes worked, and no middleware etc. It was just a static route router.
Then came AI and looking for stuff to release it upon I dug out my old projects and tried improving them with AI.
At first that was using, unimaginable nowadays, glm-4.7, then gpt-5, then glm-5.2 and a short while ago finally opus-5.5 .
And Opus did a phenomenal job. It fixed bugs those "crappy" LLMs introduced and it also increased performance, by a lot.
In 20 out of 21 benchmark tests, it beat Go's stblib router, by up to x3.3 .
4 years ago this would've been big news. Nowadays probably barely anyone even cares I'd wager, with code being generated by AI en masse.
I'll still post it.
I think it's a positive example where AI can be helpful and useful, take an existing code base which was human made and improve it.
Anyhow I'd like to invite you to try it out, maybe you'll find your own bugs, perhaps give feedback.
I'm considering to set up a github mirror so people could report issues, because this forgejo instance doesn't allow sign ups by other people than me.
Why sux? Back when gorilla/mux was a big deal, I tought naming it sux aka static mux would be a good idea. Hope this codebase doesn't sux for you ;)
I find it quite enjoyable to do the archaeology on an accepted proposal and write about it.
This one took 10y to get approval. It allows you to strip type params from composite literals and let the compiler infer the types from the context.
For example, returning zero values:
// Before
func parseTime(s string) (time.Time, error) {
if s == "" {
return time.Time{}, errors.New("empty timestamp")
}
return time.Parse(time.RFC3339, s)
}
// After
func parseTime(s string) (time.Time, error) {
if s == "" {
return {}, errors.New("empty timestamp")
}
return time.Parse(time.RFC3339, s)
}
or, passing options structs drops the redundant types:
// Before
h := slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{
Level: slog.LevelDebug,
})
// After
h := slog.NewJSONHandler(os.Stderr, {Level: slog.LevelDebug})
and so on. The spec change isn't huge but eliding types everywhere can make the code quite hard to read. So there won't be any automatic go fix to strip types wherever possible. This was a deliberate choice made by the Go team.
But it does make the code look pretty in some nested struct of type kind of scenario. Added a few more example and context here.
https://rednafi.com/shards/2026/11/go-untyped-composite-literals/
r/golang • u/jingweno • 4d ago
I maintain Upterm, a terminal-sharing tool written in Go. I posted it here a few years ago; this is an update on the released v0.41.0 functionality.
Plain SSH still joins the shared terminal. Guests can now also request their own commands, use scp/sftp, or forward ports to the host’s loopback. These requests need approval in the host terminal unless explicitly allowed for the session.
Upterm isn’t a terminal multiplexer. It can share a shell or run with tmux, and the relay can be self-hosted. Guests using plain SSH don’t need Upterm installed.
Release notes: https://github.com/owenthereal/upterm/releases/tag/v0.41.0
These features provide real access to the host, so use a disposable environment without sensitive credentials when trying them.
r/golang • u/klauspost • 4d ago
MinLZ adds a new block type: SIMD blocks
They store the output of the LZ compressor as separate Huffman-coded streams, laid out so that a CPU can decode them with vector instructions and almost no unpredictable branches.
The result is compression ratios in the range of Zstandard, with decompression 2.2-2.5× faster than Zstandard (both in Go) and about 2× faster than classic MinLZ blocks, on a single core.
At the default 2 MB block size, MinLZ SIMD "Balanced" beats zstd's fastest level on all three counts: Its output is 2% smaller, it decompresses 2.5× faster on one core (6.4 against 2.6 GB/s),and it compresses 1.6× faster on 16 threads (1.2× on one).
Against zstd's default level, SIMD Balanced output is 3.6% larger and decompresses 2.3× faster. (Geometric means over 13 data sets, 58 GB in total, Go implementations on an AMD Ryzen 9 9950X).
SIMD blocks are part of the MinLZ format (specification v1.1) and work everywhere MinLZ blocks do: streams, concurrent compression and decompression, indexes and random access, and search. We have released Go/Rust versions for now.
See more in the release notes: https://github.com/minio/minlz/releases/tag/v1.3.0
r/golang • u/andybons • 5d ago
hey all. former Go core team member here to share my process for large scale migrations (in this case, from Node to Go). I also link to some open-source projects I’ve been working on through work. Gatekeeper and Harness.
https://www.meetneptune.com/engineering/node-to-go-in-two-weeks