r/kubernetes • • 15d ago

Periodic Monthly: Who is hiring?

23 Upvotes

This monthly post can be used to share Kubernetes-related job openings within your company. Please include:

  • Name of the company

  • Location requirements (or lack thereof)

  • At least one of: a link to a job posting/application page or contact details

If you are interested in a job, please contact the poster directly.

Common reasons for comment removal:

  • Not meeting the above requirements

  • Recruiter post / recruiter listings

  • Negative, inflammatory, or abrasive tone


r/kubernetes • • 3d ago

Periodic Weekly: Share your victories thread

2 Upvotes

Got something working? Figure something out? Make progress that you are excited about? Share here!


r/kubernetes • • 1h ago

Kubernetes Gym - a Fun Way to Improve kubectl Fluency

34 Upvotes

Hey folks!

The collection of hands-on Kubernetes challenges I shared here last time was received rather well, so I figured I should bring it up again - it has grown quite a bit, and most of it is still free. But we also came up with an alternative format to improve Kubernetes skills: Kubernetes Gyms.

The new format is powered by github.com/iximiuz/shellgym and focuses on a different aspect of Kubernetes mastery - kubectl fluency - which is an important part of the job and is also very much needed for successful CKA/CKAD/CKS certification.

There are already 8 Kubernetes 101 gyms (all free), and more are coming soon. The idea is that each gym comes back on a spaced-repetition schedule so your command-line muscle memory doesn't get rusty, even if you don't get a chance to use kubectl at your day job for some reason.

Last but not least, a huge shout-out to our community authors: Kaloyan Preslavski and Omkar Shelke. They are the real heroes who made these gyms and challenges possible.

P.S. The materials above are hosted on iximiuz Labs, a freemium indie learning platform I run, but they are all available on the free tier, so this is not a sales pitch.


r/kubernetes • • 13h ago

How do you keep Kubernetes rightsizing and autoscaling configs from becoming a permanent manual job?

12 Upvotes

When running different types of clusters (prod, staging, dev), how do you keep resource allocation and autoscaling settings aligned with workload requirements as they change, to avoid unnecessary increases in the cloud bill?

Over the lifetime of a product, new services get added in a cluster, traffic patterns would change, requirements evolve, etc. CPU/memory requests could get padded, limits and replica counts can become outdated, and the node types originally chosen may no longer fit the current workload mix particularly well.

I'm particularly interested in continuous pod rightsizing alongside tools like Karpenter and KEDA. As a simple example, cpu/mem requests could remain significantly higher than workloads actually need over time. This usually leaves the cluster running far more nodes than necessary (even for smaller workloads with a few nodes, even 1 or 2 of overprovisioned and unutilized nodes could cause a significant dent in the final bill). This can be overprovisioned intentionally, but over time it does become a bottleneck in a sense that there's not enough time to address everything.

How do you keep those settings appropriate as workloads evolve? How are you handling this in practice, and lately with AI?

- Do you review the full cluster regularly, leave it to application teams to right size (where applicable), or just accept the overprovisioned capacity?

- How do you distinguish necessary headroom from avoidable waste?

- Are you using any tools that recommend specific changes and explain the expected savings and risks, ideally across the major clouds? I know this can get very workload-specific, but still would like to hear general thoughts.

- Does anyone use automated tools that continuously adjust pod resources, scaling settings, or node selection and placement? Can you share your experience with such tools? Does any of these run internally within own clusters, or do they all offer managed control plane?

Thanks


r/kubernetes • • 1d ago

Hows everyone handling secret zero outside kubernetes?

34 Upvotes

vault or openbao handing out short lived postgres logins is ok and nothing sits in a k8s secret anymore. but the app still has to log into vault first

inside the cluster thats mostly solved, vault checks the pods service account token against the TokenReview api and theres even a patch on pgsql-hackers so postgres cert auth can match spiffe ids. outside kubernetes its murkier. cloud instance identity covers vms on aws or gcp, spiffe setups like spire or hush security go wider and a lot of the rest ends up with an approle secret id baked in somewhere which is the same problem one step back

whats ideal to use for that first login outside the cluster?


r/kubernetes • • 13h ago

Scaling a hierarchical federated infrastructure to 1,500 container nodes with sub-400ms latencies (and machine-checking the Byzantine fault tolerance in Lean 4)

Thumbnail
reddit.com
0 Upvotes

r/kubernetes • • 1d ago

How do you handle prod promotion with Argo CD + Image Updater when CI is triggered by a git tag?

Thumbnail reddit.com
3 Upvotes

r/kubernetes • • 1d ago

Anyone running kubernetes-sigs/agent-sandbox in prod or lab for coding agents? Looking for real-world feedback on isolation, warm pools & latency

24 Upvotes

I’m currently evaluating kubernetes-sigs/agent-sandbox to run autonomous coding agents (like Claude Code, custom harness ,etc) on our own Kubernetes cluster, and I’d love to get feedback from anyone who has deployed it beyond a quick local test.

​Context & Current Setup:

  • ​Cluster: AWS EKS

  • ​Workload: Ephemeral dev/test environments where agents clone repos, run builds/tests, and execute arbitrary shell commands via MCP or tool calls

  • ​Isolation: Evaluating gVisor vs. Kata Containers vs recently introduced AWS nitro sandboxes

​Questions for folks actually running this:

  1. ​Cold-start latency & SandboxWarmPool: What kind of pod startup and readiness times are you seeing in practice? Are warm pools reliable enough to make interactive/agentic loops feel snappy, or are you seeing churn issues?

  2. ​Runtime performance hit: If you’re pairing agent-sandbox with gVisor or Kata, how noticeable is the performance hits

  3. ​Egress & network isolation: How are you locking down outbound network access (e.g., allowing git clone and package registries while blocking internal VPC/metadata endpoints)?

  4. ​Workspace caching: Are you doing anything clever to cache dependencies across ephemeral sandbox runs without compromising isolation between sessions?

  5. ​Hidden gotchas: Any rough edges around CRD lifecycle cleanup, orphaned PVCs/volumes, or resource quotas when an agent loop goes rogue?

​If you looked at agent-sandbox and decided to go a different route—like self-hosting E2B, rolling custom Firecracker microVMs, or just using hardened Kubernetes Jobs—I’d also love to hear what pushed you away from it.


r/kubernetes • • 2d ago

Kubefactory - Container Management Gaming Interface

462 Upvotes

A first-person factory game that views and manages your Kubernetes clusters and Docker containers.

In-browser (sandbox) demo: https://dascondor.github.io/kubefactory/
GitHub Repo: https://github.com/DasCondor/kubefactory

Edit: Thank you all for the support & feedback, if I haven't answered your question in the comments I am sorry - I will try and get with you over this coming week.

Different versions of Kubefactory 1.0.1-release vs 2.0-beta & Coming 2.1;

1.0.1 — Original release

  • Windows desktop

  • Browser demo

  • Mobile/tablet access through the browser

2.0.0-beta.2 — Available now

  • Windows, Linux, and macOS builds

  • Browser demo with phone and touch support

  • Native Kubernetes and Docker connectivity

  • Model library, LiveView, storage inspection, Krew, Porter, Bob, Gauge, and Foreman

  • VPN registry, exit-IP checks, direct pipes, VPN sidecars, and gateway views

  • Service pipelines and Jellyfin + VPN media stack

  • Text editor with Standard, Vim, and Nano modes

  • High-graphics browser fix, updated docs, screenshots, tests, and release attestations

  • Windows signing workflow wired in, pending activation

2.1 — Coming next

Further polish, fixes, testing, and features building on the 2.0 beta.

Note: beta.2 is a pre-release, and its packages are not yet code-signed.


r/kubernetes • • 1d ago

Prune=false on every Namespace with Argo CD auto-prune, is this what people actually do?

2 Upvotes

Sorry first, english is not my first language and I write this with a translator.

I never worked in a big SRE or platform team, so I want to check if what I did is normal. I run a small k3s cluster at home, plus EKS at work and a small AKS for a client. A read-only audit tool I made for my own clusters (kubegap) flagged that my Argo CD apps with automated prune also track their own Namespace.

So I tested it with throwaway apps on Argo CD 3.2.0. I removed namespace.yaml from kustomization.yaml and pushed. 2 min 48 s later the Namespace was pruned, and a Secret I had created by hand inside it was gone too.

Removing an app's directory under an ApplicationSet git generator is a different path. The Application itself gets deleted and the finalizer cascades. Prune=false does nothing there, Delete=false does.

What I changed: every Namespace manifest now has the sync-options annotation Prune=false,Delete=false (8 of them). The istio app owns 23 CRDs, so I turned automated prune off for that one. And I added an alert for apps that stay OutOfSync for 30 minutes, because a skipped prune just leaves the app OutOfSync and nobody looks.

Questions for people who run this in production:

Do you keep Namespaces inside the app at all, or use CreateNamespace=true or a separate app just for namespaces?

Is per-resource Prune=false normal, or do teams just turn automated prune off?

I read that a force sync can still delete a namespace even with Prune=false. Do you guard against that somehow?


r/kubernetes • • 1d ago

Does GKE's node kernel have the SCTP `sctp_init_cause` padding fix, or is it still shipping unpatched?

Thumbnail reddit.com
3 Upvotes

I've been checking which Linux kernels still carry an SCTP bug that panics the host: `sctp_init_cause()` checks buffer room for an error cause but not for the up-to-3 bytes of 4-byte alignment padding the write adds, so a crafted INIT (listener) or INIT-ACK (outbound association) can walk an skb one byte past its end and hit `skb_over_panic()`. It's fixed upstream in `ed6c53b27be4` (mainline 7.4-rc2, Cc: stable), introduced back in 2018.

As of 14 Nov I've read the guard in the shipped source of every maintained stable branch and several distro kernels, and none of them had the fix yet. That includes the current Ubuntu kernels (jammy 5.15, noble 6.8, resolute 7.0). GKE Standard supports SCTP on Ubuntu nodes with Dataplane V2, and Google documents it as on by default from `1.32.2-gke.1297000`. Since GKE's Ubuntu node images are built from Ubuntu kernels, my working assumption is that most GKE Ubuntu nodes are running the unpatched code. I haven't confirmed a specific GKE node-image build, which is where I could use help.

If you run SCTP workloads on GKE (telecom CNFs especially), could you check a non-production node and report back:

- `uname -r` on the node, and whether `ed6c53b27be4` is backported (the node's kernel source/changelog, not the version string, since backports move independently)

- whether `lsmod | grep '^sctp '` shows the module loaded, i.e. a workload is actually using it

- the GKE version and node image

All read-only. Please don't send anything at a listener or a live service; I'm after kernel/config state, not a crash.

If it turns out a GKE configuration is unpatched and reachable, that belongs in a report to Google's Vulnerability Reward Program, not in this thread. I'll help put that together rather than post specifics here.

Mechanism and the full affected-version matrix (no exploit) are written up here:

Here

Mainly I want to nail down one thing: is a default-config GKE Ubuntu node running the fixed kernel yet, or not?


r/kubernetes • • 1d ago

Metrics, logs and alerts for my 3-node Talos homelab

8 Upvotes

My 3-node Talos homelab had no monitoring, so I only found out something was wrong when an app stopped working.

So I added VictoriaMetrics for metrics and VictoriaLogs for logs from every node and pod. Both show up in Grafana, and alerts go to my phone on Telegram.

Healthchecks[.]io also watches the alerting itself, so I hear about it if the alerts stop. The whole stack uses about 1.3 GiB of RAM across the three nodes.

Repo: https://github.com/prateekjaindev/homelab

Write-up: https://blog.prateekjain.dev/metrics-logs-and-alerts-for-a-3-node-talos-homelab-with-victoriametrics-0f4a91b9d44e?sk=751eedebb5d54cc15561d59abb7e52d8


r/kubernetes • • 1d ago

Reviewing agent written Helm changes when the problem is cluster wide

0 Upvotes

We run about 30 services on two EKS clusters and a lot of our Helm values changes now come from agents. They're usually small and they usually look fine.

Last week one changed the default resource requests in our shared chart from 250m to 500m CPU "to reduce throttling". Review was clean. coderabbit reviewed the values file, it catches things like missing limits or a probe pointing at the wrong port, and there was nothing wrong with the YAML itself.

The problem was everything else. 30 services picked up the new default on their next deploy, the cluster autoscaler added 6 nodes overnight, and our bill for that week jumped by about $1,900 before anyone noticed.

The diff was 2 lines. The impact was the whole cluster, and nothing in the PR showed that.

What we're looking at now:

- running helm diff for every release that uses the shared chart in CI and posting the summary on the PR
- requiring a second reviewer from platform for any change to shared chart defaults
- some kind of cost estimate on PRs that change requests

How do you review changes to shared charts? Do you diff every consumer, or is it mostly "be careful with the shared one"?


r/kubernetes • • 1d ago

Kubernetes Scheduling: Can You Move an Existing Pod to Another Node Without Touching Its YAML? 🤔

Thumbnail reddit.com
0 Upvotes

r/kubernetes • • 2d ago

GitHub - danielecr/tetragon-outgoing-guard: block any process with outgoing connection (run only on k8s or docker swarm vms)

Thumbnail
github.com
4 Upvotes

How I locked down my Docker host against GSocket-style C2 implants using a 50-line Cilium Tetragon eBPF policy.

Also in docker swarm, I mean, the only services are supposed to be in container


r/kubernetes • • 1d ago

I have my KCSA day after tomorrow

2 Upvotes

as the title suggests, can I have any last day tips. Any question which you remember could be of help.
thanks


r/kubernetes • • 2d ago

Best resource to learn hands on Kubernetes

72 Upvotes

Hello, I am preparing for an interview. I have limited experience in Kubernetes with some in open shift. But most of my experience into terraform, Python, Jenkins, ansible and cloud architecture. I have interviews scheduled next week and I am looking for good resources to learn and practice Kubernetes over the weekend. Can someone suggest a good course/resource for the same? I want to do hands on over the weekend and can set up a lab in my personal laptop


r/kubernetes • • 2d ago

Free browser-based Kubernetes manifest generator (Deployment, StatefulSet, CronJob, RBAC, NetworkPolicy…). Feedback welcome

Thumbnail
reddit.com
1 Upvotes

r/kubernetes • • 2d ago

Tipps on migrating TLS Cert Management from Traefik to cert-manager

9 Upvotes

Maybe someone here has done this or something similar in the past. This is the situation right now:

I have about 20 services on a HA k3s cluster with the bundled Traefik as a single instance deployment. TLS is currently done with the basic Traefik letsencrypt certificate resolver with a http challenge.

I now want (or rather need) to move to cert-manager, to enable a HA traefik setup. The migration should happen without downtime.

If both Traefik and cert-manager use the HTTP challenge method to generate a cert, cert-manager could never complete the challenge, as Traefik would intercept the challenge call. I was thinking to switch Traefik to the TLS-ALPN challenge temporarily, so that cert-manager can generate the certs, and then switch over to the HA deployment with only cert-manager.

I guess it should work, but are there better options? Any pitfalls to watch out for? I'm thankful for any input!

Edit: I forgot to add that some domains are "customer domains" and they just point their A records to my cluster.


r/kubernetes • • 1d ago

Agent skill to create Kubernetes concepts explanation video

0 Upvotes

Hi everyone,

I made a small skill that lets your AI agent create 3Blue1Brown-style videos explaining Kubernetes concepts. The demo is below.

Tested on Claude Code Sonnet 5 and Opus 5.5

If you have more interesting ideas, feel free to contribute!

Link to GitHub: https://github.com/wielenmaker/k8s-animate


r/kubernetes • • 3d ago

Two-node Kubernetes on openRuyi RISC-V with Kata and Cloud Hypervisor

15 Upvotes

r/kubernetes • • 2d ago

golang kubernetes deployment- home lab

Thumbnail reddit.com
2 Upvotes

r/kubernetes • • 3d ago

ArgoCD, Kargo and Image Updates in Complex Deployment

20 Upvotes

Background/Environment:

I've been running off of Argocd for a few months now. I've been provisiong the K8s clusters using TF and then handing it off to argocd where:

- an Application is defined that monitors git for new ApplicationSet
- ApplicationSet manages a cluster and all of its apps. (Including cert-manager, ESO, etc) and user level type of apps, insert your favorite www pod etc or service.

Problem:

  1. I have bit rot accumulating of say app that are running a 6 month old version of cert manager that could use an update.

  2. I would like to detect that a new artifacts exists that I should pull in (Say an OCI artifacts with SQL migrations I should pull in)

Solutions?

A. I'm exploring kargo right now but it feels really convoluted, I don't mind the concepts of warehouse, stage, freight etc but the amount of code I need to basically say:

- git-clone
- yaml-update
- git-commit
- git-push
- git-open-pr

All to basically try to facilitate changing a tag in this code block:

migrations:
image:
tag: "20261109-8d43de4"

B. I looked at Argo Image Updater but it also has a similar issue of polluting my AppSet with annotations in order to drive behavior.

I have heard others mention packaging the entire app config + app into OCI and update that, I'm still trying to wrap my head around how that works.

How do others keep up with all the updates you need to do on a cluster. Between all the operators, app definitions and need to test, validate and deploy. It feels like there should be some automation that makes this into a sane pattern.


r/kubernetes • • 3d ago

CNCF also plans to introduce two additional certifications in 2027: Kubernetes AI Associate (KAIA) and Certified Kubernetes Inference Engineer (CKIE).

Post image
34 Upvotes