r/openshift • u/SteelBlade79 • Feb 28 '24
r/openshift • u/mutedsomething • 8d ago
Discussion Anyone interested in live, hands-on OpenShift study sessions? (From Production Admin perspective)
Hey everyone,
I’ve been working as a Senior OpenShift Administrator for the past few years (managing enterprise baremetal clusters, storage, networking, and day-2 operations). One thing I constantly notice is how steep the learning curve is for OpenShift—especially when you move past basic oc commands and try to understand what's actually happening under the hood with Operators, MachineConfigs, SDN/OVN, and troubleshooting broken clusters.
Official docs are great, and courses are everywhere, but most of them feel too theoretical or disconnected from real-world production scenarios.
I’m thinking of spinning up a dedicated Discord server to gather folks who are genuinely interested in learning OpenShift the practical way. The idea is to host regular, interactive live sessions where I break down core architecture, walk through real troubleshooting scenarios, and do hands-on labs together—no boring slides, just terminal, whiteboarding, and Q&A.
Whether you're preparing for the EX280/EX380 exams, transitioning from vanilla Kubernetes, or just trying to survive your first OpenShift admin role, you're welcome to join.
Drop a comment below or shoot me a DM if you're interested, and I'll send over the Discord invite link once I get the channels set up. Let me know what topics you'd love to cover first!
r/openshift • u/Personal-Ad2131 • 14d ago
Discussion The 4 OpenShift networking problems that get blamed on "the network" (OVN-Kubernetes, 4.22)
I work on OpenShift for banks, and most network escalations I see turn out to be one of four things inside the cluster. Writing them down here because each one has a recognisable symptom.
**1. Overlapping internal ranges. Symptom: everything works except one backend.**
Besides the cluster network (10.128.0.0/14) and service network (172.30.0.0/16, immutable after install), OVN-K uses a join subnet `100.64.0.0/16`, a transit switch subnet `100.88.0.0/16`, and on 4.17+ installs a masquerade subnet `169.254.0.0/17`. 100.64.0.0/10 is RFC 6598 shared space, and plenty of corporate networks use parts of it. If something your Pods need lives there, traffic to it never leaves the cluster. Set `internalJoinSubnet` / `internalTransitSwitchSubnet` at install.
**2. Policy tiers. Symptom: the NetworkPolicy allows it, still dropped.**
The evaluation order is AdminNetworkPolicy (Allow/Deny/Pass), then NetworkPolicy, then BaselineAdminNetworkPolicy. An ANP Deny can't be overridden from a namespace. Both admin tiers support audit logging.
**3. Egress source IP. Symptom: works until the Pod reschedules.**
The default is SNAT to the node IP. EgressIP gives a namespace a stable source IP that fails over between labelled nodes. Also worth knowing: egress ignores host kernel routes unless you switch to local gateway mode (`routingViaHost: true`), which is cluster-wide.
**4. MTU. Symptom: small requests work, large ones hang.**
Geneve on UDP 6081 costs 100 bytes. VPNs and some interconnects shrink the real path MTU below what the hosts assume. IPsec east-west takes more.
**Debug order that's worked for me:**
```
oc get co network dns ingress
oc cp -n openshift-ovn-kubernetes $POD:/usr/bin/ovnkube-trace -c ovnkube-cluster-manager ovnkube-trace
./ovnkube-trace -src-namespace prod -src app-xyz -dst-namespace prod -dst db -tcp -dst-port 5432
oc adm must-gather -- gather_network_logs
```
(`$POD` = any ovnkube-control-plane pod.)
I wrote a longer version covering ingress sharding, MetalLB, Multus/UDN and DNS forwarding, plus a clickable packet tracer for seven flows: https://aib-devops.com/en/blog/openshift-networking/
Disclosure: it's my blog, no ads or signup. If anyone has hit the 100.64 overlap in production, I'd like to hear how you found it.
r/openshift
•
u/Personal-Ad2131
•
16d ago
Discussion OpenShift architecture explained layer by layer, plus the 4 decisions you can't easily change after
I do OpenShift delivery for banks in the Gulf, and I keep explaining the same architecture to teams coming from vanilla Kubernetes. So I wrote it up once, against 4.22. Here's the short version. The full write-up is linked at the bottom.
**The one idea that explains most of it:** everything, including the node OS, is managed by an operator inside the cluster, and the whole platform updates from one signed release image.
- **Infrastructure:** IPI, UPI, Agent-based or Assisted. After install, the Machine API (MachineSets, MachineHealthChecks) manages machines on supported platforms.
- **Node OS:** RHCOS is immutable. Config goes through MachineConfigs and MachineConfigPools, applied by the MCO. Edit a file by hand and the MCD flags drift and marks the node Degraded.
- **Control plane:** exactly 3 nodes for prod. Bare metal allows 5 (survives 2 etcd failures). Avoid 4: it tolerates the same single failure as 3 but needs one more healthy member for quorum.
- **Cluster services:** OVN-K (the only supported CNI since 4.17), the HAProxy router, the registry (starts as `Removed` without shared storage), monitoring. One cluster operator each.
- **Lifecycle:** the CVO drives updates. Control plane first, then the MCO rolls nodes pool by pool. Pause the worker MCP to split the update across maintenance windows, but don't leave it paused.
- **Add-ons:** GitOps, Pipelines, Virtualization and the rest come through OLM, on their own release cycles.
**Decide these before day one:**
FIPS: install time only
Disconnected: mirror registry and update delivery (OSUS) planned up front
Infra nodes: much easier to plan than to retrofit. Tainted infra nodes running only infra workloads also don't count toward subscriptions
etcd backups: the operator runs etcd, but recovering from quorum loss is on you
The post has an interactive explorer. Click a component and it shows what it does, which operator owns it, its namespace, and a read-only `oc` command to inspect it: https://aib-devops.com/en/blog/openshift-architecture/
Disclosure: it's my own blog. No ads, no tracking, no signup.
Corrections welcome, especially from anyone running HCP or two-node in production. I'd like to know what I'm missing there.
r/openshift • u/Reyleigh- • Sep 15 '26
General question Cleared EX280 Today
Was expecting a 300 score but got 250 it seems there is very strict checking so make sure to double check all spellings and type slowly.
r/openshift • u/james4765 • Apr 30 '26
Good to know copy.fail (CVE-2026-31431) mitigation in OpenShift
After working through the mitigation in our RHEL hosts, I went ahead and put together the MachineConfiguration YAML necessary to disable the vulnerable driver:
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
name: 99-mitigate-cve-2026-31431-master
labels:
machineconfiguration.openshift.io/role: master
spec:
config:
ignition:
version: 3.4.0
kernelArguments:
- initcall_blacklist=algif_aead_init
---
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
name: 99-mitigate-cve-2026-31431-worker
labels:
machineconfiguration.openshift.io/role: worker
spec:
config:
ignition:
version: 3.4.0
kernelArguments:
- initcall_blacklist=algif_aead_initr/openshift • u/Icy_Football8619 • Sep 21 '25
Discussion Running local AI on OpenShift - our experience so far
We've been experimenting with hosting large open-source LLMs locally in an enterprise-ready way. The setup:
Model: GPT-OSS120B
Serving backend: vLLM
Orchestration: OpenShift (with NVIDIA GPU Operator)
Frontend: Open WebUI
Hardware: NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM)
Benchmarks
We stress-tested the setup with 5 → 200 virtual users sending both short and long prompts. Some numbers:
~3M tokens processed in 30 minutes with 200 concurrent users (~1666 tokens/sec throughput).
Latency: ~16s Time to First Token (p50), ~89 ms inter-token latency.
GPU memory stayed stable at ~97% utilization, even at high load.
System scaled better with more concurrent users – performance per user improves with concurrency.
Infrastructure notes
OpenShift made it easier to scale, monitor, and isolate workloads.
Used PersistentVolumes for model weights and EmptyDir for runtime caches.
NVIDIA GPU Operator handled most of the GPU orchestration cleanly.
Some lessons learned
Context size matters a lot: bigger context → slower throughput.
With few users, the GPU is underutilized, efficiency shows only at medium/high concurrency.
Network isolation was tricky: GPT-OSS tried to fetch stuff from the internet (e.g.
tiktoken), which breaks in restricted/air-gapped environments. Had to enforce offline mode and configure caches to make it work in a GDPR-compliant way.Monitoring & model update workflows still need improvement – these are the rough edges for production readiness.
TL;DR
Running a 120B parameter LLM locally with vLLM on OpenShift is totally possible and performs surprisingly well on modern hardware. But you have to be mindful about concurrency, context sizes, and network isolation if you’re aiming for enterprise-grade setups.
We wrote a blog with mode details of our experience so far. Check it out if you want to read more: https://blog.consol.de/ai/local-ai-gpt-oss-vllm-openshift/
Has anyone else here tried vLLM on Kubernetes/OpenShift with large models? Would love to compare throughput/latency numbers or hear about your workarounds for compliance-friendly deployments.
r/openshift • u/ProofPlane4799 • Feb 10 '26
Blog I want to share a publication that Red Hat honored me with after implementing Red Hat OpenShift.
This journey started in 2024. We were looking for an exit strategy after Broadcom created a huge nightmare. Nowadays, we are migrating our storage backend to Pure Storage using Flash Arrays, with Portworx as our new CSI, Portworx Backup as a backup tool, and synchronous and Asynchronous replication for a multi-site solution.
The endeavor has challenged our preconceived assumptions; however, a year later, it has provided perspective on all the knowledge gathered, gratitude for my team, and respect for Red Hat.
r/openshift • u/Amine-LG • Oct 22 '26
Good to know I passed EX280 and EX180, then built something to help me remember what I'd learned
Earlier this year, I passed EX280 and EX180 with strong scores.
My preparation was a bit unusual. Back in January, I used AI to help create a roughly 1,000-page study book covering the EX280 objectives. By the time I started the official Red Hat courses in March, I was already well prepared and found them relatively easy to follow.
But recently, while preparing for an interview, I realized how much I'd forgotten after not practicing regularly.
I didn't want to go through another thousand pages, especially since I no longer have access to the official course materials. So I started experimenting with something I'd actually enjoy using to refresh my knowledge.
That's how Root To Run started.
Over the past several days, I've been building it with Claude Code and Codex, repeatedly rewriting lessons, rebuilding diagrams, testing interactive exercises, and correcting technical explanations. Some things took quite a few attempts before I was happy with them.
What started as a personal learning project grew into a free, open-source platform covering Linux, containers, Kubernetes, and OpenShift.
It's still in beta, and I plan to work through the lessons myself, fixing mistakes and improving things along the way.
I figured others might find it useful too, so I'm sharing it here. Feedback, corrections, issues, and contributions are all welcome, especially from people who work with OpenShift regularly.
Website: https://roottorun.aminelg.com/
r/openshift • u/mafike1 • Sep 20 '25
Discussion Learn OpenShift the affordable way (my Single-Node setup)
Hey guys, I don’t know if this helps but during my studying journey I wrote up how I set up a Single-Node OpenShift (SNO) cluster on a budget. The write-up covers the Assisted Installer, DNS/wildcards, storage setup, monitoring, and the main pitfalls I ran into. Check it out and let me know if it’s useful:
https://github.com/mafike/Openshift-baremetal.git
r/openshift • u/David-Pasek • May 13 '25
General question Red Hat OpenShift Virtualization
Does anybody use Red Hat OpenShift Virtualization in production?
Today I had a full day test drive of Red Hat OpenShift Virtualization (Red Hat + Cisco UCS), and even the theory (presentations) sounds relatively nice, during the practice (hands-on labs), I found a lot of "challenges" due to the obvious fact that OpenShift is primarily designed and developed for K8s use case.
We are looking for a "VMware by Broadcom" alternative, and "RedHat by IBM" would be a logical Enterprise alternative for KVM-based virtualization, but ...
Even if I would accept containerized QEMU (kubevirt), storage volumes via K8s CSI orchestration (something like VMware VVOLs), and potential network complexity (multus CNI plugin), the overall platform does not seem to be ready for production-ready operations of Enterprise-ready VMs.
Is my observation correct, or does somebody use Red Hat OpenShift Virtualization for Enterprise-ready VMs?
r/openshift • u/joshthesysengineer • Apr 26 '25
Fun OKD Homelab Deployment Guide
Hey guys I am a long time creeper on this form from a few different accounts. Alot of people have helped me and I wanted to give something back especially after my struggle over the past few years learning more about openshift, containers, and linux as a whole.
Our journey starts when I interviewed for a position where they used Openshift. I never used it and up until that point I ignored kubernetes because I didn't really have a reason to have all that infrastructure. I just ran containers in proxmox and some docker containers. That was all the experience I had. Fast forward to them taking a chance on my and I was in charge of administrating a cluster and maintaining high up time. I couldn't really learn on the job because money was on the line so I bought myself a Dell r630 and went for it.
I had tons of struggles and had so many questions. I followed guide after guide and it felt like it was impossible. A redhat engineer even made an awesome video showing him deploying okd 4.5 cluster and I spent hours scrubbing through to understand what was going on. I finally deployed my cluster and learned so much and I hope I can inspire atleast one person to go for it. That being said I made a tool to help out people deploying clusters similar to mine. How the tool works is the input you put into your cluster updates the rest of the pages directions for you to build your cluster. For example when you put in what your services node's IP is it updates the the dns config file to have the ip you put in. It may be a bit buggy I just launched this after working on it all week but I wish I would've had something like it instead of just documentation that I had to make work in my use case. Hopefully it helps someone out. I'm not expert by any means but any knowledge I can share I will about my process and how I deployed in proxmox.
Check it out here: https://clusterhelper.com/
Created a YouTube Video Discussing it: https://youtu.be/xQFJpa74iQ0?si=dJGMmH-2iifayDIP
r/openshift • u/Rhopegorn • Oct 06 '26
Blog RHCOS 10 - Red Hat’s new worker node OS for OpenShift
Red Hat OpenShift uses a purpose-built, immutable operating system for cluster nodes, commonly known as Red Hat Enterprise Linux CoreOS (RHCOS). This operating system, which uses the same kernel and hardware compatibility from Red Hat Enterprise Linux (RHEL), ships with a minimal-footprint operating environment, with its configuration entirely orchestrated by OpenShift and Kubernetes-based APIs.
Starting in Q4 2026, OpenShift will ship the newest iteration of this operating system, RHCOS 10, by default for new installations, and all certified ISV containers and operators are expected to support it.
RHCOS 10 is available as a Technology Preview in OpenShift 4.21 and 4.22 already. RHCOS 9 will remain available for newer OpenShift versions at least into mid 2027.
r/openshift • u/ManofGilnockie • Apr 30 '24
Fun They told me Openshift was an easy deployment.

I'm on my first OCP deployment. I was told the assisted installer was going to be easy. Then the day came when I learned the assisted installer does not work in a disconnected environment. I have executed every method of deploying quay. Agent install and the 3rd iteration of mirror-quay got the cluster deployed. Now I need deploy a chain of Quay mirrors to pipe all the other repos I need to deploy apps on the cluster. I haven't had a bad day yet. I love what I do.
r/openshift • u/Old-Astronomer3995 • 15d ago
General question OpenShift 5.0 release
Hi
I just noticed that there is available https://docs.redhat.com/en/documentation/openshift_container_platform/5.0/html/release_notes/ocp-5-0-release-notes
but there is no videos on RedHat and OpenShift YouTube channels or any blog posts about it. It looks suspicious.
Any inside info about it?
r/openshift • u/dzuczek • Nov 20 '20
Announcing Red Hat OpenShift Service on AWS
r/openshift • u/kiarash-irandoust • Sep 24 '20
I’m sorry OpenShift, I’ve taken you for granted…(the evidence)
r/openshift • u/Danielr2010 • Jun 03 '26
Discussion Created a must-gather analyzer web page (yes, co-work with AI). Seeking some feedback
Howdy y'all. I support OCP, and some proprietary Operators, in my job. Had a hard as hell time finding a useful must-gather analyzer. omc works pretty well, but I'm a visual person.
https://github.com/fumbles/openshift-must-gather-analyzer
So I spent some time with IBM Bob and made this based off the abandoned https://github.com/elmiko/camgi.rs repo.
Let me know what you think, feedback, create an issue, etc.
r/openshift • u/Rhopegorn • Feb 25 '25
Good to know What's New for Developers in Red Hat OpenShift 4.18
Red Hat OpenShift 4.18, based on Kubernetes 1.31 and CRI-O 1.31 releases, is now Generally Available (GA). This article highlights notable updates in this release for Developers with OpenShift.
r/openshift • u/srednax • Aug 23 '24
Fun MicroShift on RHEL 9.4 for Edge
After spending 2 days trying to get Nvidia's Jetpack 6 installed on my Nvidia Jetson Orin Nano 8GB, just so I could install RHEL 9.4, I finally have a running system. The board is an ARM64-based board with a boatload of CUDA cores for AI. I also installed and configured MicroShift on it. It is not running anything major on it just yet.

This particular board is in my Hiwonder Jethexa robot, a six-legged robot with depth-sensing camera and LIDAR. The goal is to run all of the seperate components of the ROS 2 framework in pods, so I can easily exchange them for new version. I have another Nvidia Jetson Orin NX 16GB running on my network, but that's more of a desktop. It also runs RHEL 9.4 and MicroShift. The pods will be managed through ArgoCD, which runs on my mini PC running SNO (Single Node OpenShift).

I have done some tests with accessing serial ports from inside pods. The SCCs were a major hassle to sort out. In the end I just went with 'privilged' and called it a day.
The installation guide for RHEL 9.4 and Microshift on the Nvidia Jetson Orin series should be out Real Soon Now (TM). It was not written by me, I just tested it.
If you have a spare host, give MicroShift a go. It may not have all of the features of full-fat OpenShift, but for systems like these, it's perfect.
Edit: Reddit ate my robot picture.
r/openshift • u/brokenpip3 • Oct 16 '19
Introducing Red Hat OpenShift 4.2