r/CUDA • • Oct 26 '25

Built a CUDA editor because I was sick of switching tools

Post image
585 Upvotes

I was using 4 sometimes 6 different tools just to write CUDA. vs code for coding, nsight for profiling, many custom tools for benchmarking and debugging, plus pen to calc the performance "I was cooked"

So I built code editor for CUDA that does it all:

  • Profile and benchmark your kernels in real-time while you code

  • Emulate multi-GPU without the hardware

  • Get AI optimization suggestions that actually understand your GPU "you can use local llm to cost you 0$"

It's free to use if you use your local LLM :D Still needs a lot of refinement, so feel free to share anything you'd like to see in it

https://www.rightnowai.co/


r/CUDA • • Apr 27 '25

meme for the weeked

Post image
479 Upvotes

r/CUDA • • 3d ago

An Open Letter to NVIDIA's Next CEO: It's Time to Open-Source CUDA

402 Upvotes

To NVIDIA's next CEO,

As NVIDIA enters a new era, you have an opportunity to make a decision that could fundamentally reshape accelerated computing: open-source the CUDA software stack.

Not NVIDIA's hardware architecture, chip designs, or proprietary engineering. Just the software.

AMD has already demonstrated through ROCm that an open-source computing stack can coexist with proprietary hardware. NVIDIA could do the same while maintaining its technological leadership.

For nearly two decades, CUDA has been one of NVIDIA's greatest competitive advantages. But its proprietary nature also creates barriers to interoperability, independent innovation, and community participation.

Opening CUDA could benefit NVIDIA and the broader computing industry:

  • Accelerated innovation: Researchers, developers, and universities could directly improve CUDA's compilers, runtimes, and libraries.

  • Better Linux support: Distributions could integrate, maintain, and troubleshoot the software more effectively.

  • Transparency and trust: Developers and organizations could independently audit the code they depend on.

  • Longer hardware lifespans: Community maintenance could potentially extend support for older GPUs.

  • A stronger ecosystem: Community contributions could expand CUDA's capabilities while NVIDIA continues competing through superior hardware.

AI Is Changing the Equation

There's another compelling reason to act now: AI-assisted development is rapidly reducing the cost and complexity of reverse engineering proprietary software.

With sufficient inference tokens, computing resources, and persistence, increasingly sophisticated AI agents are enabling individuals and small teams to undertake projects that once required substantial engineering organizations.

Reimplementing CUDA remains extraordinarily difficult, but projects like ZLUDA already demonstrate that independent compatibility efforts are possible. AI will likely accelerate that trend.

If open-source CUDA-compatible implementations become increasingly inevitable, wouldn't NVIDIA be better positioned by leading that transition instead of resisting it?

By opening CUDA itself, NVIDIA could establish the reference implementation, shape development priorities, influence standards, and remain central to the ecosystem.

By keeping it closed, NVIDIA risks watching independent alternatives evolve outside its influence.

An Opportunity for Lasting Leadership

I'm not suggesting NVIDIA release everything overnight. A phased transition beginning with the compiler, runtime, and core libraries would be reasonable.

Nor am I suggesting CUDA must immediately become hardware-agnostic. NVIDIA can continue optimizing its software for its own GPUs while maintaining proprietary hardware architectures.

The point is simple: NVIDIA's competitive advantage should come from building the best technology, not from restricting access to the software ecosystem surrounding it.

Jensen Huang helped establish NVIDIA as the leader in accelerated computing. His successor has an opportunity to establish an equally enduring legacy: making CUDA an open foundation for the next generation of computing.

NVIDIA has demonstrated that it can lead through engineering excellence. Now demonstrate that it can lead through openness.

Sincerely,

A Linux user, GPU enthusiast, and advocate for open-source computing.

ADDENDUM:

I fully understand that CUDA is NVIDIA's cash cow and one of its greatest competitive advantages. That's precisely why I believe NVIDIA should consider open-sourcing it.

CUDA's dominance depends heavily on developer dependency, but that lock-in isn't permanent. Open-source alternatives, hardware-agnostic frameworks, and CUDA compatibility projects are steadily eroding it. AI-assisted development could accelerate that process dramatically, potentially enabling small teams to create viable alternatives much sooner than expected.

The real threat isn't someone cloning CUDA tomorrow. It's CUDA gradually becoming irrelevant as competing open-source ecosystems mature.

If developers can run their workloads efficiently across competing hardware without depending on CUDA, NVIDIA loses much of its software-driven competitive advantage.

By open-sourcing CUDA now, NVIDIA could help establish it as the universal standard for accelerated computing while retaining substantial influence over its development. Its proprietary hardware, optimizations, and engineering expertise would remain competitive advantages.

Competition is coming regardless. The question is whether NVIDIA wants to lead an open computing ecosystem or eventually adapt to one established by its competitors.


r/CUDA • • Jan 17 '25

LeetGPU – Write and execute CUDA on the web, no GPU required, for free

319 Upvotes

We found that there was a significant hardware barrier for anyone trying to learn CUDA programming. Renting and buying NVIDIA GPUs can be expensive, installing drivers can be a pain, submitting jobs can cause you to wait in long queues, etc.

That's why we built LeetGPU.com, an online CUDA playground for anyone to write and execute CUDA code without needing a GPU and for free.

We emulate GPUs on CPUs using two modes: functional and cycle accurate. Functional mode executes your code fast and provides you with the output of your CUDA program. Cycle accurate mode models the GPU architecture and provides you also with the time your program would have taken on actual hardware. We have used open-source simulators and stood on the shoulders of giants. See the help page on leetgpu.com/playground for more info.

Currently we support most core CUDA Runtime API features and a range of NVIDIA GPUs to simulate on. We're also working on supporting more features and adding more GPU options.

Please try it out and let us know what you think!


r/CUDA • • Mar 10 '25

Would learning CUDA help me land a job at Nvidia?

318 Upvotes

I have a few years of experience in Java and Angular but pay is shitty. I was wondering if I learn CUDA, would that help me land a job at Nvidia? Any advice or suggestions is greatly appreciated. Thank you!


r/CUDA • • Nov 30 '25

RX 5700 XT now has full CUDA Driver API access – 51 °C

Post image
263 Upvotes

“RX 5700 XT, 6-year-old card.
No ROCm, no ZLUDA, no PTX translation.
Just two DLLs → full CUDA Driver API access.
51 °C while running cuLaunchKernel.
Proof attached.”

Update 2025-12-03:

Verified that the CUDA API can be fully replaced, with complete PTX compatibility.

The underlying resource library supports up to 256-bit atomic operations.

Full system-level SVM capability is enabled.

Multi-modal topology functionality is available.

Complete zero-copy networking capability is implemented.

Direct universal bridging support for all three major GPU vendors is achieved.

Note: The library will be released this weekend, and detailed evidence of compatibility will be demonstrated via a scheduled live session.

Update 2025-12-08: Lu Ban Preview v3.0.0 — NOW LIVE 292 functions. Pure C. Zero vendor lock-in.

New in this build: • 92 embedded cJSON (zero external deps) • 27 new retryix_gpu_* register-level functions (WinRing0 direct access) • Complete svm_atomic_* + zerocopy_* stack • Clock control, VRAM r/w, doorbell ring, soft reset…

Download & test: https://github.com/Retryixagi/Retryixagi-RetryIX-OpenCL-V3.0.0-Lu-Ban_Preview

⚠️ This is a PREVIEW build.
Extreme functions (GPU register tweaking, aggressive clock, raw RDMA) are fully exposed.
Your card won’t burn (we keep it under 60 °C), but you might accidentally turn it into a rocket.
Play responsibly. You’ve been warned.

Live demo + Q&A this weekend. Bring your old cards — they’re about to feel young again.

One DLL to rule them all.
No CUDA. No ROCm. Just Lu Ban.

RetryIX #LuBan #OpenCL #CUDA #ZeroCopy #256bitAtomics #HeterogeneousComputing #Taiwan


r/CUDA • • Feb 28 '25

LeetGPU Challenges - LeetCode for CUDA Programming

224 Upvotes

Following the incredible response to LeetGPU Playground, we're excited to introduce LeetGPU Challenges - a competitive platform where you can put your CUDA skills to the test by writing the most optimized GPU kernels.

We’ve curated a growing set of problems, from matrix multiplication and agent simulation to multi-head self-attention, with new challenges dropping every few days!

We’re also working on some exciting upcoming features, including:

  • Support for PyTorch, TensorFlow, JAX, and TinyGrad

  • Multi-GPU execution

  • H100, V100, and A100 support

Give it a shot at LeetGPU.com/challenges and let us know what you think!


r/CUDA • • Jan 08 '26

I made 64 swarm agents compete to write gpu kernels

205 Upvotes

I got annoyed by how slow torch.compile(mode='max-autotune') is. on H100 it's still 3 to 5x slower than hand written cuda

the problem is nobody has time to write cuda by hand. it takes weeks

i tried something different. instead of one agent writing a kernel, i launched 64 agents in parallel. 32 write kernels, 32 judge them. they compete and teh fastest kernel wins

the core is inference speed. nemotron 3 nano 30b runs at 250k tokens per second across all the swarms. at that speed you can explore thousands of kernel variations in minutes.

there's also an evolutionary search running on top. map-elites with 4 islands. agents migrate between islands when they find something good.

  • llama 3.1 8b: torch.compile gets 42.3ms. this gets 8.2ms. same gpu

  • Qwen2.5-7B: 4.23×

  • Mistral-7B: 3.38×

planning to open source it soon. main issue is token cost. 64 agents at 250k tokens per second burns through credits fast. still figuring out how to make it cheap enough to run.

if anyone's working on kernel stuff or agent systems would love to hear what you think because from the results, we can make something stronger after I open-source it:D

https://rightnowai.co/forge


r/CUDA • • Aug 28 '26

Understand Blackwell B200 attention kernel from scratch in CUDA (Visual Guide)

Post image
188 Upvotes

I spent last couple months building a visual guide to B200 attention in CUDA/PTX: 14 progressively optimized kernels and 60 diagrams, going from a naive baseline to 94.4% of FlashAttention-4.

The blog first intuitively explains the baseline, then adds one optimization at a time, with detailed diagrams, concise explanations, and code.

And in the capstone project, you'll generate videos using the kernel you understand.

📝 Blog post: https://iaroslavelistratov.github.io/b200-attention/ 

⭐ Repo: https://github.com/IaroslavElistratov/b200-attention


r/CUDA • • Dec 01 '25

What is the best way to become a CUDA/GPU Kernel Engineer?

178 Upvotes

Hello. I'm very interested to become a CUDA or GPU engineer. Currently, I'm working as a software engineer and studying Master's in Computer Engineering. I have taken classes in Machine Learning and NLP. I like studying in subjects that are related to AI and I want to dive deeper. I have come across CUDA in some YouTube videos and I got very interested to it. I want to learn parallel programming and GPU engineering in AI applications but I'm concerned that if there are any pre-requisites that I should have done before starting on CUDA. I'm pretty much beginner in this field therefore I wonder if I should train some models in high-level frameworks like PyTorch beforehand, and later start on CUDA to make further optimizations. Any comment will be appreciated. Thanks.


r/CUDA • • Oct 27 '25

Lessons learned from GPU Programming in Python (Numba, CuPy, NCCL, mpi4py) — and how it compares to C-CUDA

163 Upvotes

I’m Lena Oden, a professor of Technical Computer Science. I teach and research high-performance computing and GPU programming. Over the last few years, I’ve been exploring how well Python frameworks can compete with C-CUDA — both in single-GPU and multi-GPU contexts.

A few years ago, I published a paper called “Lessons learned from comparing C-CUDA and Python-Numba for GPU-Computing.” [1]
In it, I compared the performance and generated PTX code of C-CUDA and Numba-CUDA kernels, and discussed some practical tips on how to write more efficient Numba GPU code.

More recently, I’ve published a new article:
“Efficient Multi-GPU Programming in Python: Reducing Synchronization and Access Overheads.”[2]
This one focuses on Python-based multi-GPU programming, using Numba, CuPy, NCCL, and mpi4py, and analyzes where synchronization and data access overheads occur — along with strategies to mitigate them.

I’d be really interested to hear from others who use these tools:

  • Do you also work with Numba, CuPy, or Python-based multi-GPU setups?

  • What kinds of applications are you using ( I am really interested in "real world" applications.

  • Any tricks or pain points you’d like to share?

If anyone doesn’t have access to the papers, feel free to message me — I’m happy to share a copy.

Looking forward to exchanging experiences!

— Lena


r/CUDA • • Feb 13 '26

In robotics, CUDA commands the highest salary premium (+72%, $214K median) - analysis of 5,878 job postings [OC]

Thumbnail
gallery
162 Upvotes
Analyzed 5,878 robotics job postings. Of those, 2,181 disclosed compensation (market baseline: $130K).

Top 5 salary premiums:
- CUDA: $214,838 (+72%)
- JAX: $212,750 (+66%)
- TensorRT: $212,750 (+66%)
- K8s: $210,700 (+65%)
- MLOps: $207,500 (+63%)

Full skills landscape (treemap) based on 5,878 total job postings in the second image.

r/CUDA • • Jun 17 '26

Breaking into GPU Infrastructure / GPU Programming Feels Overwhelming. How Did You Figure Out What to Learn?

127 Upvotes

I have 10+ years of software engineering experience, mostly backend development and infrastructure.

Lately I’ve become interested in GPU infrastructure, HPC, performance engineering, and eventually GPU programming. I’ve been reading books like AI Systems Performance Engineering, Programming Massively Parallel Processors, and Computer Architecture: A Quantitative Approach.

The problem is that every time I look at job descriptions, I end up with a completely different list of skills.

Some roles want:

  • CUDA and GPU kernel optimization

  • Computer architecture knowledge

  • NCCL, RDMA, InfiniBand

  • Kubernetes and Slurm

  • Distributed training

  • Performance profiling and benchmarking

  • Linux kernel knowledge

  • Cloud infrastructure

Other roles seem much more focused on operating GPU clusters and supporting AI workloads at scale.

I’m considering doing a master’s degree, but even when I look at programs like OMSCS, Computer Engineering, or Systems-focused master’s degrees, it feels like they teach foundational concepts but not necessarily the practical skills companies are hiring for.

As someone coming from a traditional software engineering background, I’m struggling to identify:

  1. What skills are truly foundational versus “nice to have”?

  2. If you had 6–12 months to prepare for GPU infrastructure or GPU performance engineering roles, what would you focus on first?

  3. Did a master’s degree help you break into this field, or was self-study and project work more valuable?

  4. For those already working in GPU infrastructure, ML infrastructure, HPC, or GPU programming, what did your path actually look like?

Right now it feels like there are five different careers hiding behind the phrase “GPU engineer,” and I’m trying to figure out which path is the most realistic transition from a backend/infrastructure background.

I’d appreciate hearing from people who made a similar transition.


r/CUDA • • Sep 05 '25

CUDA docs, for humans

127 Upvotes

My colleague at Modal has been expanding his magnum opus: a beautiful, visual, and most importantly, understandable, guide to GPUs: https://modal.com/gpu-glossary

He recently added a whole new section on understanding GPU performance metrics. Whether you're just starting to learn what GPU bottlenecks exist or want to deepen your understanding of performance profiles, there's something here for you.


r/CUDA • • Dec 15 '25

How to get into GPU programming?

121 Upvotes

I have experience developing bare metal code for microcontrollers and I have a really boring job using it to control electromechanical systems. I took a course in computer architecture and parallel programming in my Masters and I would love to do something along those lines. Can I still switch to this domain as my career without having any experience in it, but having done courses and projects? Thanks


r/CUDA • • Feb 16 '25

I made an animated video explaining what Tensor Cores are

Thumbnail
youtu.be
120 Upvotes

r/CUDA • • Jul 12 '26 WOW

Open sourced my GPU engineering learning notes

119 Upvotes

Been documenting my GPU architecture learning journey publicly, chapter by chapter. It's MIT licensed and structured like a book (sequential, builds concept by concept). Looking for people to point out mistakes, suggest better explanations, or add diagrams

https://github.com/BEFORE-GPU-ENGINEERING/BEFORE-GPU-ENGINEERING


r/CUDA • • Sep 23 '26

Giveaway: Grokking Parallel Programming with CUDA is now in MEAP — 5 free ebooks + 50% off

Thumbnail
gallery
115 Upvotes

Hi again, r/CUDA!

We’re back with another CUDA release from Manning, and we have five ebooks to give away.

Grokking Parallel Programming: With Examples in CUDA by SungHwan Yun has just entered the Manning Early Access Program.

Learning CUDA syntax is one thing. Learning to look at a problem and see what can run concurrently—while navigating memory access, synchronization, and race conditions—is the real challenge. That’s the way of thinking this book aims to develop.

It starts with the fundamentals and builds into practical topics including:

• Kernels, threads, blocks, and grids
• CPU–GPU data movement
• Map and Reduction patterns
• Race conditions and synchronization
• Out-of-bounds memory access
• Knowing when parallelization is—or isn’t—worthwhile

The book is aimed at programmers who know basic C but are new to CUDA and parallel programming. The examples can run in a free browser-based environment, so you don’t need your own NVIDIA GPU to get started.

Giveaway: 5 free ebooks

To enter, leave a comment that adds something useful to the discussion. For example:

• What was the hardest CUDA concept for you to learn?
• What do beginner CUDA resources commonly explain poorly?
• What problem would you like to parallelize?
• What advice would you give someone writing their first kernel?

We’ll award an ebook to each of the five comments that contribute the most to the conversation—not simply those with the most votes. One entry per person. I’ll contact the selected members by DM.

50% off for r/CUDA

Use code MLYUN50RE for 50% off:

https://www.manning.com/books/grokking-parallel-programming

Thanks again to the mods for letting us share this—and to everyone here who has welcomed our previous posts.

What made CUDA finally “click” for you? Or, if you’re still learning, what’s the biggest thing standing in your way?

Thank you.

Cheers,

Stjepan


r/CUDA • • Jan 07 '26

I built an open source GPU database with 2,824 GPUs

Thumbnail
github.com
114 Upvotes

I needed GPU specs for a project and couldn't find a good structured database. So I built one.

2,824 GPUs across NVIDIA, AMD, and Intel. Each GPU has up to 55 fields including architecture, memory, clock speeds, and kernel development specs like warp size, max threads per block, shared memory per SM, and registers per SM.

NVIDIA: 1,286 GPUs

AMD: 1,292 GPUs

Intel: 180 GPUs

Free to use. Apache 2.0 license.

GitHub: https://github.com/RightNow-AI/RightNow-GPU-Database


r/CUDA • • Dec 04 '25

How to start learning GPU architecture and low-level GPU development?

112 Upvotes

I'm trying to get into the GPU world and I’m a bit confused about the right starting point. I have some experience with embedded systems, FPGA work, and programming in C/Python/Verilog, but GPUs feel like a much bigger area.

I’ve come across topics like CUDA, OpenCL, pipelining, RISC-V — but I’m not sure what order to learn things or what resources are best for beginners.

What I’m looking for:

A clear starting path to learn GPU architecture / GPU firmware / compute programming

Beginner-friendly resources, books, or courses

Any recommended hands-on projects to build understanding

Any pointers would be really helpful!


r/CUDA • • Feb 27 '25

Tensara: Leetcode for CUDA kernels!

Thumbnail
tensara.org
110 Upvotes

r/CUDA • • Nov 04 '25

My interview process with NVIDIA for Senior Deep Learning Engineer — is this normal?

107 Upvotes

Hey everyone,

I wanted to share my experience interviewing with NVIDIA for a Senior Deep Learning Engineer position and ask if this kind of delay is normal.

  • Round 1:

    • Interview 1 (DSA): A data structures & algorithms round. At the end, the interviewer told me I was moving forward.

    • Interview 2 (Hiring Manager): Focused on project alignment, technical details of my past work, and NVIDIA’s software stack. The next day, I got confirmation that I had passed Round 1.

  • Round 2: They scheduled two technical interviews — Deep Learning Fundamentals and OOP. I completed both (the OOP one was last Monday).

After that, I haven’t received any updates. I reached out to the recruiter yesterday, and she said she’d check with the team and get back to me, but so far there’s been no response. My candidate portal still shows that I’m “in process.”

What’s confusing is that when I had my first interview, the role was open on their website. Then it disappeared for a while, and now it’s visible again both on their careers page and LinkedIn.

Apparently there’s a Round 3 with three more interviews if I pass this one, but I have no idea where things stand right now.

Is this kind of silence normal with NVIDIA’s hiring process?
Would love to hear from anyone who’s been through something similar.


r/CUDA • • Apr 05 '26

A Beginner’s Guide to GPU Memory Hierarchies: Mapping 2D Tiled GEMM to Hardware [Source + Commentary]

Thumbnail
gallery
105 Upvotes

r/CUDA • • Dec 05 '25

Nvidia released cuTile Python

Thumbnail github.com
99 Upvotes