NVIDIA is teaching Rust to write the code that actually runs on the GPU

01

WHAT ACTUALLY CHANGED

NVIDIA announced CUDA Rust on September 8, committing to grow native Rust GPU programming through 2027 and beyond. Developers could already control a GPU from Rust, but the kernel doing the parallel work often had to be written in CUDA C++ or another language. The new tools compile Rust kernels directly into PTX, the instruction format that NVIDIA drivers turn into executable GPU code.

The first track, cuda-oxide, follows the familiar SIMT model where a programmer describes what one thread should do and launches thousands of copies. It is a custom Rust compiler backend that routes kernel functions through Rust MIR, the Pliron intermediate representation framework, LLVM, and finally PTX. Host code and device code can live in one Rust file and build through a Cargo command.

cuda-oxide makes some concurrency rules explicit in the type system. Its DisjointSlice type gives each thread exclusive write access to its own element. A launch contract records the intended dimensions and block size, then validates the requested launch against that declaration and the actual device limits. Conflicting mutable and immutable access can fail during compilation instead of becoming a race that appears three weeks later on one expensive machine.

The second track, cutile-rs, operates on tiles of data rather than individual threads. A programmer partitions a mutable tensor into non-overlapping pieces. The compiler then decides how those tiles map onto GPU warps, blocks, and Tensor Cores. cutile-rs uses stable Rust and compiles kernels at runtime through CUDA Tile IR, while cuda-oxide currently needs a pinned nightly Rust toolchain and a local LLVM setup.

NVIDIA recommends trying the tile approach first and dropping to SIMT when a job needs direct control of threads or memory. The research behind cutile-rs reports 7 terabytes per second on element-wise operations and about 2 petaflops for matrix multiplication on a B200 GPU, roughly 96 percent of cuBLAS performance. Those measurements come from the project authors, and both tools remain experimental.

02

WHY THIS MATTERS

The software around AI models is steadily filling with Rust. Inference engines, serving systems, drivers, and agent runtimes all need high performance without making every pointer a tiny wager. NVIDIA says its Nova Linux driver is written in Rust and its Dynamo inference platform has a Rust core. The kernel itself was the awkward gap where developers often had to leave the language and its safety rules behind.

GPU mistakes are uniquely irritating because parallel workers do not arrive at shared memory in a guaranteed order. Two threads can touch the same address, one can overwrite the other, and the result may change with timing, workload, or hardware. A test can pass repeatedly before the bug emerges under production pressure. Moving more ownership checks into compilation makes some of those failures ordinary errors instead of expensive folklore.

The two-track design also admits that there is no single correct level of control. Tile programming asks the compiler to manage thread mapping and memory layout, giving up some low-level freedom for portability and safety. SIMT exposes the machinery when expert developers need to squeeze the hardware directly. One is a well-marked highway. The other lets you open the hood while moving.

This could make custom AI infrastructure easier to build in a single language. A Rust inference engine could eventually contain its host logic, scheduling, memory ownership, and performance-critical kernels without crossing a foreign-language boundary for the final few meters. NVIDIA also says it plans interoperability with CUDA C++ and CUDA Python, so adopting one frontend should not isolate a project from the others.

The responsible reaction is curiosity, not migration panic. cuda-oxide is early alpha. cutile-rs has incomplete feature coverage and APIs that will change. Shared memory in the SIMT path still requires unsafe Rust, which is also where many fast kernels do their most important work. The projects show a credible direction, but they are currently a laboratory bench with excellent warning labels.

03

WHERE IT COULD HELP

• Write native CUDA kernels without leaving a Rust codebase
• Catch conflicting buffer access before launching work on a GPU
• Build tile-based matrix and attention operations for AI inference
• Prototype safer custom kernels for serving and scientific computing
• Compare high-level tile portability with low-level SIMT control