GPU Programming with Triton (Accelerate AI training and inference)

Until recently, writing GPU kernels for LLM training and inference meant learning low-level programming tools like CUDA and C++. Triton, an open source, Python-based DSL created by OpenAI, bridges the gap between high-level machine learning frameworks and low-level GPU programming. Triton is built into PyTorch 2 and backed by NVIDIA, Intel, AMD, and Red Hat.

In this book, you'll learn how to work within the Triton ecosystem, from writing your first kernel to implementing advanced LLM features like FlashAttention and Native Sparse Attention. You'll use Triton to deliver the kernel-level control, fusion power, and acceleration that frameworks like PyTorch need under the hood without dropping down to CUDA and C++.

In this practical book written for readers with no previous GPU programming experience, author Harshwardhan Fartale introduces Triton’s innovative block-level programming model that replaces the tedious manipulation of low-level threads required by CUDA. Written for the latest hardware and LLMs, this book teaches GPU programming and Triton together, in Python, by profiling real workloads, identifying bottlenecks, and understanding why each optimization (coalescing, tiling, shared memory, reductions, and fusion) actually works.

As you go, you'll build the kernels that power modern AI systems, including FlashAttention, Native Sparse Attention, sparse matrix multiplication, and on-chip fused operations. You'll learn to profile real workloads, find the bottlenecks, wrap your kernel for production, and integrate it end to end into PyTorch. Each chapter includes handpicked practice problems designed to build the fluency that makes working in Triton feel like second nature.

What's inside

• Writing production-grade Triton kernels
• Core optimization techniques
• Building FlashAttention, Native Sparse Attention, and sparse matrix multiplication from scratch
• Profiling real workloads and integrating custom Triton kernels into PyTorch
• Reasoning about how GPUs actually execute your code

About the reader

For ML engineers and researchers comfortable with Python and PyTorch.

About the author

Harshwardhan Fartale is a researcher and engineer based in Bangalore, where he builds machine learning systems for scientific and defense applications. He has delivered courses in generative AI, machine learning, and MLOps to audiences ranging from university students to national research bodies.