Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.

Skip to main content

Modern PyTorch Guide home page

Official Docs
GitHub
GitHub

Building Models

Community
Forums

Fine-tuning & Adaptation

Transfer Learning
PEFT & LoRA
QLoRA
RLHF
Warmstarting

Custom Operators

Custom Python Operators
C++ Extensions
CUDA Kernels
Triton Kernels
Dispatching
Backend integration
Double backward

Advanced Parallelism

Megatron-LM
DeepSpeed ZeRO
Expert Parallelism
3D Parallelism
Device mesh
Symmetric memory

Research Tools

torch.func
torch.fx
functorch
FlashAttention
Vmap
Jacobian hessian

Extending PyTorch

Overview
Autograd functions
Cpp frontend
Custom backends
Privateuse1

Sparse & Structured Tensors

Sparse tensors
Nested tensors
Masked tensors
Operations

Advanced Features

Complex numbers
Foreach map
Packaging
Hub
Autoload extensions
Torch modes

Low-Level & Internals

Fake tensors
Aten ir
Provenance tracking
Minifier
Logging

Custom Python Operators

Custom Operators

Custom Python Operators

Extend PyTorch with Python functions

Custom Python Operators

torch.library, custom autograd functions in Python.

x github linkedin

Powered byThis documentation is built and hosted on Mintlify, a developer documentation platform