Skip to main content

Modern PyTorch Guide home page

Official Docs
GitHub
GitHub

Building Models

Community
Forums

Fine-tuning & Adaptation

Transfer Learning
PEFT & LoRA
QLoRA
RLHF
Warmstarting

Custom Operators

Custom Python Operators
C++ Extensions
CUDA Kernels
Triton Kernels
Dispatching
Backend integration
Double backward

Advanced Parallelism

Megatron-LM
DeepSpeed ZeRO
Expert Parallelism
3D Parallelism
Device mesh
Symmetric memory

Research Tools

torch.func
torch.fx
functorch
FlashAttention
Vmap
Jacobian hessian

Extending PyTorch

Overview
Autograd functions
Cpp frontend
Custom backends
Privateuse1

Sparse & Structured Tensors

Sparse tensors
Nested tensors
Masked tensors
Operations

Advanced Features

Complex numbers
Foreach map
Packaging
Hub
Autoload extensions
Torch modes

Low-Level & Internals

Fake tensors
Aten ir
Provenance tracking
Minifier
Logging

C++ Extensions

Custom Operators

C++ Extensions

High-performance custom operators

Documentation Index
Fetch the complete documentation index at: https://newtorch.aboneda.com/llms.txt
Use this file to discover all available pages before exploring further.

C++ Extensions

torch.utils.cpp_extension, pybind11, ATen/C++ API.

Custom Python Operators CUDA Kernels

⌘I

x github linkedin

Powered byThis documentation is built and hosted on Mintlify, a developer documentation platform