Skip to main content

Modern PyTorch Guide home page

Official Docs
GitHub
GitHub

Building Models

Community
Forums

Architecture Patterns

Multi-Layer Perceptrons
Convolutional Neural Networks
RNNs & LSTMs
Gru
Transformers
Attention mechanisms
Autoencoders
Vae
Diffusion Models
Gans
Spatial transformer

Loss Functions

Regression Losses
Classification Losses
Contrastive Losses
Custom Loss Functions
Adversarial

Optimizers

Optimizers Overview
SGD & Momentum
Adam & Variants
Learning Rate Schedulers
Compiled optimizer
Zero redundancy

Training Loops

Basic Training Loop
Validation & Metrics
Checkpointing
Early Stopping
Mixed Precision Training
Gradient accumulation
Grad clipping

Debugging & Visualization

Common Errors
Gradient Issues
Profiling
TensorBoard Integration
Anomaly detection
Gradcheck
Visualizing gradients

Advanced Techniques

Parametrizations
Pruning
Knowledge distillation
Model ensembling
Per sample gradients

Parallel computing

Overview

Device Management

Cpu
Cuda basics
Mps
Xpu
Streams events

Learning Rate Schedulers

Optimizers

Learning Rate Schedulers

Dynamic learning rate adjustments

Learning Rate Schedulers

StepLR, CosineAnnealingLR, OneCycleLR, ReduceLROnPlateau.

Adam & Variants Compiled optimizer

⌘I

x github linkedin