Skip to main content

Modern PyTorch Guide home page

Official Docs
GitHub
GitHub

Building Models

Community
Forums

Architecture Patterns

Multi-Layer Perceptrons
Convolutional Neural Networks
RNNs & LSTMs
Gru
Transformers
Attention mechanisms
Autoencoders
Vae
Diffusion Models
Gans
Spatial transformer

Loss Functions

Regression Losses
Classification Losses
Contrastive Losses
Custom Loss Functions
Adversarial

Optimizers

Optimizers Overview
SGD & Momentum
Adam & Variants
Learning Rate Schedulers
Compiled optimizer
Zero redundancy

Training Loops

Basic Training Loop
Validation & Metrics
Checkpointing
Early Stopping
Mixed Precision Training
Gradient accumulation
Grad clipping

Debugging & Visualization

Common Errors
Gradient Issues
Profiling
TensorBoard Integration
Anomaly detection
Gradcheck
Visualizing gradients

Advanced Techniques

Parametrizations
Pruning
Knowledge distillation
Model ensembling
Per sample gradients

Parallel computing

Overview

Device Management

Cpu
Cuda basics
Mps
Xpu
Streams events

Transformers

Architecture Patterns

Transformers

Attention is all you need - the dominant architecture

Transformers

Self-attention, multi-head attention, positional encoding, encoder-decoder architecture.

Gru Attention mechanisms

⌘I

x github linkedin