Skip to main content

Modern PyTorch Guide home page

Official Docs
GitHub
GitHub

Building Models

Community
Forums

Architecture Patterns

Multi-Layer Perceptrons
Convolutional Neural Networks
RNNs & LSTMs
Gru
Transformers
Attention mechanisms
Autoencoders
Vae
Diffusion Models
Gans
Spatial transformer

Loss Functions

Regression Losses
Classification Losses
Contrastive Losses
Custom Loss Functions
Adversarial

Optimizers

Optimizers Overview
SGD & Momentum
Adam & Variants
Learning Rate Schedulers
Compiled optimizer
Zero redundancy

Training Loops

Basic Training Loop
Validation & Metrics
Checkpointing
Early Stopping
Mixed Precision Training
Gradient accumulation
Grad clipping

Debugging & Visualization

Common Errors
Gradient Issues
Profiling
TensorBoard Integration
Anomaly detection
Gradcheck
Visualizing gradients

Advanced Techniques

Parametrizations
Pruning
Knowledge distillation
Model ensembling
Per sample gradients

Parallel computing

Overview

Device Management

Cpu
Cuda basics
Mps
Xpu
Streams events

Adam & Variants

Optimizers

Adam & Variants

AdamW, AdaFactor, LAMB, and modern optimizers

Documentation Index
Fetch the complete documentation index at: https://newtorch.aboneda.com/llms.txt
Use this file to discover all available pages before exploring further.

Adam & Variants

Adam, AdamW (decoupled weight decay), AdaGrad, RMSprop.

SGD & Momentum Learning Rate Schedulers

⌘I

x github linkedin

Powered byThis documentation is built and hosted on Mintlify, a developer documentation platform