Where this topic leads
Topics that build on Information Theory Foundations
Once you have Information Theory Foundations, these are the topics that cite it as a prerequisite. Pick by tier and the area you want to push into next.
Editor's suggested next (3)
Core flagship topics (3)
- CLIP, OpenCLIP, and SigLIP: Contrastive Language-Image Pretraininglayer 4 · beyond-llms
- Cross-Entropy Loss: MLE, KL Divergence, and Classificationlayer 1 · ml-methods
- KL Divergencelayer 1 · foundations
Standard topics (5)
- Bits, Nats, Perplexity, and BPBlayer 3 · llm-construction
- Kelly Criterionlayer 2 · decision-theory
- Perplexity and Language Model Evaluationlayer 3 · llm-construction
- Representation Learning Theorylayer 3 · modern-generalization
- Token Prediction and Language Modelinglayer 3 · llm-construction
Advanced or specialty topics (2)
- Information Bottlenecklayer 3 · modern-generalization
- Tokenization and Information Theorylayer 4 · llm-construction