Prerequisite chain
Prerequisites for BERT and the Pretrain-Finetune Paradigm
Topics you need before working through BERT and the Pretrain-Finetune Paradigm. Direct prerequisites are listed first; transitive prerequisites (the chain reachable through them) follow.
Direct prerequisites (2)
- Transformer Architecturelayer 4, tier 2
- Token Prediction and Language Modelinglayer 3, tier 2
Reachable through the chain (18)
These topics are not directly cited as prerequisites but are reached transitively by following the chain upward. Working through the direct prerequisites pulls these in.
- Attention Mechanism Theorylayer 4, tier 2
- Matrix Operations and Propertieslayer 0A, tier 1
- Sets, Functions, and Relationslayer 0A, tier 1
- Basic Logic and Proof Techniqueslayer 0A, tier 2
- Softmax and Numerical Stabilitylayer 1, tier 1
- Feedforward Networks and Backpropagationlayer 2, tier 1
- Differentiation in Rnlayer 0A, tier 1
- Vectors, Matrices, and Linear Mapslayer 0A, tier 1
- Continuity in Rⁿlayer 0A, tier 1
- Metric Spaces, Convergence, and Completenesslayer 0A, tier 1
- Matrix Calculuslayer 1, tier 1
- The Jacobian Matrixlayer 0A, tier 1
- The Hessian Matrixlayer 0A, tier 1
- Eigenvalues and Eigenvectorslayer 0A, tier 1
- Activation Functionslayer 1, tier 1
- Convex Optimization Basicslayer 1, tier 1
- Information Theory Foundationslayer 0B, tier 2
- Common Probability Distributionslayer 0A, tier 1