AI Learning Dashboard
From classical search to the building blocks of GPT. Every demo runs live in your browser and starts itself — pick a topic, watch, replay. The language-model modules come from my graduate coursework in Auburn's COMP 6970: Generative AI & Foundation Models.
A language model is one loop: predict the next token, sample, repeat. This one learned character statistics from Tiny Shakespeare and writes live — with the real probability distribution on screen and the same temperature knob the ChatGPT API exposes.
What's happening
A language model is just “predict the next token, repeat.” This one learned character statistics from Tiny Shakespeare and now writes new text by sampling from its predicted distribution — watch the probability bars shift with every keystroke it types.
Temperature
samples from the true learned distribution — switch mid-generation and watch the writing style change.
First
Waiting for the first prediction…
The highlighted row is the character the model actually sampled. At low temperature it almost always takes the top bar; at high temperature the long tail gets real chances — same knob as in the ChatGPT API.
Next-Token Prediction
The entire objective is Σ log p(next | context) — classification over the vocabulary, repeated. GPT differs from this demo by parameters and architecture, not by objective: my 5.24M-parameter from-scratch GPT trained on exactly this loss.
Temperature
Divide the logits by T before softmax. Low T sharpens the distribution toward safe, repetitive text; high T flattens it toward creative chaos. Flip between Focused and Wild mid-generation and watch the prose change character.
Greedy, Top-k, Top-p
Always taking the argmax degenerates into loops, so real systems sample: top-k keeps a fixed number of candidates, while top-p keeps the smallest set covering probability p — adapting to how confident the model is.
Perplexity
The standard LM score: exp(cross-entropy), read as "how many options is the model effectively choosing between?" Guessing digits at random scores 10. My course GPT reached 183 on Penn Treebank — then jumped to 441 after fine-tuning, a lesson in catastrophic forgetting.