Let’s find the birthday cake.
Start with a training run
Team & speed
Each round makes one averaged learning update.
At the beginning, every door starts with 100 weight points. The explorers haven’t learned which paths lead to the birthday cake yet.
Nobody knows the route yet. Send your first explorers.
Learning curve
Hover or tap for a recorded sample. Focus the chart: ← / → inspect points, ↑ / ↓ select runs, and Escape dismisses.
The curve starts before training. Older points may be thinned to keep it compact.
About this experiment: training and inference
TRY. LEARN. REMEMBER.
The explorers explore.
The doors remember.
Send explorers through the maze. Watch their discoveries become knowledge that the next explorer can use.
Training is a bit like sending a bunch of explorers through a whole lot of doors, over and over and over. Their choices are random, but the learned weights increasingly guide where they go. After each round, successes and failures update the odds for the next explorers. This maze is a simplified learning analogy, not a literal brain or full neural network.
Train for a while, then switch to Inference. One explorer uses the learned parameters without changing them. The door’s weight points summarize its visit probability. It takes less work than sending a team and updating what they learn.
Practice changes the doors,
not the explorers.
Training tries many routes and uses results to adjust the odds. Inference sends one explorer through the same learned maze, without changing anything it knows.
One attempt isn’t guaranteed to work. Better odds are not certainty.
Reset clears the current run and keeps previous comparisons. New maze clears comparisons, chooses the goal again and keeps every door connected to all three doors in the next row.
Experiment settings
A small glossary
- Training
- Training changes a model’s parameters using examples or feedback. It is not hand-coding a separate response for every question. Training large frontier models can require enormous computing resources; small models can be trained with far less.
- Neural network
- A model of connected computational units whose adjustable weights are learned during training. This door maze is a simplified learning analogy, not a literal brain or full neural network.
- Inference
- Inference is what happens when you ask ChatGPT a question: the model uses what it learned to generate a response. Serving inference for many users also requires substantial computing infrastructure, including data centers.
- Weights
- In neural networks, weights are learned connection strengths, commonly stored in arrays alongside biases. Our nine door numbers are a simplified summary of visit likelihood. The actual toy model stores 21 logits that become conditional choice probabilities. Door colors compare the chance of reaching a door with the starting one-third. The small bar’s tick marks that starting chance. Red means less often reached, not impossible or wrong; all A and B doors can still lead to the cake.
- Probability
- A chance, not a promise. A 70% chance means about 70 out of 100 independent journeys would succeed on average. It does not guarantee 70 successes in any particular hundred.
- Learning curve
- A record of success probability after completed learning updates. It can improve or dip. Journeys count training experience used; rounds count averaged updates. Neither axis measures electricity or wall-clock compute. Old samples may be thinned, but are never replaced with invented measurements.
- Model and parameters
- The retained map: the numbers used to make future choices. Here that means 3 START logits, a 3×3 A→B array, and a 3×3 B→C array. Inference uses them without needing the history of every training journey.
More explorers can stand for more training experience or parallel work, and more doors can suggest a larger model. This is an illustration, not a literal neural-network architecture or a scaling law.
Inference takes electricity. So does scrolling Instagram. The useful question is what we get from that energy.
Where the analogy breaks down
An analogy is useful because it leaves things out. The important part is knowing what.
The doors show one important idea: experience changes numbers, and those numbers influence future choices. But this demo is a tiny reinforcement-learning system, not a literal neural network. It learns one task in one maze. Real models learn from varied inputs and must respond to new ones. The maze’s door states, choices and outcomes are data too.
Imagine the forest explorers carrying notebooks. They record terrain, tree species, animal sightings and the routes they take. Those notebooks are the dataset. The marks that guide future explorers represent what was learned.
Finding the lake teaches something about navigation. Learning about bears or tree species requires those observations too. A successful route alone doesn’t contain a complete description of the forest.
Read the original sources
Many neural-network units contribute simultaneously; inputs do not literally take a single doorway per layer. Our visible door points summarize visitation probabilities, not raw neural-network weights. Deep Learning, Chapter 6: Deep Feedforward Networks (textbook chapter) explains how connected units build representations.
For a language model, the observations are often text. During pretraining, GPT-style models learn patterns by predicting the next token from its context. At inference time, your question supplies new context, and the learned parameters help generate a response. The research paper Improving Language Understanding by Generative Pre-Training describes this pretraining approach.
That ability to apply learning to new inputs is what our fixed maze mostly leaves out.
For the policy-learning idea used in this demo, see OpenAI’s Spinning Up: Intro to Policy Optimization (technical guide).
Watch next
- 3Blue1Brown: But what is a neural network?
See how many units and connections contribute, and what learned weights represent. - 3Blue1Brown: Backpropagation, intuitively
Understand credit assignment: backpropagation computes gradients, and an optimizer uses them to update parameters. - 3Blue1Brown: Large Language Models explained briefly
A short, nontechnical overview of tokens, context, training and generating responses.
What’s really happening? A closer look for curious minds.
Read the door summary
What if we start through A1?
A door’s visit likelihood is different from the chance of finding the birthday cake after entering that door. Open the A1 example to compare them.
A learning analogy, not a literal brain
This is a small reinforcement-learning system. The doors hold trainable numbers called logits. A softmax turns those numbers into selection probabilities. Real neural networks usually activate many neurons at once and store weights and biases, not door-choice odds.
Feedback and credit assignment
Finding the birthday cake earns a reward of 1; missing it earns 0. After a whole round, a policy-gradient update compares each result with a moving reward baseline. A failed route does not prove every door on it was wrong. The backward animation illustrates credit assignment; it is not conventional neural-network backpropagation.
Training versus inference
Training repeats attempts and changes parameters. Inference samples from those same parameters but never updates them. More training often improves the odds, not necessarily every round. This toy counts attempts, not real GPU energy or neural-network compute costs. The inference suggestion appears at an 80% exact chance of reaching the goal. This is a guide for this demo, not proof that learning is complete.
Explore, then exploit
Probabilities keep several routes possible, letting explorers discover useful choices. As rewards accumulate, useful routes become more likely. Some early choices are equally useful and need not become certain.