Lexicon · I · The model
Training
Also: pre-training
The manufacturing phase of a model: months of computation over vast amounts of data to adjust its weights. It happens once, and it is what costs fortunes.
The analogy
Casting a bell. Weeks of preparation, one pour at very high temperature, bronze that sets. From then on, the bell rings thousands of times without the metal changing. Training is the casting; every use is just a chime. Wanting to "fix the bell" afterwards means recasting, not retouching.
A concrete example
Training a frontier model mobilizes tens of thousands of specialized chips for months, at a cost in the hundreds of millions of dollars. That is why so few players produce them. And that is why the sales line "our AI learns continuously" always deserves the same question: are you actually retraining the model, or is its harness just accumulating notes?
What it is NOT
It is not memorization: the model does not store the texts it has seen, it extracts statistical regularities from them. And it is not continuous: once deployed, the model learns nothing from its users; its "knowledge cutoff date" comes precisely from that. What looks like memory is almost always the harness at work.
Often confused with
The distinction that changes the conversation.
- Inference
- Training builds the model, once, at colossal cost; inference uses it, on every request, for pennies. Believing a model 'learns' from your conversations while you use it is confusing the two: in use, its weights no longer move. Your conversations may however be kept to feed a future training run, depending on the offering and your settings.
- Fine-tuning
- Fine-tuning is a touch-up: you start from an already-trained model and lightly adjust its weights on targeted data. Useful, but it is not remanufacturing the model: the bronze is already cast, you are chiseling the surface.