THE SAME MACHINERY AS CHATGPT, ABOUT ONE HUNDRED-THOUSANDTH THE SIZE
IT LEARNED WHAT A MATCH REPORT LOOKS LIKE · AND NOTHING ABOUT SOCCER
Three versions of one small model, trained from scratch on 15,540 real international matches. They write fluent, perfectly formatted reports and confident answers — with invented scorelines, players who never existed and cities in the wrong country. Ask who won Brazil against Germany in 2014 and all three get it wrong. Large models are this same machine, scaled until the guesses are usually right. This one is small enough to watch it guess.
Low repeats the corpus. High invents countries.
FINE-TUNING CHANGES BEHAVIOUR, NOT KNOWLEDGE
ONE BASE MODEL PRETRAINED FROM RANDOM INITIALISATION ON 2.5 MB OF INTERNATIONAL MATCH RECORDS 1872–2026 · 15,540 FIXTURES WITH GOALSCORERS · CHARACTER-LEVEL TOKENISER, 197 TOKENS, NO QUESTION MARK AMONG THEM · 4,901,061 PARAMETERS, 6 LAYERS, 8 HEADS, 256 CHARACTER CONTEXT · TWO COPIES THEN FINE-TUNED FROM THOSE SAME WEIGHTS ON 42,000 GENERATED QUESTION AND ANSWER PAIRS · ONE ON THE PAIRS ALONE, ONE ON THE PAIRS MIXED WITH THE ORIGINAL RECORDS · IDENTICAL STEPS, IDENTICAL LEARNING RATE, ONLY THE DATA MIX DIFFERS · THE PAIRS-ALONE VERSION LEARNED TO ANSWER AND LOST THE ABILITY TO WRITE A MATCH REPORT, WHICH IS CATASTROPHIC FORGETTING · THE MIXED VERSION KEPT BOTH · NEITHER BECAME MORE ACCURATE: ASK EITHER WHO WON BRAZIL AGAINST GERMANY IN 2014 AND BOTH ANSWER CONFIDENTLY AND WRONGLY · TWELVE MINUTES OF FINE-TUNING TAUGHT THESE MODELS TO ANSWER AND NOTHING TRUE · QUANTISED TO FLOAT16 AND RUN IN PLAIN JAVASCRIPT WITH A KV CACHE · NO SERVER, NO API, NOTHING LEAVES THIS PAGE
MODELS AND PAGE BY ADEE1T · MIT LICENCE