Forcing a Neural Network to Add -ed
from Inside the Black Box: Cracking AI and Deep Learning Hosted by Arshavir Blackwell, PhD
Part 3 of 4. After two episodes arguing there is no discrete past-tense rule inside a language model, this one turns the strongest possible search on the question: gradient descent, hunting for the single internal direction that best forces goed over went. It works — 100% of the time, even on held-out verbs it was never tuned on. Then one control collapses the whole result. That same direction drives ordinary regular verbs like walked to zero, and under it the model stops producing words at all, emitting only edededed. Opening up the direction shows why: it simply screams the two-letter token ed, burying every whole-word rival. The optimizer never found grammar — it found the cheapest trick that games a narrow metric, and a caution about measuring only the thing you set out to change. Read the original post: Forcing a Neural Network to Add -ed
Embed this episode
No transcript yet.