Miss it here,
not in the exam.
Build a skill, test your curiosity, or take on a friend. Every answer comes with the why.
Try a card · AI & LLM Engineering
In a transformer-based language model, what is the primary purpose of the softmax function applied to the attention scores (the dot products of queries and keys)?
The softmax function converts the raw attention scores (logits from query-key dot products) into a valid probability distribution where all weights are non-negative and sum to 1. This means each token's representation is a weighted combination of all value vectors, with the weights reflecting learned relevance. Without softmax, the raw scores could be arbitrarily large or negative, making the weighted sum unstable and uninterpretable. The option about eliminating negatives is partially true (softmax outputs are non-negative) but mischaracterises the purpose — softmax normalises, it does not simply clip negatives. The option about equal weights describes mean pooling, not attention. The option restricting softmax to training is wrong — softmax is applied at every forward pass, including inference.
Find your next rabbit hole
- Cloud certAWS SAA-C03218 questions →
- AccountingAccounting: GAAP & IFRS332 questions →
- TaxTax: UK/US/UAE/KSA/EU280 questions →
- AI skillsAI & LLM Engineering425 questions →
- TriviaTrivia218 questions →
- StudentsSchool184 questions →
- FinanceStock Market & Investing113 questions →
- DrivingUK Driving Theory160 questions →
No matching subjects. Try AI, tax or science.
Better with a little competition.
Pick a trivia topic, finish a round, then send a friend your challenge.
Understand it. Then test it.
All guides →Loss aversion vs disposition effect (and 5 more bias pairs investors confuse)
Loss aversion vs disposition effect, anchoring vs confirmation bias, and 4 more behavioral finance bias pairs told apart, with worked investor examples.
Sports word origins and myths, explained: hat-trick, derby, birdie & more
The real, sourced stories behind sports words like hat-trick, derby, birdie and the Ashes, plus two famous sporting "facts" that turn out to be myths.
Turn yesterday’s misses into progress
Revisit questions you missed with a fresh round of practice.