What does cosine similarity measure in word embedding spaces?

What does cosine similarity measure in word embedding spaces?

• A. The angle between two word vectors in vector space.
• B. The difference in their magnitudes.
• C. The frequency of each word in the corpus.
• D. The probability of co-occurrence within a window.

Answer: (A)

The main idea tested is how cosine similarity quantifies how aligned two word vectors are in direction, not how long they are. It uses the cosine of the angle between the two vectors: cos(theta) = (u · v) / (||u|| ||v||). By dividing by the magnitudes, this measure focuses on orientation rather than length, so two words have high similarity when their vectors point in roughly the same direction even if one vector is longer than the other. In word embeddings, semantically similar words tend to occupy nearby directions in the vector space, which is why a large cosine similarity indicates similar meaning. Conversely, if the vectors point in very different directions, the angle is large and the cosine is small or even negative, signaling less similarity. This is why it’s about angle or direction, not about the lengths, which is why the option describing the angle between vectors is the best answer.

The main idea tested is how cosine similarity quantifies how aligned two word vectors are in direction, not how long they are. It uses the cosine of the angle between the two vectors: cos(theta) = (u · v) / (||u|| ||v||). By dividing by the magnitudes, this measure focuses on orientation rather than length, so two words have high similarity when their vectors point in roughly the same direction even if one vector is longer than the other. In word embeddings, semantically similar words tend to occupy nearby directions in the vector space, which is why a large cosine similarity indicates similar meaning. Conversely, if the vectors point in very different directions, the angle is large and the cosine is small or even negative, signaling less similarity. This is why it’s about angle or direction, not about the lengths, which is why the option describing the angle between vectors is the best answer.