Coffee + milk ≠ latte

Yesterday I wrote about the canonical example of how vector embeddings of words add: “king” − “man” + “woman” ≈ “queen” This should be interpreted as saying that the word vector for king, minus the word vector for man, plus the word vector for woman, is in some sense close to the word vector for queen. This post will look at another example. Is the expression “coffee” + “milk” ≈ “latte” true in some sense? Notation In this post I will use “foo” to mean the vector embedding of the word foo. Coffee + milk The cosine similarity between “coffee” + “milk” and “latte” is about 0.63. And for reasons given in the previous post, this is a large value of cosine similarity. But there are 11 words that are more similar to “milk” + “coffee” than “latte”. Here are the top 12 matches in order. coffee milk tea drink chocolate cream breakfast ice beer vanilla starbucks latte There are two questions to resolve. First, why isn’t latte one of the closest words? Second, why is the cosine similarity large…