Lab · Bag of words词袋练习 · 词袋
When do two headlines count as saying the same thing?两则标题什么时候算在说同一件事？
A classmate has two headlines about a training program and has to say, in one line of a tutorial note, whether they make the same claim. The headlines are “The new training program raised wages for the workers in the plant” and “The new training program lowered wages for the workers in the plant.” A bag of words counts the words and drops the order. The counts barely move when “raised” becomes “lowered,” so a high score can be written up as agreement.一个同学有两则关于培训项目的标题，要在导修笔记里用一句话说明，它们是不是在说同一件事。标题是 “The new training program raised wages for the workers in the plant” 和 “The new training program lowered wages for the workers in the plant”。词袋只数词出现几次，并把顺序丢掉。raised 换成 lowered 时，计数几乎不动，于是一个高分可以被写成“两则标题说的是一回事”。
Why it matters.为什么要紧。 The agreement score only sees the words you kept. Shared little words, a missed synonym, or one verb that flips the claim can move the score without telling you whether the two headlines agree.一致分数只看见你留下的词。一起出现的小词、没认出来的同义词，或一个把意思说反的动词，都可以把分数推动，却不能告诉你两则标题是不是在说同一件事。
Try this试一试 Start with “Opposite claims” and keep little words such as the and for. Then drop them. Switch to “Different words” and compare the agreement score.从“相反的主张”开始，留下 the、for 这类小词。再去掉它们。换到“用词不同”，比较一致分数。
A.A. The new training program raised wages for the workers in the plant.
B.B. The new training program lowered wages for the workers in the plant.
How to read the chart怎么读这张图
If you read this wrong要是读错了
If the classmate writes “the headlines agree” because the score is high, the tutorial note can tell the class the program is uncontroversial. In the first pair, raised and lowered flip whether pay went up, and the shared words keep the score high.如果同学因为分数高，就在笔记里写“两则标题一致”，导修上就会让人以为这个项目没有争议。第一对里，raised 和 lowered 把工资到底涨没涨说反了，一起出现的词仍把分数维持得很高。
What this picture is enough for这张图够说明什么
Enough to see that the agreement score is just the word counts you kept, including the and for.看得出：这个“一致”分数，就是你留下的那些词数，包括 the 和 for。
Not enough for a tutorial note that says the headlines agree. You still need the headlines that actually ran, a word list you can defend, and a check that the score follows whether pay rose, not the words the two lines happen to share.还不够拿来在导修笔记里写“两则标题一致”。你仍需要真正刊出的标题、一份说得清为什么留这些词的词表，并检查分数跟的是工资涨没涨，而不是两句碰巧共用的那些词。
The equation, and where it stops式子、假设和边界
The headlines are lowercased, and only letters are kept. Each headline becomes a list of how often each word appears. Cosine similarity is the angle between those two lists: do they point the same way, ignoring which headline is longer? For counts that are never negative, it falls between 0 and 1. If a headline has no words left, there is no cosine.标题先改成小写，只留下字母。每则标题变成每个词出现了几次。余弦相似度是这两份清单的夹角：它们是不是朝同一个方向，不管哪则标题更长。词数不会是负数时，它落在 0 和 1 之间。如果一则标题已经没有剩下的词，余弦就算不出来。
A document–term matrix is the same idea as a table: one row per headline, one column per word, and the cell is a count. This page shows two rows. It does not down-weight common words, chop words down to stems, or treat synonyms as the same word.文档—词矩阵就是这样一张表：一行一则标题，一列一个词，格子里是次数。这一页只显示两行。它不会把常见词的分量调低，不会把词切成词干，也不会把同义词当成同一个词。
Easy to get wrong.容易踩的坑。 A high cosine is easy to read as “these headlines say the same thing.” The opposite-claims pair is there to block that reading: one verb flips the claim, and the score can stay high. A low cosine is easy to read as “unrelated.” The different-words pair is there to block that reading too. Another mistake is to treat the list of little words as neutral. Choosing that list changes the score.余弦高，很容易被读成“这两则标题在说同一件事”。意思相反那一对就是为了挡住这种读法：一个动词把意思说反，分数仍可以很高。余弦低，也很容易被读成“没关系”。用词不同那一对就是为了挡住这种读法。另一个错，是把小词名单当成中性的。这份名单怎么定，分数就会怎么变。