Год выпуска: 2014 Автор: Anis Zaman Издательство: LAP Lambert Academic Publishing Страниц: 72 ISBN: 9783659616389
Описание
This book presents various algorithms to compute semantic similarities between english texts. I explored three different algorithms for computing English sentence similarity. The first algorithm, which is well-explored in the literature [Salton and Buckley, 1988, Wu and Salton, 1981], weights words in each sentence according to term frequency and inverse document frequency (tf-idf ) and uses no semantic information. The second algorithm uses measures of the semantic distance between words belonging to the same part of speech. The third algorithm combines the tf-idf scores and the semantic distance scores between words. I evaluated the performance of the second and third algorithms on two data sets: O’Shea’s set of sentence pairs with human similarity judgements [Li et al., Aug, Rubenstein and Goodenough, 1965], and Microsoft Research’s sentence-level paraphrase dataset [Rus et al., 2012]. On O’Shea’s data set, the third algorithm more accurately matches human judgments than the...
