NLTK单词语料库没有短语“okay”，“ok”，“Okay”？
> from nltk.corpus import words
> words.words().__contains__("check")
> True
> words.words().__contains__("okay")
> False
> len(words.words())
> 236736
有什么想法吗？
TL;DR
from nltk.corpus import words
from nltk.corpus import wordnet
manywords = words.words() + wordnet.words()
In Long
来自docs https://github.com/nltk/nltk_data/blob/gh-pages/packages/corpora/words.xml, the nltk.corpus.words单词是来自“的单词列表”http://en.wikipedia.org/wiki/Words_(Unix) http://en.wikipedia.org/wiki/Words_(Unix)
在 Unix 中，你可以这样做：
ls /usr/share/dict/
并阅读自述文件：
$ cd /usr/share/dict/
/usr/share/dict$ cat README
# @(#)README 8.1 (Berkeley) 6/5/93
# $FreeBSD$
WEB ---- (introduction provided by jaw@riacs) -------------------------
Welcome to web2 (Webster's Second International) all 234,936 words worth.
The 1934 copyright has lapsed, according to the supplier. The
supplemental 'web2a' list contains hyphenated terms as well as assorted
noun and adverbial phrases. The wordlist makes a dandy 'grep' victim.
-- James A. Woods {ihnp4,hplabs}!ames!jaw (or jaw@riacs)
Country names are stored in the file /usr/share/misc/iso3166.
FreeBSD Maintenance Notes ---------------------------------------------
Note that FreeBSD is not maintaining a historical document, we're
maintaining a list of current [American] English spellings.
A few words have been removed because their spellings have depreciated.
This list of words includes:
corelation (and its derivatives) "correlation" is the preferred spelling
freen typographical error in original file
freend archaic spelling no longer in use;
masks common typo in modern text
--
A list of technical terms has been added in the file 'freebsd'. This
word list contains FreeBSD/Unix lexicon that is used by the system
documentation. It makes a great ispell(1) personal dictionary to
supplement the standard English language dictionary.
因为这是一个固定的列表234,936，肯定有该列表中不存在的单词。
如果您需要扩展单词列表，您可以使用 WordNet 中的单词添加到列表中nltk.corpus.wordnet.words().
最有可能的是，您所需要的只是足够大的文本语料库，例如维基百科转储然后对其进行标记并提取所有独特的单词。
本文内容由网友自发贡献，版权归原作者所有，本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容，请联系:hwhale#tublm.com(使用前将#替换为@)
