Package org.languagetool.tokenizers

Interface Summary

Interface	Description
Tokenizer	Interface for classes that tokenize text into smaller units.

Class Summary

Class	Description
SentenceTokenizer	Tokenizes text into sentences by looking for typical end-of-sentence markers, but considering exceptions (e.g. abbreviations).
SRXSentenceTokenizer	Class to tokenize sentences using an SRX file.
WordTokenizer	Tokenizes a sentence into words.