org.languagetool.tokenizers.ro

Class RomanianWordTokenizer

Object

RomanianWordTokenizer

All Implemented Interfaces:

Tokenizer

public class RomanianWordTokenizer
extends Object
implements Tokenizer

Tokenizes a sentence into words. Punctuation and whitespace gets its own token. Like EnglishWordTokenizer except for some characters: eg: "-'

Since:

20.02.2009 19:53:50

Author:

Ionuț Păduraru

Constructor Summary

Constructors

Constructor and Description
RomanianWordTokenizer()

Method Summary

Methods

Modifier and Type	Method and Description
List<String>	tokenize(String text)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RomanianWordTokenizer

public RomanianWordTokenizer()

Method Detail

tokenize

public List<String> tokenize(String text)

Specified by:

tokenize in interface Tokenizer