wasteLexer.h File Reference

Declarations of the mid level lexer for the waste tokenizer. More...

Go to the source code of this file.

Classes
class	mootTokenLexer
	flex++ lexer for moot PoS tagger native text input (guts for moot::TokenReaderNative) More...
class	moot::wasteLexerToken
class	moot::wasteLexer
	Mid-level scanner stage performs (optional) hyphenation normalization and text classification. More...
class	moot::wasteLexerReader
	Mid-level scanner stage, wraps moot::wasteLexer in moot::TokenReader API. More...

Namespaces
	moot

Typedefs
typedef struct yy_buffer_state *	YY_BUFFER_STATE

Enumerations
enum	moot::wasteLexer_state {   moot::ls_flush = 0x0001, moot::ls_hyph = 0x0002, moot::ls_head = 0x0004, moot::ls_tail = 0x0008,   moot::ls_nl = 0x0010, moot::ls_sb_fw = 0x0020, moot::ls_wb_fw = 0x0040, moot::ls_blanked = 0x0080 }

Variables
static const int	moot::ls_init = (ls_wb_fw | ls_sb_fw | ls_blanked)
static const int	moot::ls_head_hyph = ( ls_head | ls_hyph )
static const int	moot::ls_head_hyph_nl = ( ls_head_hyph | ls_nl )

Detailed Description

Author

kmw

Date

2013

Typedef Documentation

YY_BUFFER_STATE

typedef struct yy_buffer_state* YY_BUFFER_STATE