Main Page | Directories | Namespace List | Class Hierarchy | Alphabetical List | Class List | File List | Namespace Members | Class Members | File Members

mootToken.h File Reference

Go to the source code of this file.

Namespaces

Classes

Typedefs

Enumerations

Functions

Variables


Typedef Documentation

typedef list<mootToken> moot::mootSentence
 

Sentences are just lists of mootToken objects

typedef set<mootTagString> moot::mootTagSet
 

Tagset (read "lexical class") type

typedef string moot::mootTagString
 

Tag-string type

typedef mootTokenTypeE moot::mootTokenType
 

typedef string moot::mootTokString
 

Token-string type


Enumeration Type Documentation

enum mootTokenFlavor
 

Enum for TnT-style token typification

Enumeration values:
TokFlavorAlpha  (Mostly) alphabetic token: "foo", "bar", "foo2bar"
TokFlavorCard  : Digits-only: "42"
TokFlavorCardPunct  : Digits single-char punctuation suffix: "42."
TokFlavorCardSuffix  : Digits with (almost any) suffix: "42nd"
TokFlavorCardSeps  : Digits with interpunctuation: "420.24/7"
TokFlavorUnknown  : Special "Unknown" token-type
NTokFlavors  Not really a token-type

enum mootTokenTypeE
 

Enumeration values:
TokTypeUnknown  we dunno what it is -- could be anything
TokTypeVanilla  plain "vanilla" token (+/-besttag,+/-analyses)
TokTypeLibXML  plain XML token; much like 'Vanilla'
TokTypeXMLRaw  Raw XML text (for lossless XML I/O)
TokTypeComment  a comment, should be ignored by processing routines
TokTypeEOS  end-of-sentence
TokTypeEOF  end-of-file
TokTypeUser  user-defined token type: use in conjunction with 'user_data'
NTokTypes  number of token-types (not a type itself)


Function Documentation

bool isTokFlavorName const mootTokString tokstr  )  [inline]
 

Returns true iff is a pseudo-identifier for a non-alpha type Used during HMM and trie compilation

mootTokenFlavor tokenFlavor const mootTokString token  )  [inline]
 

Get the TokenType for a given token

bool tokenFlavor_isCardPunctChar const char  c  )  [inline]
 

TnT compatibility hack

bool tokenFlavor_isCardSuffixChar const char  c  )  [inline]
 

TnT compatibility hack


Variable Documentation

const char* moot::mootTokenFlavorNames[NTokFlavors]
 

Convert token-types to symbolic names

const char* moot::mootTokenTypeNames[NTokTypes]
 

Useful for debugging token types


Generated on Mon Jun 27 13:05:25 2005 for libmoot by  doxygen 1.3.8-20040913