StringTokenizer Class Reference

#include <string_tokenizer.h>

List of all members.

Public Member Functions

• StringTokenizer (const char *_text, const char *_delims)
• ~StringTokenizer ()
• const char * operator() ()
• string next_token ()
• const char * get_rest ()
• char * val ()
• int count ()
• bool has_next ()
• StringTokenizer (const char *_text, const char *_delims)
• ~StringTokenizer ()
• const char * operator() ()
• string next_token ()
• const char * get_rest () const
• const char * val () const
• int count () const
• bool has_next () const
• void reinitialize (const char *_text, const char *_delims)

Private Member Functions

• bool is_delim (char ch)
• bool is_delim (char ch) const
• void initialize (const char *_text, const char *_delims)

Private Attributes

• char * text_ptr
• char * text
• int i
• string delims
• char * _val

---

Detailed Description

define TOKENIZER_USES_VECTOR_BOOL if you want to use vector<bool> in tokenizer. It must work faster with it, but it does not. The Reenterable replacement of strtok().

Usage:

char *text = "word, word2;word3" StringTokenizer token(text, " ,;"); const char *word; while((word = token())){ handle token here }

The Reenterable replacement of strtok().

Usage:

char *text = "word, word2;word3" StringTokenizer token(text, " ,;"); const char *word; while((word = token())){ handle token here }

---

Constructor & Destructor Documentation

StringTokenizer::StringTokenizer	(	const char *	_text,
		const char *	_delims
	)			[inline]

References text, and text_ptr.

StringTokenizer::~StringTokenizer

(

)

[inline]

References text_ptr.

StringTokenizer::StringTokenizer	(	const char *	_text,
		const char *	_delims
	)

StringTokenizer::~StringTokenizer

(

)

---

Member Function Documentation

bool StringTokenizer::is_delim

(

char

ch

)

[inline, private]

References delims.

Referenced by operator()().

Here is the caller graph for this function:

const char * StringTokenizer::operator()

(

)

[inline]

References _val, i, is_delim(), and text.

Referenced by next_token().

Here is the call graph for this function:

Here is the caller graph for this function:

string StringTokenizer::next_token

(

)

[inline]

References operator()().

Here is the call graph for this function:

const char* StringTokenizer::get_rest

(

)

[inline]

References text.

char* StringTokenizer::val

(

)

[inline]

References _val.

Referenced by TItemContainer::BuildDomens(), TItemContainer::BuildDomItems(), TItemContainer::BuildFields(), TItemContainer::BuildOneFieldFormat(), MorphoWizard::load_wizard(), CConcIndexator::LoadOptionsFromString(), mkdirhier(), parse_gra_descriptors(), CHomonym::ProcessLemmaAndGrammems(), CAgramtab::ProcessPOSAndGrammems(), CHighlightTags::ReadFromString(), TRoss::ReadFromStrWithOneSignatura(), and CHitBorders::RegisterBorderIndices().

Here is the caller graph for this function:

int StringTokenizer::count

(

)

[inline]

References i.

bool StringTokenizer::has_next

(

)

[inline]

References text.

bool StringTokenizer::is_delim

(

char

ch

)

const [private]

References delims.

void StringTokenizer::initialize	(	const char *	_text,
		const char *	_delims
	)			[private]

References delims, i, text, and text_ptr.

Referenced by reinitialize().

Here is the caller graph for this function:

const char* StringTokenizer::operator()

(

)

string StringTokenizer::next_token

(

)

const char * StringTokenizer::get_rest

(

)

const

References text.

const char * StringTokenizer::val

(

)

const

References _val.

int StringTokenizer::count

(

)

const

References i.

bool StringTokenizer::has_next

(

)

const

References text.

void StringTokenizer::reinitialize	(	const char *	_text,
		const char *	_delims
	)

References initialize(), and text_ptr.

Here is the call graph for this function:

---

Member Data Documentation

char * StringTokenizer::text_ptr [private]

Referenced by initialize(), reinitialize(), StringTokenizer(), and ~StringTokenizer().

char * StringTokenizer::text [private]

Referenced by get_rest(), has_next(), initialize(), operator()(), and StringTokenizer().

int StringTokenizer::i [private]

Referenced by count(), initialize(), and operator()().

string StringTokenizer::delims [private]

Referenced by initialize(), and is_delim().

char * StringTokenizer::_val [private]

Referenced by operator()(), and val().

---

The documentation for this class was generated from the following files:

• string_tokenizer.h
• util_classes.h
• utilit.cpp