Raw text scanner class returning mootToken; wraps wasteScanner.
Public Member Functions | |
Constructors etc. | |
| wasteTokenScanner (int fmt=tiofText, const std::string &name="wasteTokenScanner") | |
| virtual | ~wasteTokenScanner (void) |
Input Selection | |
| virtual void | from_mstream (mootio::mistream *mistreamp) |
| virtual void | close (void) |
Token-Level Access | |
| virtual mootTokenType | get_token (void) |
| virtual mootTokenType | get_sentence (void) |
Diagnostics | |
| virtual size_t | line_number (void) |
| virtual size_t | line_number (size_t n) |
| virtual size_t | column_number (void) |
| virtual size_t | column_number (size_t n) |
| virtual mootio::ByteOffset | byte_number (void) |
| virtual mootio::ByteOffset | byte_number (size_t n) |
 Public Member Functions inherited from moot::TokenReader | |
| TokenReader (int fmt=tiofUnknown, const std::string &name="TokenReader") | |
| virtual | ~TokenReader (void) |
| void | tr_clear (void) |
| virtual void | from_mstream (mootio::mistream &mis) |
| virtual void | from_filename (const char *filename) |
| virtual void | from_file (FILE *file) |
| virtual void | from_fd (int fd) |
| virtual void | from_buffer (const void *buf, size_t len) |
| virtual void | from_string (const char *s) |
| virtual void | from_cxxstream (std::istream &is) |
| virtual bool | opened (void) |
| mootToken * | token (void) |
| mootSentence * | sentence (void) |
| virtual void | reader_name (const std::string &myname) |
| virtual void | carp (const char *fmt,...) |
Public Attributes | |
| wasteScanner | scanner |
| mootToken | wts_token |
| mootSentence | wts_sentence |
| Public Attributes inherited from moot::TokenReader | |
| int | tr_format |
| std::string | tr_name |
| mootio::mistream * | tr_istream |
| bool | tr_istream_created |
| mootToken * | tr_token |
| mootSentence * | tr_sentence |
| void * | tr_data |
Additional Inherited Members | |
| Static Public Member Functions inherited from moot::TokenIO | |
| static int | parse_format_string (const std::string &fmtString) |
| static int | guess_filename_format (const char *filename) |
| static bool | is_empty_format (int fmt) |
| static int | sanitize_format (int fmt, int fmt_implied=tiofNone, int fmt_default=tiofNone) |
| static int | parse_format_request (const char *request, const char *filename=__null, int fmt_implied=tiofNone, int fmt_default=tiofNone) |
| static std::string | format_canonical_string (int fmt) |
| static class TokenReader * | new_reader (int fmt) |
| static class TokenWriter * | new_writer (int fmt) |
| static class TokenReader * | file_reader (const char *filename, const char *fmt_request=__null, int fmt_implied=tiofNone, int fmt_default=tiofNone) |
| static class TokenWriter * | file_writer (const char *filename, const char *fmt_request=__null, int fmt_implied=tiofNone, int fmt_default=tiofNone) |
| static size_t | pipe_tokens (class TokenReader *reader, class TokenWriter *writer) |
| static size_t | pipe_sentences (class TokenReader *reader, class TokenWriter *writer) |
| Static Public Attributes inherited from moot::TokenReader | |
| static const size_t | TR_DEFAULT_BUFSIZE = 256 |
Constructor & Destructor Documentation
◆ wasteTokenScanner()
| moot::wasteTokenScanner::wasteTokenScanner | ( | int | fmt = tiofText, |
| const std::string & | name = "wasteTokenScanner" |
||
| ) |
Default constructor
- Parameters
-
fmt bitmask of moot::TokenIOFormat flags. - set moot::tiofTagged to assign token tags to wasteScannerTypeNames[]
- set moot::tiofAnalyzed to assign single analysis-strings to wasteScannerTypeNames[]
name name of current input source
◆ ~wasteTokenScanner()
|
virtual |
Default destructor
Member Function Documentation
◆ from_mstream()
|
virtual |
Select input from a mootio::mistream pointer.
Reimplemented from moot::TokenReader.
◆ close()
|
virtual |
Finish input from currently selected source & perform any required cleanup operations.
Reimplemented from moot::TokenReader.
◆ get_token()
|
virtual |
Get the next token from the buffer. On completion, current token (if any) is in *tr_token.
Reimplemented from moot::TokenReader.
◆ get_sentence()
|
virtual |
Buffer the remaining tokens from the currently selected input stream as a mootSentence. On completion, buffered tokens (if any) are in *tr_sentence.
Reimplemented from moot::TokenReader.
◆ line_number() [1/2]
|
inlinevirtual |
Get current line number. Descendants may override this method.
Reimplemented from moot::TokenReader.
References moot::GenericLexer::theLine.
◆ line_number() [2/2]
|
inlinevirtual |
Set current line number. Descendants may override this method.
Reimplemented from moot::TokenReader.
References moot::GenericLexer::theLine.
◆ column_number() [1/2]
|
inlinevirtual |
Get current column number. Descendants may override this method.
Reimplemented from moot::TokenReader.
References moot::GenericLexer::theColumn.
◆ column_number() [2/2]
|
inlinevirtual |
Set current column number. Descendants may override this method.
Reimplemented from moot::TokenReader.
References moot::GenericLexer::theColumn.
◆ byte_number() [1/2]
|
inlinevirtual |
Get current byte number. Descendants may override this method.
Reimplemented from moot::TokenReader.
References moot::GenericLexer::theByte.
◆ byte_number() [2/2]
|
inlinevirtual |
Get current byte number. Descendants may override this method.
Reimplemented from moot::TokenReader.
References moot::GenericLexer::theByte.
Member Data Documentation
◆ scanner
| wasteScanner moot::wasteTokenScanner::scanner |
underlying scanner object
◆ wts_token
| mootToken moot::wasteTokenScanner::wts_token |
token buffer for get_token()
◆ wts_sentence
| mootSentence moot::wasteTokenScanner::wts_sentence |
sentence/document buffer for get_sentence()
The documentation for this class was generated from the following file:
