Inheritance diagram for GncTokenizer:
Public Member Functions | |
| GncTokenizer (const GncTokenizer &)=default | |
| GncTokenizer & | operator= (const GncTokenizer &)=default |
| GncTokenizer (GncTokenizer &&)=default | |
| GncTokenizer & | operator= (GncTokenizer &&)=default |
| virtual void | load_file (const std::string &path) |
| const std::string & | current_file () |
| void | encoding (const std::string &encoding) |
| const std::string & | encoding () |
| virtual int | tokenize ()=0 |
| const std::vector< StrVec > & | get_tokens () |
Protected Attributes | |
| std::string | m_utf8_contents |
| std::vector< StrVec > | m_tokenized_contents |
Detailed Description
Definition at line 57 of file gnc-tokenizer.hpp.
The documentation for this class was generated from the following files:
