Tokenizer class that breaks text into tokens. More...
#include <style_tokenizer.h>
Public Member Functions | |
StyleTokenizer (const std::string &text) | |
Construct tokenizer for a text string. More... | |
void | read (StyleToken &out_token, bool eat_whitespace, bool eat_comments=true) |
Reads the next token. More... | |
std::vector< StyleToken > | read_property_value (StyleToken &token, bool &out_important_flag) |
Reads all tokens belonging to a property value. More... | |
Static Public Member Functions | |
static std::vector< StyleToken > | tokenize (const std::string &text) |
Returns all tokens except start/end whitespace and comments. More... | |
Tokenizer class that breaks text into tokens.
uicore::StyleTokenizer::StyleTokenizer | ( | const std::string & | text | ) |
Construct tokenizer for a text string.
void uicore::StyleTokenizer::read | ( | StyleToken & | out_token, |
bool | eat_whitespace, | ||
bool | eat_comments = true |
||
) |
Reads the next token.
std::vector<StyleToken> uicore::StyleTokenizer::read_property_value | ( | StyleToken & | token, |
bool & | out_important_flag | ||
) |
Reads all tokens belonging to a property value.
|
static |
Returns all tokens except start/end whitespace and comments.