- isAtBracket
Determines if the underlying input is looking at a bracket. By default all
supplied brackets are che
- isAtStartOfIdentifier
Determines if the underlying input is looking at a valid character to start an
identifier By default
- isIdentifierChar
Determines if the given Char is a valid identifier part. By default, letters,
digits and '_' are val
- isSymbolCharacter
Determines if the given Char is a symbol character. By default these are all
non-control characters,
- <init>
Creates a new tokenizer for the given input
- addError
Adds a parse error to the internal problem collector. It is preferred to collect
as much errors as p
- addStringDelimiter
Adds a new string delimiter character along with the character used to escape
string within it.
- canConsumeThisString
Checks if the next characters, starting from the current, match the given
string.
- consume
- current
- fetch
- fetchId
Reads and returns an identifier