The result of a line tokenization.
The tokenization end state. A pointer will be held to this and the object should not be modified by the tokenizer after the pointer is returned.
The list of tokens on the line.
The result of a line tokenization.