Figuring out whether a character should become part of a number or a name, for instance, is typical lexer stuff though. For that you have to classify it.