Lecture 3: From State Machines to Regular Expressions

  • Tokenizers as Finite state machines
  • Regular Expressions notation
  • Tokenizer spec
  • Generic getNextToken
  • Single-line and Multi-line comments