tokenize

#concept

tokenize >> convert a string to a sequence of integers

tokenization is similar to this mathematical technique presented in the 1930s: ____ >> Godel numbering

References §

1. Myself

Notes §