All the articles with the tag "tokenization".
Before the transformer can do anything, it must turn your prompt into numbers. Here is exactly how that works, from raw characters to dense vectors.
