what is a token in ai

A clear explanation of tokenization — how AI language models break text into pieces — including why it affects cost, speed, and model behavior.