Logits are the raw numerical scores that language models compute for each token in their vocabulary before making a selection. These scores are converted to probabilities and used internally for token selection. Accessing and analyzing them directly opens up possibilities for controlling and understanding model behavior.

We'll cover common sampling techniques like temperature adjustment, top-k, and top-p filtering, and beam search.

Then we will see how logits can be used to evaluate model uncertainty, causing hallucinations.

And we will talk about structured generation to use language models in deterministic projects. We will see how the logit values can be used to guide the generation process. Lastly we will explore the libraries like outlines and guidance by showcasing some example snippets about how to use them.

If "token by token" is your only answer when someone asks how LLMs generate text, come join us and let's dig deeper together!