Phrase Bias Tutorial

What is Biasing?
NovelAI at its heart is an advanced text prediction system that constructs responses one token at a time. The closest human language analogue for a token would be a syllable, although it does not usually map or correspond 1:1; there are words with multiple syllables that are a single token.

For each token, NovelAI picks from a list of probable tokens -- the token is selected from a list that is formed up influenced by:
 * The Sigurd (GPT) model itself.
 * What text already exists in the context sent, including Story, activated Lorebook entries, Author's Note, and Memory.
 * The module in use (such as Cross-Genre, 19th Century Romance, and Lovecraft)
 * Generation settings.
 * Token bans.
 * ... and now Phrase Biases.

Each 'token has a probability assigned to it by the above process. For example, if we do:, the following top token probabilities are calculated: From the above table, we can see that the two most probable tokens are  and , and that the following two probable tokens are   and. The generation node then picks one of them randomly accordingly to the weights above.

When we generate for a number of tokens such as, the above process is repeated   times. With Token Streaming enabled, these tokens are fed back to the NovelAI UI one at a time.

What if we could adjust the probabilities of these tokens? Well, with Phrase Biasing, we can!