NovelAI Features

⬅ Back to Lander

-

= Detailed Concepts =

This section seeks to explain the various phrases you'll encounter while using NovelAI.

-
 * Detailed Concepts
 * Artificial Intelligence
 * Tokens
 * Input Field
 * Story Library
 * Story Options
 * Generation Options
 * Context
 * Scenarios
 * Formats
 * Advanced Options
 * Account Settings

'GPT' Artificial Intelligence
Science fiction has us think of artificial intelligence as something for robots and alien spaceships, but the reality is very different. In this context, an artificial intelligence is quite unlike the natural intelligence exhibited by humans. The phrase that conjures thoughts of robotic assassins and intelligent holographic people is actually artificial general intelligence, or AGI.

The eponymous AI that NovelAI uses is a text transformer. This can be thought of as a very, very advanced form of autocorrect - it chooses words in sequence based on recognized patterns. This works by converting text to Tokens, running a series of pattern recognition tests to determine the probability of the next possible Token in the sequence, then returning the result.

No, you are not talking to a machine with thoughts and feelings. While NovelAI is very advanced, it is purely designed as a literature generation service. At times, you may feel like the AI understands you, and it may even seem emotional. This is it performing a very convincing facsimile of human literature - exactly what it was designed to do.

The AI is only capable of producing text that looks convincingly human, without understanding language on its own.

As such, NovelAI has no understanding of morality, as it has no understanding of anything. Please keep this in mind when experiencing the narratives presented in your Stories, and remember you can always ban certain Tokens from appearing.

Parameters
Think of an AI like a human brain. Inside the brain, you find neurons, and those neurons are connected by synapses.

The number of parameters is the number of synapses. It's a measure of the density of the neural network's connections. The denser it is, the richer the connections, and the broader the scope of the AI's creativity.

The model is, in reality, a vector space and each parameter is a set of floating point numbers that constitutes a vector. Each Vector is a connection between tokens, and that network of vectors is an attempt to represents human language mathematically.

A model with more parameters takes considerably more Memory on the machine's hardware, but is capable of greater creativity and more 'natural' language.

⬆ Return to Page Top

-

Tokens
The AI interprets text by converting it to tokens. Tokens are how the AI sees pieces of text. Much like morphemes, Tokens are combined to form words or sentences. Because the AI has no judgment of its own, it relies on evaluating the relationships between Tokens based on its training data, then determining the most likely Token to come next in the sequence.

Think of it as a huge game of probabilities. There is no winner, but there are more likely answers, and the AI picks from those.



For example, a raincoat is composed of two Tokens - the Token for rain and the Token for coat. When put together, the AI recognizes the pattern as raincoat, and evaluates its training material to figure out what Tokens are associated with raincoat. Not all Tokens are whole words - some Tokens are as simple as punctuation marks, spaces, and even partial words like mah.

Because the AI is quite powerful, it's capable of evaluating patterns with up to 2048 Tokens in total. When you hit Send, the Current Context is fed to the AI as Tokens, the AI estimates the most likely next word in the sequence, then repeats the process until the Completion is returned.

Because NovelAI works by identifying links between Tokens, including a Token will cause it to have a higher chance of appearing. This means that writing, a negative, will cause the AI to still consider   as a possibility.

This is similar to ironic process theory. Instead, you should phrase positively -  - where possible.

If you wish to experiment with Tokenization, the Author's Note and Memory fields provide an accurate Token count for everything you type within.

⬆ Return to Page Top

-

Input Field
Here is the Input Field. This section of the interface is used to control the edit history of the Story, as well as being a place to type additions to your Story.



The edit history can be thought of as a timeline - every change you make is a step forward in the timeline. Undoing will take a step backward, Redoing will take a step forward, and Retry will make a new attempt at the last Completion. The Retry Tree is basically a way to jump between timelines - any time the AI sends a Completion that's on the same point as an existing Completion, it will create a new entry in the Retry Tree.

More information can be found in their dedicated categories below.

Undo ↩
This option will take a step backwards in the edit history.
 * It will never overwrite anything in the edit history, only stepping backwards.
 * This will remove entire sentences at a time from the point you began editing, and will remove entire Completions by the AI.

Redo ↪
This option will take a step forwards in the edit history.
 * It will never overwrite anything in the edit history, only stepping forwards, and always with the most recent entry in the Retry Tree.
 * This functions inversely to Undo - instead of removing entire sentences and Completions, it will return them.
 * This is not affected by the location of your text cursor.

Retry 🔁
This option will remove the last Completion, then make a new one.
 * While doing this, it will also create a new entry in the Retry Tree.
 * This does not affect any text you edit - if there's no Completion at this point in the edit history, it's the same as hitting Send.

Retry Tree 0


This option provides a list of all the attempted Completions at this point in the Edit History.
 * You can choose the one you felt was the most appropriate.
 * This does not count as a Completion.

Send ➡
This option will place all of the text in the input field onto the end of your Story, send all of the Current Context to the AI, then ask the AI to send a Completion.

This is a new step in the edit history, and does not create a new entry in the Retry Tree.

⬆ Return to Page Top

-

Story Library


Also known as the "left panel", this part of the interface controls everything outside of your Story. This includes the Settings menu, as well as the list of your Stories.

The menu icon (Ξ) contains the Settings menu, and is also where you can log out. Below them is the Search Field (🔍), which allows you to search through your Story collection by typing next to the magnifying glass.

Every Story displays its Title, Description, the date it was last edited, and whether or not it is marked as a Favorite. The Title and Description can be modified by switching to the Story, then adjusting them in the Story Options. You can switch to a Story by simply clicking on it in the Story Library.

A new Story can be started by selecting the New Story button (+), found at the bottom of the Story Library.

⬆ Return to Page Top

-

Story Options


This section of the interface controls settings specific to your currently active Story. There are two tabs here - Story and Options.

Story Title
The name you gave to the Story, and the one you look for when using Search.

Story Description
A short blurb to explain the contents of the Story, or a witty quip to help you identify it at a glance.

Create a new tag
Adds a new tag which you can search for.

Saved Tags
All the tags that apply to this story. You can remove them by clicking in the little cross next to them. The ordering does not matter.

Memory, Author's Note and Context have their own entries in this section.

⬆ Return to Page Top

-

Generation Options


The Advanced Options have their own dedicated entry in this section.

Randomness (Temperature)
True to its name, the Randomness setting (or "Temperature") increases the likelihood of less-expected tokens during text generation. This works by dividing logits by the Temperature before sampling. In plain English, this means the next part of the sentence will be more unexpected, as elements that have less of a chance of appearing are granted a greater likelyhood of being used.

Max Output Length
This setting will adjust the highest number of Tokens returned at once by the AI in each Completion. It will not always hit this target, but it will never exceed it. Be aware that reducing the Output Length and enabling the Trim AI Responses option may cause undesirable effects such as very short sentences.

Min Output Length
Much like the previous setting, this one adjusts the lowest number of Tokens returned at once by the AI in each Completion. It will never be below this target, but may exceed it. Setting this option towards its maximum also makes the AI more likely to produce long sentences.

⬆ Return to Page Top

-

Context


View Last Context opens a window which displays all the tokens sent to the AI for the previous generation. This helps you check if anything you feel is important was omitted. View Current Context does the same, but for the input you're about to send.



⬆ Return to Page Top

-

Prompt
The prompt is displayed in cream by default. It is the first piece of text fed into the AI. If you have put anything into the Memory or Author's note, they will be inserted before it in the context before being sent to the AI.



⬆ Return to Page Top

-

Memory
By default, the Memory is inserted at the top of the context, before anything else. Its position may be adjusted for a stronger (closer to the bottom) or a weaker (further to the top) effect. Use it to make the AI remember context elements.



⬆ Return to Page Top

-

Author's Note
The Author's Note is similar to Memory, but it is inserted three newlines by default before the last token in the input. It has a greater influence as a result, but try to keep it short to prevent leaking. Leaking is an anomaly where elements of the Author's Note or Memory or their styling "leaks' into the generation, making it look out of place. The A/N's position may be adjusted for a stronger (closer to the bottom) or a weaker (further to the top) effect.



Why use Brackets?
Bracketed text is interpreted by the AI as "Read this, but this is not part of the text." This helps it keep things into memory without trying to continue from them as if they were sentences in the text.

⬆ Return to Page Top

-

Lorebook


The Lorebook allows you to create entries for specific elements in your story. This helps the AI have more information about characters, places, items, concepts and so on.

Basics
Type an entry's name then press Enter to access it, or click View All to open the main lorebook window directly.

Click ➕ Add Entry to create a new entry. You can name it in the first text box. The second contains the entry's contents. Only the content and settings are read by the AI. The entry name is just an identifier.

Enabled determines if the entry will be inserted in the context if detected. If it is disabled, it won't trigger. This is useful to reduce context cluttering if you don't need details about specific things.

Keys are all the words that the AI will associate with this entry. If the AI reads this word, then the connected entry will be inserted in the context.

Advanced Lorebook Settings


Search Range: Determines how many characters of text will be read by the AI when it looks for lorebook keys.

Force Activation: If turned on, the entry will ALWAYS be in the context (if it can fit in there).

Prefix & Suffix: These two are intended to work in tandem to allow for lengthier entries without losing coherence when the entry is trimmed. For example, you could add the prefix  and the suffix   to encapsulate the entirety of your entry despite trimming. If your entry read as, and the last sentence was trimmed, it would still read as   despite the end of the entry being trimmed - your prefix and suffix still remain. You may also use a  (a newline marker), which helps isolate the entry further by separating it with a full newline.

Reserved Tokens: Keeps this number of tokens in the context window for this entry. This will overwrite other content if necessary! Only use if it's very important.

Priority: The higher the priority, the earlier the entry is processed. Entries with low priority may be dropped to save room for higher priority ones.

Insertion Position: How far from the top (if positive) or the bottom (if negative) will the entry be inserted in the window. The unit is defined in Insertion Type. It can be a number of tokens, sentences, and newlines.

As an example, if you set it to -3 Newline, then it will insert the entry's text as soon as it finds the third newline, reading back from the bottom of the window. -1 will mean it is always placed at the very bottom of the Context, just as positive 1 will always place it at the very top of the Context. 0 will always be the very top.

Trim Direction: If the entry needs to be inserted partially due to lack of room in the context window, should it trim from the beginning towards the end, (top) end towards the beginning, (bottom) or omit the entire entry if it can't fit fully (Do Not Trim)?

⬆ Return to Page Top

Context Viewer
The Context Viewer is a powerful tool to identify what elements were used by the AI in the last generation. This helps you diagnose Memory, Author's Note and Lorebook usage. Check for bloat, trimmed entries, or ones that take too much space using this tool.



Identifiers
Lists the Identifier of each element of the context that describes it's origin from one of the following:
 * Story: From the main text.
 * Memory: From the Memory block.
 * Author's Note: From the Author's Note block.
 * Display Name of a Lorebook Entry: The name that you gave that entry in the Lorebook, not its keys.

Inclusion
Lists if this element is included in the context:
 * Included: Successfully inserted in the context.
 * Partially Included: Inserted in the context but some trimming was performed.
 * Not Included: Insertion in the context failed or was not attempted.

Reason
Lists the reason for this element's inclusion or omission:

Included
 * Default: Reserved to Story, Memory and Author's Note. Included by default.
 * Key Activated: This Lorebook entry was triggered by one of its keys.
 * Forced: This Lorebook entry was activated because it was set to Forced.

Omitted
 * Disabled: This Lorebook entry was omitted because it was disabled.
 * No key: This Lorebook entry was ommitted because it could not find any of its keys in the text.
 * No space: This entry was ommitted because it could not be allocated enough tokens to fit.
 * No text: This entry was deactivated because it contains no text.

Key
Lists the key that triggered this Lorebook entry.

Rsvd Tokens
Lists the amount of tokens reserved for this entry. This is usually lower than the Reserved Tokens setting of that entry, as that setting is the upper limit.

Tokens
Lists how many tokens are used by this entry solely on its own. Tokenization can cause a couple extra (or sometimes less) tokens to be used when this entry is placed in the text.

Trim Type
Lists how this entry was trimmed. There are four trim steps, which occur in this sequence: -
 * Fit the entire entry without trimming. (No Trim) If it doesn't fit, go to the next step:
 * The entry was trimmed to a new line character inside its text. (New Line) If this results in the entry having less than 30% of its allocated token content inserted, go to the next step:
 * The entry was trimmed to a sentence delimited (period, ellipse, semicolon) (Sentence) If this causes the entry to have less than 30% of its allocated token content inserted, go to the next step:
 * The entry is trimmed by the individual token, and then all the content that can fit in the space that remains is inserted. (Token) If this STILL fails, this is likely because the Prefix and Suffix can't fit in the context, so the entire entry is omitted.

Scenarios
You can create exportable scenarios from the options menu as well as importing them the same way you import a story, by dropping the file on the webpage or by clicking on the import story button when creating a new story.

Importing a scenario with placeholders
When importing a scenario, you may be asked to fill in some information. Those are called placeholders. You can simply edit those fields to your liking.

Turning on Import Settings will import the Generation settings that were used by the Scenario author.

putnplaceholders

Adding a placeholder
You can add a placeholder anywhere within your story's prompt, memory, author's note and lorebook entries.

Placeholders have to be written in this format:. The placeholder is divided into three parts:


 * id: the only mandatory part of the placeholder, it has to be unique. If you have more than one instance of the id, it will use the same value for each of the placeholders.
 * default: the default content of the field when importing the value.
 * description: the text displayed above the input field when importing the value. If there is no description set, the text displayed will be the id. You may want to word this like a question for ease.

To get the field from the picture above, you have to enter

Note for the lorebook entries
For the lorebook entries, you can add placeholders on the title of an entry, its descriptions and its keys. If you want to use regex for your keys, you have to prefix the regexp with a  for example to match with the name of a character, you have to write

⬆ Return to Page Top

Formats
Formats are different ways of writing Memory, Author's Note and Lorebook entries. Contributor Valahraban wrote an extensive research report on several formats, which goes into great details into various formats, their utility, and how to use them.

NovelAI does not recommend, endorse, or otherwise support any format type in particular.

⬆ Return to Page Top

-

Advanced Options
These settings allow you to fine-tune the generation settings to your linking. These get really technical so only explore them if you like messing with the finer things. Otherwise, leave them to their defaults; they're usually good as is.

Most of them deal with the Pool of possible tokens. To understand what this means, look at these examples:

could result in "up, down, left, right, across, around, fancy" and so on.

would result in fewer potential matches, such as "hot, bright, with".



Top-K Sampling
This setting affects the pool of tokens the AI will pick from by only selecting the most likely tokens, then redistributing the probability for those that remain. The pool will only contain the K most likely tokens. If the setting is set to 10, then your pool will contain the 10 most likely tokens. (Top-10 Sampling).

In plain English, lowering this setting causes more consistent Completions at the cost of creativity.

Nucleus Sampling
Relating to the previous setting, this adds up the probability of each potential Token in descending order of likelihood until it reaches the value specified. This value is an inverse percentage of likelihood for the next Token - therefore, lowering this value creates a smaller subset of probable Tokens.

In plain English, lowering this setting causes more consistent Completions at the cost of creativity.

As an example, if the most likely token has 30% chance, the second 25, the third 20, the fourth 10, the fifth 5, and the sixth 3, and your setting is at 0.9 (90%), then you would do: 30+25+20+10+5 = 90. The sixth most likely token and onwards will be removed from the pool.

Tail-Free Sampling
A tail in this context is the least-likely subset of Tokens to be chosen in a Completion. This alternative sampling method works by trimming the least-likely tokens by searching for the estimated tail' probability, removing that tail to the best of its ability, then re-normalizing the remaining sample.

This method may have a smaller impact on creativity while maintaining consistency. However, take note that it tends to behave strangely if your context does not contain a lot of data.

Repetition Penalty
Because text generation is based on patterns, repetition is a constant concern. The Repetition Penalty introduces an artificial dampener to the probability of a token depending on the frequency of its appearance in the Current Context.

As such, increasing this value makes a word less likely to appear for each time it shows up in the text. Do take note that this can get really awkward with words that are recurrent in the current context, such as names, or objects being discussed. With high Repetition Penalty, the AI may find itself unable to use a word repeatedly, and will need to substitute it with another which may be inappropriate.

Repetition Penalty Range
Defines the number of tokens that will be checked for repetitions, starting from the last token generated. The larger the range, the more tokens are checked.

Repetition Penalty Slope
The penalty to repeated tokens is applied differently based on distance from the final token. The distribution of that penalty follows a S-shaped curve. If the sloping is set to 0, that curve will be completely flat. All tokens will be penalized equally. If it is set to a very high value, it'll act more like two steps: Early tokens will receive little to no penalty, but later ones will be considerably penalized.

End of Sampling Token ID
If you wish to end your Completions upon reaching a specific Token, simply add it to this field. Doing so will cause the Completion to end prematurely upon generating the Token.

This can be used to trim the output to single sentences by inputting punctuation, or to increase the accuracy of Lorebook entries by pausing Completion when a defined word is reached.

Ban Token
Any Tokens added here will have their likelihoods reduced to zero. This means they will not appear in Completions. As this adjusts the relationships between Tokens, this will have an impact on the phrasing chosen by the AI. Be careful about what you ban, because this can heavily disrupt output if used incorrectly.

Banned Tokens
Relating to the previous setting, this field shows every Token currently blacklisted for generation. Clicking one of these tags will remove it from the list.

Ban Bracket Generation
At times, you may wish to include hints to the AI that are not considered for text generation. These can be encapsulated in square brackets ([ and ]) to relay information that will affect the Current Context while not being considered part of the actual text.

These most often take the form of hints. For more information on what to put between square brackets, see Keeping Track.

Trim AI Responses
A sentence delimiter separates subsequent text from the previous clause. Trimming AI responses to the last delimiter will prevent words from appearing after the latest delimiter in the generation.

Basically, this setting prevents dangling words from appearing like this:

To this:

Keep in mind that this can cause very short outputs depending on the current settings.

⬆ Return to Page Top

Account Settings
The Account Settings can be accessed from the Story Library, then clicking on the Menu button (Ξ), then Settings.

There are three settings currently:


 * Storage Location: Switches between using Local Storage (Off) and Serverside storage (On) by default.


 * Force 1024 Token Context Limit: If you have the Scroll or Opus Subscription plan, you can choose to use the Tablet plan context window by turning this setting On. This lets you demo for users with less money to spend, for example.


 * Switch between GPT-J 6B and Calliope: In the Off Position, you you will be using Calliope to generate text output. Calliope is a finetuned version of EleutherAI's GPT-NEO 2.7B model. GPT-J is a larger model, but not currently finetuned.

⬆ Return to Page Top