# Understanding AI Model's Token Limit AI models, like the ones used for creating chatbots, have a maximum token limit of 4096, which includes both the input text and the generated output. ### What are Tokens? In the context of natural language processing and AI models, tokens are individual units of text that the model processes. These units can vary in size, often representing words or characters. For instance, in the sentence "Hello, how are you?" there are six tokens: "Hello," ",", "how," "are," "you," and "?". Tokens are essential for the model to understand and generate text effectively. ### Token Limitations and Budget Given that the AI model has a maximum token limit of 4096, both the input text and the generated output from the model must fit within this limitation. Further, the number of tokens expected in the response (max\_new\_tokens) also limits the available tokens for generating text. Keeping these constraints in mind is crucial when creating chatbots to ensure the conversation remains within the token budget. ### Chatbot's Personality and Example Dialogue In the process of defining a chatbot and providing example dialogues, tokens are consumed. The chatbot's personality description, including traits, background information, and other details, can significantly impact the available token count. As a best practice, aim to keep the chatbot's personality description within the range of 900-2000 tokens to leave room for other aspects of the conversation. ### Message History Message history refers to the ongoing conversation or interaction with the chatbot. It includes the user's input, the chatbot's responses, and any contextual information needed for the conversation to flow naturally. However, due to the token limitations, the length of the message history that can be included is restricted. ### Optimizing Token Usage To make the most of the limited token budget, it's important to be concise and prioritize essential information. Here are a few tips to optimize token usage: 1. Keep the chatbot's personality description and examples brief but effective, aiming for a total character definition within the range of 800 to 2000 tokens. 2. Use concise language and avoid unnecessary verbosity in the Personality and Scenario. 3. Consider summarizing or paraphrasing information to save tokens while maintaining clarity. ### Conclusion Creating chatbots using AI models offers exciting possibilities, but it's essential to work within the limitations of token budgets. By understanding these constraints and optimizing token usage, you can create engaging chatbots while maintaining conversational coherence. With a clear understanding of tokens and their impact, you're ready to bring your chatbots to life! --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.hiwaifu.com/product-guides/creating-chatbots/understanding-ai-models-token-limit.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.