Welcome to The Crafting AI Prompts Framework, your guide to mastering the art of prompt creation, whether you're using ChatGPT, Microsoft Co-pilot, Claude, or any other AI model. If you're wondering how to craft a prompt, generate prompts, use prompting techniques, be secure, ethical and compliant, or create maintainable and sustainable prompts, you're in the right place. Here, we dive into the details of crafting effective and creative prompts, exploring what Prompt Engineering is and discussing various techniques and strategies.

We also focus on creating prompts that are maintainable and sustainable, so you won't end up with databases full of unused prompts like many companies do. In the Architecture section, you'll learn how to set up your prompts responsibly, ensuring they are reusable and adaptable over time. With these insights, you'll be equipped to create and improve prompts that work across different AI models while maintaining long-term efficiency and effectiveness.

And with great new tools, there are also risks. We will also discuss adversarial prompting, to make you aware of the potential risks and how to prevent them.

Before we dive into these topics, it's important to understand the basics of Generative AI, which is the technology behind creating new content like text, images, videos or sounds. Large Language Models (LLMs), a key part of Generative AI, are based on vast datasets and complex algorithms that allow them to understand and generate human-like text. Additionally, we'll touch on Diffusion Models, another type of Generative AI, which are used in creating high-quality images and other media. This foundational knowledge will make it easier to grasp the concepts and techniques of Prompt Engineering.

In the next section, we will explain what LLMs are, providing you with the essential background needed to fully appreciate the subsequent discussions on Prompt Engineering and the framework we've created.

What is an LLM?

A Large Language Model (LLM) is a type of artificial intelligence designed to recognize and generate human-like text based on vast amounts of data. These models are trained using extensive datasets from books, articles, websites, and other text sources to learn language patterns, grammar, context, and semantics. LLMs can perform various tasks such as language translation, text summarization, question answering, and even creative writing. They leverage complex algorithms and deep learning techniques to predict and generate coherent and contextually appropriate text, making them powerful tools for natural language processing applications.

LLMs are the foundation behind many well-known Generative AI applications like ChatGPT, Microsoft Copilot, and Google Gemini. These tools are revolutionizing various aspects of our daily lives, and their impressive capabilities are likely what brought you to this guide. By understanding LLMs, you'll gain insights into the technology powering these applications and how to harness their full potential through effective Prompt Engineering.

Prediction based

Before understanding how Prompt Engineering works, it's essential to grasp the concept of "predictions." Predictions are at the heart of how Large Language Models (LLMs) like ChatGPT function. LLMs are designed to predict the next token in a sequence based on the context provided by the previous tokens. Tokens can be words, parts of words, or even characters, depending on the specific model and its training. This process involves analyzing vast amounts of text data to learn language patterns and relationships between tokens. When you input a prompt, the model generates a response by predicting each subsequent token, creating coherent and contextually appropriate text.

Understanding that LLMs are prediction-based is crucial because it highlights that these models can occasionally make mistakes. Since they rely on probabilities derived from their training data, their predictions might not always be accurate or relevant. This inherent unpredictability underscores the importance of crafting precise and well-structured prompts to guide the model effectively.

For a deeper understanding of how predictions work and how LLMs operate, the below video from Code.org offers an excellent explanation. Watching this video will give you a solid foundation before diving into the Crafting AI Prompts Framework.

Token-Based

In Prompt Engineering, it's crucial to understand how tokens work. A great way to grasp what a token is, is by experimenting in OpenAI's Playground. Recently, many models were tested with the question: "How many R's are there in the word 'strawberry'?" Interestingly, most models provided the incorrect answer (1 or 2). In Prompt Engineering we call those incorrect answers: "Hallicunations". A hallucination refers to instances where an AI model generates information that is either incorrect, fabricated, or not grounded in the provided input or any external knowledge. Essentially, the AI "makes things up" that may appear plausible but are factually inaccurate or irrelevant.

But how can this happen, and how can we fix it?

This is a perfect example of how tokens work. Let's break it down in OpenAI's Playground:
First, let's tokenize the question: "How many R's are there in the word 'strawberry'?"

At the bottom of the image, you can see that the question is highlighted in different colors. Each color represents a different token, making it clear where each token starts and ends, which helps illustrate how the model processes the text.

As shown, the question is tokenized into 11 tokens, with "strawberry" being considered as a single token.
This means the model interprets "strawberry" as one entity, which can make answering the question challenging. Keep this in mind when crafting prompts.

In a large language model (LLM), a token is simply a number that represents a word, part of a word, or even a single character. Instead of working directly with text, the LLM breaks down the input into these tokens, which are then converted into numbers that the model can process. So, when we say "token," we’re really talking about a number that the model uses to understand and generate language.

In the image above, you can see how the text is broken down into numbers, which makes it easy to see how mistakes can happen along the way.

But if we change it, adding spaces or something else between "strawberry," each part gets its own token.

And if we use this modified text to ask a question in ChatGPT, we still get the correct results.

Mastering tokenization is a crucial aspect of Prompt Engineering, as it directly influences how language models interpret and respond to questions. By understanding how tokens work, we can craft more precise prompts that yield accurate responses from the model. This knowledge also helps us determine where the model excels and where it falls short, allowing us to make informed decisions about when to use Generative AI tools. For instance, in the "strawberry" example, we can solve the problem by adding spaces between the words. However, this approach isn't always practical. In other cases, we might need to ask the model to solve a problem using a programming language, like Python, and then obtain the result. By doing so, the model can leverage its strengths in programming to provide the correct answer.

Understanding the limits of a large language model (LLM) allows you to approach challenges more effectively. For example, if the model struggles to process a specific task or input, using a programming language like Python can help compute results or solve the problem. However, even programming is not always the right solution. Some challenges might require domain-specific tools, human expertise, or other specialized approaches that neither an LLM nor programming can address effectively.

This flexibility is crucial because not every problem can or should be solved with a single tool or method. A key aspect of Prompt Engineering is recognizing these boundaries, adapting your approach, and knowing when to combine or switch between tools to achieve the best outcome. By understanding and working within these limitations, you can maximize your effectiveness and solve problems in a sustainable, well-rounded way.

Context Window

A context window in a large language model (LLM) defines the maximum number of tokens the model can process at one time, effectively serving as the model's "current memory." Tokens, in this case, represent the fundamental units of text (words, subwords, or characters) that the model uses to comprehend and generate responses. The context window represents the working memory of the model, crucial for keeping dialogue or text generation coherent and relevant to the ongoing context. It is a fixed limit, typically specified by the architecture of the model (e.g., 4,096 tokens in GPT-3 or significantly larger in more advanced systems). Within this limit, the entire interaction—composed of both user input and the model's generated output—must fit for the model to function effectively.

The context window includes several key components:

• System Prompt: This is often a hidden prompt used to guide the model’s behavior, tone, or style. For example, it may define rules for politeness, formality, or task-specific constraints.
• User Prompt: This is the input provided by the user, whether a single question, an instruction, or a complex request involving detailed information or even added files.
• Previous Conversation History: In conversational use cases, maintaining coherence often requires appending past interactions, which consume tokens in the context window.
• Model Output: If you're including the model’s previous responses in the interaction, this too takes up space in the context window.

When the combined size of these components exceeds the context window, the model can no longer retain all the input data. In such cases, truncation occurs, where the oldest tokens (often from the beginning of the interaction or earlier parts of the system prompt) are removed. This can lead to the model losing critical context, which may affect its ability to provide accurate or relevant responses.

Another important aspect of the context window is that any information not included in it cannot be accessed by the model. For example, if you ask the model to play a game where it thinks of an animal and you guess it, the game won't work as expected. Why? The model has no persistent memory to store the chosen animal. Instead, it generates responses based on the immediate context of your prompts and its previous responses in the context window. Essentially, the model doesn't "remember" the animal—it appears to play along, but it’s simply generating plausible answers each time based on the conversation so far. This is because the context window is the only "memory" the model has, and information outside of it is inaccessible. Understanding this limitation is crucial for prompt engineers when designing interactions with large language models.

A deep understanding of how the context window operates is essential for maximizing the performance of LLMs, especially for complex or multi-step workflows. It allows engineers to craft prompts that balance depth, specificity, and brevity within the constraints of the model.

LLM Configurations

Before crafting a prompt, it's crucial to understand how LLM output parameters influence the results. These configuration options control the how, not just the what, of the outputs—and fine-tuning them can significantly enhance performance for specific tasks.

In many LLM-powered products and services—such as chatbots, virtual assistants, and other LLM tools—output parameters are typically pre-configured. However, when integrating LLMs into custom projects, it's essential to set these parameters appropriately. Default values are often provided, but customizing them to suit specific use cases is vital. For instance, when using a third-party API for text generation, specifying parameters like the maximum response length, response format, and the number of responses the model should generate is important.

Together, these settings shape the tone, accuracy, and usability of the LLM's outputs. Whether you aim for concise summaries, creative brainstorming, or deterministic answers, configuring these values is key to unlocking the model’s full potential.

What are Diffusion Models?

Diffusion models are a type of Generative AI designed to create high-quality images and other media by gradually transforming simple, random noise into structured outputs. These models work through a process of iterative refinement, starting with a noisy image and slowly "denoising" it, step by step, until it resembles the desired output. The process is akin to reversing the natural diffusion process, where noise spreads out and disrupts an image; instead, the model learns to reconstruct the image from this noise, guided by patterns it has learned from vast datasets.

Diffusion models are particularly powerful in generating detailed and realistic images, making them an essential tool in areas like art generation, medical imaging, and video creation. They offer an alternative approach to traditional methods of image generation, allowing for more controlled and fine-tuned outputs. Just like LLMs are the backbone of text-based AI applications, diffusion models are becoming foundational in the realm of AI-driven visual content creation, opening up new possibilities for creative and technical fields alike. Understanding diffusion models adds another layer to your grasp of Generative AI, enhancing your ability to work with and create prompts that guide these models effectively.

Now that you have a solid understanding of what Large Language Models (LLMs) and Diffusion Models are and how they work, we can move forward to the exciting world of Prompt Engineering. Ready to continue? Let's dive into the world of Prompt Engineering.