This is an early draft of a chapter from my upcoming book…

In understanding the Uplift system’s success, we need to understand the cognitive architecture and the graph system core to its contextual learning and key to creating dynamically creating prompts into the generative AI system used to produce helpful content contextually aware.

The generative artificial intelligence (AI) field has made remarkable strides in recent years and can generate highly sophisticated responses across various applications.  One of the critical factors in generating effective responses from generative AI systems is the design of the prompt or query that initiates the system’s output.  This process, known as prompt engineering or prompt framing, involves designing a prompt that provides the necessary context and information for the system to generate a relevant and accurate response.

Prompt engineering is a critical step in using generative AI systems effectively, as the quality and accuracy of the system’s output are directly tied to the quality of the prompt.  An effective prompt should be specific, unambiguous, and focused on a specific task or goal to provide the necessary context and information for the generative AI system to produce an accurate response.  By designing effective prompts, it is possible to leverage the full potential of generative AI systems to provide valuable insights and assist with a wide range of tasks and goals.

However, several common mistakes can occur in prompt engineering, limiting the effectiveness and accuracy of the generative AI system’s output.  These mistakes include providing too little or too much information, using ambiguous or vague language, and failing to consider potential biases.  To prevent bias, it is essential to carefully consider the language used in the prompt, the training data used to train the generative AI system, and to test the system’s output for potential biases.

Despite these challenges, prompt engineering remains an essential aspect of using generative AI systems effectively and is an ongoing research and development area.  By following best practices in prompt engineering and taking steps to mitigate potential biases, it is possible to create generative AI systems that are more accurate, reliable, and unbiased.  The following chapter will explore the principles of prompt engineering in more detail and provide practical guidance for designing effective prompts for generative AI systems.

What is prompt engineering?

Prompt engineering is a process of designing and crafting prompts for machine learning models that are intended to generate natural language responses.  The goal of prompt engineering is to create prompts that will elicit the desired output from the model, whether the text that answers a specific question generates new text based on a given context or performs a particular task.

Prompt engineering involves several steps, including defining the task or problem the model intends to solve, identifying relevant data sources and selecting appropriate training data, designing and refining prompts, and testing the model’s performance using various evaluation metrics.

The quality of the prompts used in training a language model can significantly impact the model’s performance and ability to generalize to new inputs.  Effective, prompt engineering requires a deep understanding of the underlying model architecture, the nature of the task, and the nuances of natural language.

The Basics of Generative AI Prompts

Generating effective prompts for using generative AI like Chat GPT involves several key steps:

  1. Define the task or goal: First, we need to define the specific task or goal we want the generative AI to accomplish. This task could be anything from answering questions to generating text based on a prompt.
  2. Identify the relevant data sources: We must identify the data sources used to train the model. This source could include text corpora, social media posts, customer support logs, or any other source of relevant text.
  3. Select appropriate training data: Once we have identified the data sources, we need to select the appropriate training data that will be used to train the model. This data should represent the inputs the model will encounter in the real world.
  4. Design and refine prompts: Our designed prompts should be clear and concise and provide enough context for the model to generate a coherent response. Refine the prompts by testing them with different inputs and adjusting them as needed.
  5. Evaluate model performance: Finally, we must evaluate the model’s performance using various metrics such as complexity, accuracy, and coherence. This evaluation will help us identify areas where the model is performing well and areas for further improvement.

In addition to these steps, it is also essential to consider the ethical and social implications of the prompts we are designing.  Generative AI has the potential to be used in a variety of applications, and it is crucial to ensure that the prompts we are designing are ethical, fair, and unbiased.

Common Mistakes People Make

People make several common mistakes when designing prompts for generative AI systems.  Here are a few:

  1. Poorly defined goals: One of the biggest mistakes people make is clearly defining their goals. It is essential to have a specific task or goal when designing prompts.  Otherwise, the resulting outputs may be vague or irrelevant.
  2. Insufficient training data: Another common mistake is insufficient training data to adequately represent the range of inputs the model may encounter in the real world. This lack of data can lead to poor performance and a lack of generalization.
  3. Biased training data: Training data that is biased or skewed can lead to biased or discriminatory outputs. It is vital to carefully select diverse training data representative of the target audience.
  4. Overly complex prompts: Too complex or confusing prompts can lead to poor performance and lack of coherence in the generated outputs. Keep the prompts simple and concise and provide enough context for the model to generate a coherent response.
  5. Lack of evaluation: Evaluating the model’s performance using various metrics and test inputs is essential. Lack of evaluation can lead to poor performance and an inability to identify areas for improvement.
  6. Failure to consider ethical implications: Generative AI has the potential to be used in a variety of applications, and it is essential to consider the ethical and social implications of the prompts being designed. Prompts that are biased or discriminatory can negatively impact society, and it is crucial to ensure that the prompts being designed are fair and unbiased.

Avoiding these common mistakes can help ensure that the prompts designed with generative AI systems are effective and generate outputs that meet the desired goals.

Other Approaches

There are other approaches to building effective prompts for use with generative AI.  Here are a few:

  1. Interactive prompts: Instead of designing prompts in advance, interactive prompts involve a dialogue between the user and the generative AI system. This approach allows the system to adapt to the user’s inputs and generate outputs tailored to their specific needs.
  2. Transfer learning: Transfer learning involves using a pre-trained language model as a starting point and then fine-tuning it on a specific task or domain. This approach can be more efficient and effective than training a language model from scratch and can result in better performance on specific tasks.
  3. Human-in-the-loop: Human-in-the-loop approaches (like Uplift) use human input to guide the generative AI system by providing feedback on generated outputs or actively participating in the generation process. This process can help ensure that the generated outputs are relevant, accurate, and meet the desired goals.
  4. Reinforcement learning: Reinforcement learning involves training the generative AI system through trial and error, receiving feedback on its outputs, and adjusting its behavior accordingly. This approach can be particularly effective for complex tasks where there may need to be a clear right or wrong answer.
  5. Multi-modal prompts combine text, images, and other input forms to generate outputs incorporating multiple modalities. This approach can be particularly effective for image captioning or video summarization tasks.

Each approach has its strengths and weaknesses, and the most effective approach will depend on the specific task and context in which the generative AI system will be used.

Avoiding Bias in Prompts

There are several ways to help ensure that a prompt does not produce bias in the response from a generative AI system:

  1. Careful selection of training data: One of the most important ways to prevent bias in the response is to carefully select training data representative of the target audience and free from bias. This process can involve selecting data from a diverse range of sources and taking steps to remove any bias that may be present in the data.
  2. Regular monitoring and evaluation: It is essential to regularly monitor and evaluate the performance of the generative AI system to identify any biases that may be present in the responses. This review process can involve reviewing a sample of generated responses and checking for any patterns of bias.  This process also can include using the generative AI system to check its output for bias.
  3. Use debiasing techniques: Various techniques can remove bias from the training data or the generated responses. Debiasing can include techniques such as counterfactual data augmentation, which involves creating new data by modifying existing data to remove bias.
  4. Involvement of diverse stakeholders: It is vital to involve diverse stakeholders in designing and evaluating the generative AI system, including individuals from different demographic groups whom the system may impact. This process can help identify and address any biases early in development.
  5. Regular updates and refinements: It is essential to regularly update and refine the generative AI system to ensure that it produces unbiased responses over time. This action can involve updating the training data, refining the prompts to generate responses, and implementing new debiasing techniques.

Taking these steps ensures that a prompt does not produce bias in the response from a generative AI system.  However, it is important to recognize that bias can be challenging to eliminate.  Ongoing monitoring and evaluation will be needed to ensure the system produces fair and unbiased responses over time.

Training and Data Support for Prompt Engineering

Adding details and training data before the prompt is essential when using an engineered prompt with generative AI for several reasons:

  1. Increases specificity and relevance: Providing additional details and training data before the prompt can help make the prompt more specific and relevant to the desired output. This extra detail can help ensure the generated response is more accurate and useful.
  2. Improves naturalness: When the prompt is conversational or human-like, providing additional details and training data can help improve the naturalness of the generated response. This style can help make the response more engaging and satisfying for the user.
  3. Reduces ambiguity: Providing additional details and training data can help reduce ambiguity in the prompt and clarify what the user is asking for. This additional material can help improve the accuracy and relevance of the generated response.
  4. Enables personalization: When additional details and training data are provided, the generative AI system can use that information to personalize the generated response to the specific needs and preferences of the user. This additional detail can make the response more relevant and valuable to the user.

Overall, adding details and training data before the prompt can help improve the generated response’s accuracy, relevance, and naturalness and enable personalization for the user.  It is an important step in the prompt engineering process and can help ensure that the generative AI system can effectively and efficiently generate high-quality responses.

This process also reduces the likelihood of the generative AI system having responses that are not just contextually aware but stay in context and reduce hallucinations by the underlying model.  Such errors in output can occur when things are vague or open to interpretation.

Let us look at how we can use our prompts.

What Steps Do we take in our program to Call Generative AI

Let us start with these steps:

  1. Define the initial prompt: Start by defining the initial prompt that will be used to call Chat GPT. This prompt should be carefully engineered to elicit a relevant and helpful response to our application.
  2. Call Chat GPT with the initial prompt: Use a C# library (if you are a C# engineer) or SDK to request the Chat GPT API with the initial prompt. The response from Chat GPT will be a generated text that can be used as the basis for the following prompt.
  3. Extract relevant information from the response: Parse the response from Chat GPT to extract the relevant information that will be used to construct the following prompt. This parsing could involve using natural language processing (NLP) techniques to identify key phrases, entities, or concepts in the generated text.
  4. Construct the next prompt: Based on the extracted information, construct the following prompt that will be used to call Chat GPT again. This prompt should elicit a relevant and informative response based on the information extracted from the previous response.
  5. Call Chat GPT with the following prompt: Use the (in this case, as a .NET developer) C# library or SDK to call Chat GPT API again with the next prompt. The response from Chat GPT will be a newly generated text.
  6. Validate the answer with Chat GPT: Finally, use Chat GPT again to validate the answer provided in the generated text. This process could involve asking a follow-up question or comparing the generated text to a pre-defined set of correct answers.

By following these steps, we can create a program that uses an engineered prompt to make a call to Chat GPT and, then, based on the response, creates another prompt and uses that to call Chat GPT again.  The generated text can then be validated using Chat GPT to ensure the response is accurate and relevant.

Here is an example in C# that uses the Open AI API to make a call to Chat GPT and then, based on the response, creates another prompt and uses that to call Chat GPT again.  It then uses Chat GPT to validate the answer:

using system;

using system.Collections.Generic;

using System.Net.Http;

using system.Threading.Tasks;

using Newtonsoft.Json;

 

class Program

{

private static readonly HttpClient client = new HttpClient();

 

static async Task Main(string[] args)

{

// Define the initial prompt

string prompt = “What is the capital of France?”;

 

// Call the OpenAI API with the initial prompt

string response = await CallOpenAI(prompt);

 

// Extract relevant information from the response

string capital = ExtractCapital(response);

 

// Construct the next prompt

string nextPrompt = $”What is the population of {capital}?”;

 

// Call the OpenAI API with the next prompt

string nextResponse = await CallOpenAI(nextPrompt);

 

// Validate the answer with Chat GPT

bool answerValid = ValidateAnswer(nextResponse);

 

Console.WriteLine($”The response to ‘{nextPrompt}’ is ‘{nextResponse}’ and is {(answerValid ? “valid” : “invalid”)}”);

}

 

static async Task<string> CallOpenAI(string prompt)

{

// Set up the request parameters

string apiKey = “YOUR_OPENAI_API_KEY”;

string endpoint = “https://api.openai.com/v1/engines/davinci-codex/completions”;

string requestBody = JsonConvert.SerializeObject(new

{

prompt = prompt,

max_tokens = 150,

n = 1,

stop = “\n”

});

 

// Make the API call

HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, endpoint);

request.Headers.Add(“Authorization”, $”Bearer {apiKey}”);

request.Content = new StringContent(requestBody, System.Text.Encoding.UTF8, “application/json”);

HttpResponseMessage response = await client.SendAsync(request);

string responseBody = await response.Content.ReadAsStringAsync();

 

// Parse and return the response

dynamic result = JsonConvert.DeserializeObject(responseBody);

return result.choices[0].text.ToString().Trim();

}

 

static string ExtractCapital(string response)

{

// Use NLP techniques to extract the capital from the response

// For this example, we’ll just return a hard-coded value

return “Paris”;

}

 

static bool ValidateAnswer(string response)

{

// Use Chat GPT to validate the answer

// For this example, we’ll just return a hard-coded value

return true;

}

}

Note that this example is for illustrative purposes only, and the implementation of `ExtractCapital` and `ValidateAnswer` will depend on your specific use case.

Structuring Input Data Before Your Prompt

When using a prompt with GPT, it is best to structure the input data to provide context and relevant information to the model.  Here are some tips for structuring input data before the prompt:

  1. Use clear and concise language: Generative AI is designed to work with natural language, so it is important to provide input data that is easy for the model to understand. Avoid using complex or convoluted language and use simple sentence structures whenever possible.
  2. Provide relevant information: The more relevant information you provide to the model, the better it will generate accurate and informative responses. This could include information about the topic being discussed, the context of the conversation, or any other details that might be relevant.
  3. Use consistent formatting: Consistent formatting can help the model understand the structure of the input data and generate more coherent responses. For example, use bullet points or numbered lists to break down complex information or use bold or italic text to highlight important details.
  4. Include examples: Providing examples can help the model better understand the responses we seek. For example, let us ask a generative AI model to generate marketing copy for a product.  We might provide examples of similar products and the types of language that have been effective in the past.
  5. Avoid bias: Finally, it is essential to avoid input data that might introduce bias into the model’s responses. This avoidance could include using discriminatory language or providing data representing only one perspective on a topic.  It is essential to be mindful of any potential biases in our input data and to take steps to mitigate them where possible.

Using A Generative Model to Check its Work

There are several ways we can use a generative AI model to check its work:

  1. Cross-validation: One way to check the accuracy of a generative AI model is to use cross-validation. This cross-validation involves training the model on a subset of the data and then using the remaining data to test the model’s accuracy.  By comparing the model’s predictions with the actual results, we can understand how accurate the model is.
  2. Self-attention: Some generative AI models, such as GPT-2 and GPT-3, use self-attention to help them check their work. Self-attention allows the model to look back at the previous words it has generated and use that information to inform its next prediction.  This process can help the model avoid repeating itself or generating responses that are inconsistent with previous predictions.
  3. Language modeling evaluation: Another way to check the accuracy of a generative AI model is to use language modeling evaluation techniques. These involve feeding the model a set of sentences or phrases and evaluating its ability to predict the next word in the sequence.  By comparing the model’s predictions with the actual results, we can understand how accurate the model is at predicting language.
  4. Human evaluation: Finally, one of the most effective ways to check the accuracy of a generative AI model is to have humans evaluate its output. By having human evaluators read and rate the quality of the model’s output, we can understand how well it performs and where it might need improvement.  This process can be time-consuming, but it can also provide valuable insights into the strengths and weaknesses of the model.

Checking Output for Bias and Logical Fallacies

Checking responses from a generative AI system for bias and logical fallacies can be challenging.  However, there are a few techniques that can be used to help identify and mitigate these issues programmatically:

  1. Use bias detection tools: Several natural language processing (NLP) libraries and tools can help identify text bias. For example, the Google Cloud Natural Language API includes a feature called “entity sentiment analysis” that can identify the sentiment of specific entities in text.  Other tools like IBM Watson Personality Insights can help detect personality traits that may indicate bias.
  2. Check for logical fallacies: NLP tools can detect common logical fallacies in text. For example, the Stanford NLP Group offers a “CoreNLP” tool that includes a module for detecting common logical fallacies like ad hominem attacks and false dichotomies.
  3. Compared to known data sets: One approach to identifying bias is to compare the output of the generative AI system to known data sets free from bias. For example, we could compare the system’s output to a data set of news articles vetted for bias by human editors.
  4. Human evaluation: Human evaluation is the most effective way to detect bias and logical fallacies in the text. We could use crowdsourcing platforms like Amazon Mechanical Turk to have humans review and evaluate the output of the generative AI system for bias and logical fallacies.

It is worth noting that while these techniques can be effective in identifying bias and logical fallacies, they could be more foolproof.  It is important to approach generative AI systems cautiously and consider the potential ethical implications of using them in various contexts.

Graph Data

When structuring graph data for use in information structured before a prompt for use with generative AI systems, it is important to remember the nature of graph data and how the AI system can best leverage it.  Here are some tips for structuring graph data for use with generative AI systems:

  1. Use a graph database: Graph databases are specifically designed to store and manage graph data and can be a powerful tool for organizing and accessing graph data for use with generative AI systems. Uplift, for example, uses a custom graph database system, but we can use popular graph databases, including Neo4j and Amazon Neptune.
  2. Use a query language: Graph databases typically have query languages, such as Cypher for Neo4j and Gremlin for Amazon Neptune. These query languages allow us to easily retrieve and manipulate graph data in a structured way that generative AI systems can leverage.
  3. Use node and edge labels: When structuring graph data, it is important to use clear and consistent labels for nodes and edges. This can help the generative AI system understand the relationships between different pieces of data and generate more accurate and relevant responses.
  4. Use property values: Besides labels, including property values for nodes and edges can be helpful. These values can provide additional context and information that the generative AI system can leverage.  For example, storing information about a person in a graph database might include properties like age, occupation, and location.
  5. Preprocess the data: It can be helpful to preprocess the graph data before using it with a generative AI system. This preprocessing might involve filtering out irrelevant data, aggregating data to reduce complexity, or normalizing the data to ensure consistency.  By preprocessing the data, we can help ensure that the generative AI system works with high-quality and relevant data.

Now let us look at a specific example.  We will start with the following prompt:

Prompt 1: “Tell me about fish.”

This simple and general prompt will likely produce a broad range of responses.  It needs to be more specific to guide the generative AI system toward providing valuable and relevant information.  While it may be helpful in some contexts, it does not provide enough context or information to generate a specific response.

Prompt 2: “What are some common characteristics of cichlids?”

This prompt is more specific than the first, focusing on a particular type of fish.  It provides a more precise context and specific information, allowing the generative AI system to generate a more focused and relevant response.  However, it is still relatively broad and may result in various responses depending on the generative AI system and training data.

Take a look at this last example.

Prompt 3: “What are the most effective strategies for breeding African rift valley cichlids, and what are some of the most common challenges breeders face?”

This prompt is highly specific and detailed, providing much contextual information upfront.  It is designed to guide the generative AI system toward producing a specific response relevant to African rift valley cichlid breeding.  This specificity and detail level will likely produce more accurate and informative responses.

Additional information that would make the last prompt even better:

– Information about specific species of African rift valley cichlids.

– Details about optimal water conditions and temperature for breeding.

– Insights into the genetics of African rift valley cichlids and their breeding patterns.

– Case studies or examples of successful cichlid breeders and their techniques.

Discussion

From all of this, we can infer the following about prompt engineering:

  1. Prompt engineering involves creating a well-structured prompt that provides the context and information for a generative AI system to generate a relevant and accurate response.
  2. Effective prompts should be specific, unambiguous, and focused on a specific task or goal.
  3. Common mistakes in prompt engineering include providing too little or too much information, using ambiguous or vague language, and failing to consider potential biases.
  4. To prevent bias, it is essential to carefully consider the language used in the prompt and the training data used to train the generative AI system and test its output for potential biases.
  5. Structured data like graphs can provide additional context and information to the generative AI system, potentially leading to more accurate and relevant responses.
  6. Human evaluation of the output of generative AI systems can help identify biases and logical fallacies in the system’s responses.

Prompt engineering is an important aspect of using generative AI systems effectively.  Careful consideration should be given to the design of prompts; the training data used to train the system, and the evaluation of the system’s output.

Conclusion

In conclusion, the chapter on prompt engineering has highlighted the importance of designing effective prompts when using generative AI systems.  Effective prompts provide the necessary context and information for the system to generate relevant and accurate responses while minimizing potential biases, errors, and hallucinations.

To design effective prompts, it is vital to consider the specific task or goal the generative AI system will be used for and to provide unambiguous language that accurately conveys this task or goal.  Additionally, structured data like graphs can provide additional context and information to the system, potentially leading to more accurate and relevant responses.

Common mistakes in prompt engineering include providing too little or too much information, using ambiguous or vague language, and failing to consider potential biases.  To prevent bias, it is important to carefully consider the language used in the prompt, the training data used to train the generative AI system, and to test the system’s output for potential biases.

The chapter has also highlighted the importance of human evaluation of the output of generative AI systems, which can help identify biases and logical fallacies in the system’s responses.  Combining human evaluation with practical, prompt engineering makes it possible to create generative AI systems that are more accurate, reliable, and unbiased.

Overall, prompt engineering is an essential aspect of using generative AI systems effectively, and it requires careful consideration and attention to detail.  By following best practices in prompt engineering and taking steps to mitigate potential biases, it is possible to create systems that use generative AI to provide valuable insights and assist with a wide range of tasks and goals.