Guidelines for prompt engineering with ChatGPT
by Konstantina Lazaridou, Machine Learning Engineer
Did you know that DeepLearning.AI, in collaboration with OpenAI, offers a free short course on prompt engineering techniques for ChatGPT for developers? You can find more information about the course here. We have participated in the course and have summarised in this article the main takeaways and tips on how to write effective prompts for large language models (LLMs) and in particular ChatGPT.
There are two important strategies to keep in mind when writing prompts for ChatGPT, either in the playground environment or in a software application: write clear instructions and give the model time to think
Write clear and specific instructions
When communicating with ChatGPT, it is essential to be precise on the task that we ask it to do, in order to maximise the chances that it answers with the information we expect. It helps to use delimiters to wrap the text that we want ChatGPT to process. For instance, in a prompt for email classification, you can write “Please classify the following text that is enclosed in triple quotes into spam or not spam“ and then include the email text and wrap it in triple quotes. This way the model knows where to focus on. It also helps to ask the model to form its output in a particular format, for instance the json format, in case we wish to process the output further in our implementation.
Additionally, you can request different handling for different cases, for instance “If no answer is found for this task, please don’t make up one and return ‘Unknown’” can be added to the end of the prompt to condition the behaviour of ChatGPT. If the original prompt does not result in the desired outcome, you can also give some examples to the model (few shot prompting) before asking it to do the final task. This can be done either by providing the examples during a prior interaction or inside the task prompt itself. Finally, as explained later in the course, it is often necessary to work iteratively with an LLM, because it is rarely the case that the requested task is solved immediately. Hence, after prompting the model for the first time, one can also continue interacting with it further, by making the requirements more and more precise, until the problem description is clear enough for the LLM.
Give the model time to think
Some tasks might be complicated for ChatGPT to complete and might require complex and sophisticated reasoning, for example solving mathematical exercises. In this case, it helps to let the model know what kind of steps it should take to approach the task, and hence encourage it to follow a reasonable step-by-step thought process. Clarifying the output format can enable the model to understand the task as mentioned earlier, but it could also support it to tackle complex problems successfully. Another good technique to follow with complex problems is to ask ChatGPT to find a way to solve the task on its own and based on the given examples, before trying to solve the task for the new input example. This way, it is encouraged to take the necessary steps and be methodical, instead of rushing or making up information in order to provide a quick answer for the new data point. Finally, you can also try requesting the model to provide reasons for its decisions and this is a strategy that might increase the chances that the prompt works well in the first attempt. Generally, asking for explainability or relevant sources can potentially prevent the model from hallucinating (creating new and baseless facts) additional information.
Conclusion
Our impression from the course is that it provides very insightful information regarding prompts, especially how to interact with ChatGPT and how to achieve precisely the desired result. The rest of the course contains examples of prompts separated into groups of similar use cases, i.e., how to write proper prompts to summarise or expand text, change the text (e.g., translate it), classify it and extract information from it and, finally, how to create a chatbot for a given application (with system, user and assistant roles). Dealing with potential hallucinations or not evidence-based generated text still remains an open issue, but the course provides useful prompt engineering techniques that are a step towards this direction.