Prompt Generation

When writing a prompt, it may be useful to access or modify some of the variables passed as input to the AiService. Qute can be used to automatically handle these variables within the prompt.

For example, suppose you want to create a prompt that, given a conversation and a follow-up question, rephrases the follow-up question as a standalone question. Qute simplifies this by allowing you to define the prompt in the following format:

@SystemMessage("""
    Given the following conversation and a follow-up question,
    rephrase the follow-up question to be a standalone question.

    Context:
    {#for m in chatMessages}
        {#if m.type.name() == "USER"}
            User: {m.text()}
        {/if}
        {#if m.type.name() == "AI"}
            Assistant: {m.text()}
        {/if}
    {/for}""")
public String rephrase(List<ChatMessage> chatMessages, @UserMessage String question);

In this example, the chatMessages list is automatically processed by Qute and transformed into the following format:

User: <text>
Assistant: <text>
...

This allows for the dynamic construction of prompts based on the provided input. For more information on how to use Qute, see the official documentation.

ChatMessage Formatting with TemplateExtensions

In the previous section we described how to use Qute to dynamically manage variables passed to an AiService. To simplify the prompt structure, a TemplateExtension is provided for List<ChatMessage> objects that provides methods to automatically format the contents of the list. This means that whenever a List<ChatMessage> is passed as a parameter to an AiService, the extension methods can be used to format the list without having to manually write loops or conditionals.

The list of extension methods are:

  • extractDialogue(userPrefix, assistantPrefix, delimiter):
    Formats the conversation by applying custom prefixes for user and assistant messages, and custom delimiter to separate them. This method is the most flexible and allows full customisation of the output format.

  • extractDialogue(delimiter):
    Formats the conversation using the default prefixes (User: and Assistant:) but allows for the specification of a custom delimiter between messages.

  • extractDialogue():
    Provides the simplest formatting, using the default prefixes (User: and Assistant:) and separating messages with a newline. This is useful for basic formatting without the need for additional customization.

Example 1: Using custom prefixes and delimiter:

@SystemMessage("""
    Given the following conversation and a follow-up question,
    rephrase the follow-up question to be a standalone question.

    Context:
    {chatMessages.extractDialogue("U:", "A:", "|")}""")
public String rephrase(List<ChatMessage> chatMessages, @UserMessage String question);

This would format the conversation using U: and A: as prefixes, and | as the delimiter between messages.

Example 2: Using a custom delimiter:

@SystemMessage("""
    Given the following conversation and a follow-up question,
    rephrase the follow-up question to be a standalone question.

    Context:
    {chatMessages.extractDialogue("-")}""")
public String rephrase(List<ChatMessage> chatMessages, @UserMessage String question);

In this case, the conversation will be formatted with the default User: and Assistant: prefixes, but messages will be separated by -.

Example 3: Using the default formatting:

@SystemMessage("""
    Given the following conversation and a follow-up question,
    rephrase the follow-up question to be a standalone question.

    Context:
    {chatMessages.extractDialogue}""")
public String rephrase(List<ChatMessage> chatMessages, @UserMessage String question);

This will format the conversation using the default prefixes (User: and Assistant:) and a newline between each message, resulting in a simple structured output.

Using the chat_memory placeholder

When working with AiService instances that have memory enabled, you have access to a special placeholder called chat_memory. This placeholder allows you to refer directly to the list of ChatMessage objects stored in the memory of the AiService, simplifying your prompt construction.

Instead of passing the List<ChatMessage> as a parameter, you can use the chat_memory placeholder in your @SystemMessage to automatically include the conversation history.

Since chat_memory refers to a List<ChatMessage>, you can use the TemplateExtension methods available for List<ChatMessage> to format the list directly in the prompt.

Example:

@SystemMessage("""
    Given the following conversation and a follow-up question,
    rephrase the follow-up question to be a standalone question.

    Context:
    {chat_memory.extractDialogue}""")
public String rephrase(@UserMessage String question);