Prompt/Response Optimiser

Summary: Prompt/response optimiser refines the prompts/responses according to the desired input or output content and format.

Context: Users may struggle with writing effective prompts, especially considering the injection of comprehensive context. Similarly, it may be difficult for users to understand the agent’s outputs in certain cases.

Problem: How to generate effective prompts and standardised responses that are aligned with users’ goals or objectives?

Forces:

  • Standardisation. Prompts and responses may vary in structure, format, and content, which will lead to potential confusion or inconsistent behaviours of the agent.
  • Goal alignment. Ensuring that prompts and responses are aligned with the ultimate goal or objective can facilitate the agent to achieve desired results.
  • Interoperability. The generated prompts and responses may be directly input to other components, external tools or agents for completing further tasks.

Solution: Fig. 1 illustrates a high-level graphical representation of prompt/response optimiser. A user may input initial prompts to the agent, however, such prompts may be ineffective due to the lack of relevant context, unintentional injection attacks, redundancy, etc. In this regard, prompt/response optimiser can construct refined prompts and responses adhering to predefined constraints and specifications. These constraints and specifications outline the desired content and format for the inputs and outputs, ensuring alignment with the ultimate goal. A prompt or response template is often used in the prompt/response optimiser as a factory for creating specific instances of prompts or responses [1, 2]. This template offers a structured approach to standardise the queries and responses, improving the accuracy of the responses and facilitate their interoperations with external tools or agents. For instance, a prompt template can contain the instructions to an agent, some examples for few-shot learning, and the question/goal for the agent to work.

Figure 1. Prompt/response optimiser.

Benefits:

  • Standardisation. Prompt/response optimiser can create standardised prompts and responses regarding the requirements specified in the template.
  • Goal alignment. The optimised prompts and responses adhere to user-defined conditions, hence they can achieve higher accuracy and relevance to the goals.
  • Interoperability. Interoperability between agent and external tools is facilitated by prompt/response optimiser, which can provide consistent and well-defined prompts and responses for task execution.
  • Adaptability. Prompt/response optimiser can accommodate different constraints, specifications, or domain-specific requirements by refining the template with a knowledge base.

Drawbacks:

  • Underspecification. In certain cases, it may be difficult for prompt/response optimiser to capture and incorporate all relevant contextual information effectively, especially considering the ambiguity of users’ input, and dependency on context engineering. Consequently, the optimiser may struggle to generate appropriate prompts or responses.
  • Maintenance overhead. Updating and maintaining prompt or response templates may cause significant overhead. Changes in requirements may necessitate modifying multiple templates, which is time-consuming and error-prone.

Known uses:

  • LangChain. LangChain provides prompt templates for practitioners to develop custom foundation model-based agents.
  • Amazon Bedrock. Users can configure prompt templates in Amazon Bedrock, defining how the agent should evaluate and use the prompts.
  • Dialogflow. Dialogflow allows users to create generators to specify agent behaviours and responses at runtime.

Related patterns:

  • Passive goal creator and proactive goal creator can first handle users’ inputs and transfer the goals and relevant context information to prompt/response optimiser for prompt refinement.
  • Self-reflection, cross-reflection, and human-reflection. The reflection patterns can be applied to assess and refine the output of prompt/response optimiser.
  • Agent adapter. Prompt/response optimiser can improve users’ inputs, and the optimised prompts can be sent to
    other agents for goal achievement, while agent adapter focuses more on the utilisation of external tools.

References:

[1] A. Zhao, D. Huang, Q. Xu, M. Lin, Y.-J. Liu, and G. Huang, “Expel: Llm agents are experiential learners,” arXiv preprint arXiv:2308.10144, 2023.

[2] R. Schumann, W. Zhu, W. Feng, T.-J. Fu, S. Riezler, and W. Y. Wang, “Velma: Verbalization embodiment of llm agents for vision and language navigation in street view,” arXiv preprint arXiv:2307.06082, 2023.