One-Shot Model Querying

Summary: The foundation model is accessed in a single instance to generate all necessary steps for the plan.

Context: When users interact with the agent for specific goals, the included foundation model is queried for plan generation.

Problem: How can the agent generate the steps for a plan efficiently?

Forces:

  • Efficiency. For certain pressing tasks, the agent should be able to conduct planning and respond in a short amount of time.
  • Overhead. Users need to pay for each interaction with commercial foundation models.

Solution: Fig. 1 illustrates interactions between user and agent within one-shot model query. After a user specifies goals and constraints in one prompt, the agent will query the incorporated foundation model to generate a corresponding response (e.g. plan for action). The foundation model does not require multiple interactions to comprehend the context and requirements. In this manner, the agent can devise a multi-step plan to achieve a broad goal, and provide a holistic explanation for this plan without delving into detailed reasoning steps.

Figure 1. One-shot model querying.

Benefits:

  • Efficiency. The agent can generate a plan to achieve users’ goals by querying the underlying foundation model only once, which saves consumed time.
  • Cost-efficiency. Users’ expenses can be reduced since the foundation model is queried for one time.
  • Simplicity. One-shot model querying can satisfy the tasks that do not require complex action plans.

Drawbacks:

  • Oversimplification. For complex tasks, one-shot model querying may not be able to fully capture all requirements at one time, hence oversimplifying the tasks and cannot return a correct response.
  • Lack of explainability. One-shot model querying may suffer the lack of explainability as the incorporated foundation model is queried only once, which may not provide detailed reasoning steps for plan generation.
  • Size of the context window. The response quality may be constrained considering the foundation models’ current capability of handling long conversational contexts and the token limits.

Known uses: One-shot model querying can be considered configuration or use by default when a user is leveraging a foundation model, while CoT and Zero-shot-CoT both exemplify this pattern [1, 2].

Related patterns:

  • Incremental model querying. Incremental model querying can be regarded an alternative of one-shot model querying with iteration.
  • Single-path plan generator. One-shot model querying enables the generation of single-path plans by only querying the foundation model for one time.
  • Multimodal guardrails. Multimodal guardrails serve as an intermediate layer, managing the inputs and outputs of model querying.

References:

[1] L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin et al., “A survey on large language model based autonomous agents,” Frontiers of Computer Science, vol. 18, no. 6, pp. 1–26, 2024.
[2] L. Wang, W. Xu, Y. Lan, Z. Hu, Y. Lan, R. K.-W. Lee, and E.-P. Lim, “Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 2609–2634. [Online]. Available: https://aclanthology.org/2023.acl-long.147