MP-3: Quantum workflows orchestrator

Summary: In the context of hybrid quantum-classical workflows (Weder et al. 2020; Weder et al. 2021; Leymann and Barzen 2021) or quantum micro-service architecture (O’Meara et al. 2023), this pattern is designed to automatically deploy quantum components to the most suitable quantum provider, as well as provide automated scheduling of quantum programs to run continuously, or according to time-based or event-based schedules.

Problem: In quantum AI systems, the technological stack significantly expands in both size and complexity, leading to increased interdependencies and interactions among various system components. The challenge involves identifying, deploying, and executing quantum components while ensuring compatibility, performance, and availability of the entire application, given that quantum hardware platforms have NISQ-era limitations.

Solution: The software development and architectural designs of quantum AI systems necessitate tools that can
structure software transactions into workflows. A workflow is defined as a technology that specifies the partial order of a collection of heterogeneous activities aimed at achieving a composite goal (Leymann and Barzen 2021). The specific sequence of these activities is dictated by a workflow model. Typically represented as nodes in a directed graph, workflow models delineate the required tasks, their execution sequence, and the data flow between them. A workflow engine, or orchestrator, facilitates the automatic orchestration, deployment, and execution (Weder et al. 2020). The orchestrator is often implemented as middleware that analyses high-level programming language code, identifies quantum components, executes a classic-quantum split, and generates corresponding API functions. This middleware pattern, which can be implemented on either the server side or client side, generally includes components such as an orchestrator (De Stefano et al. 2022; O’Meara et al. 2023), a language translator/generator (De Stefano et al. 2022; Romero-Alvarez et al. 2024), and a scheduler (De Stefano et al. 2022; O’Meara et al. 2023).

Benefits:

  • Autonomy. The scheduling and deployment of quantum workflows are automated, requiring no human intervention.
  • Hardware decoupling. This pattern promotes the separation of quantum software from specific hardware and interface constraints.
  • Availability. The primary purpose of this pattern is to ensure the availability of quantum services for the application when needed.

Drawbacks:

  • Reliability. This pattern introduces a single point of failure, which means that if one part of the middleware fails, it could potentially disrupt the entire service, causing system-wide outages.
  • Response time. This pattern can result in extended response times in the development of quantum AI systems. Each added middleware layer introduces additional complexity, potentially delaying the delivery of data and responses to user requests.
  • Performance. As increasing volumes of processes and data are channeled through the additional middleware layers inherent in this pattern, these layers may become overwhelmed. This congestion can slow down throughput and consequently degrade the overall performance of the system.

Known uses:

  • Saurabh et al. 2023 considered integration patterns in quantum middleware and used well-established high-performance computing abstractions for managing workloads, tasks, and resources to integrate quantum computing into HPC systems.
  • Weder et al. 2021 presented a prototypical implementation that integrates topology and workflow orchestrations, utilising the Camunda workflow engine and the OpenTOSCA ecosystem.
  • Aparicio-Morales et al. 2023 propose an on-the-fly cost minimiser for quantum microservices, as part of a middleware service. This tool identifies the most cost-effective classical and quantum machines for deploying hybrid microservice-based applications.