Workspace 6.21.5
Running nested workflows in parallel on a local machine
[ Previous: Directing the execution of parallel loops to specific machines ] [ Up: Introductory Tutorials ] [ Next: Performing SSH operations in your workflows ]

Introduction

Workspace can be used to parallelise (and distribute) aspects of a workflow. In this tutorial we will run multiple nested workspaces asynchronously (i.e. in parallel). Our nested Workspace will simulate a lengthy operation via the Pause operation. We will then mark this workflow as running asynchrously, copying it and then set up our workflow to run both copies simultaneously. Once both parallel workflows have completed we will write some text to indicate completion to the workspace log window.

Contents

Configuring a workspace for parallel execution

  1. Open the workspace editor.
  2. In the operation catalogue, locate the LogText operation and drag this onto your workflow.
  3. In the operation catalogue, locate the Pause operation and drag this onto your workflow.
  4. Connect the LogText operation's dependency output to the Pause operation's dependency input.
    Connecting LogText to Pause
    .
  5. Right click the Pause operation's Dependencies output and choose the "Create workspace output" menu item.
    Pause output
    .
  6. In the operation catalogue, locate the WorkspaceInput operation and drag this onto your workflow.
  7. Connect the WorkspaceInput's output labelled "Input" to the LogText's "Text to log" input.
    Connecting WorkspaceInput to LogText
    .
  8. Select all the operations, right click and choose the "Nested workspace from selection" menu item.
    Nested workspace
    .
  9. Select the nested workspace, right click and select the "Properties..." menu item.
  10. Set the Update behaviour to Atomic.
    Nested workspace properties
    .
  11. Remove the WorkspaceInput and the WorkspaceOutput operation from your workflow.
  12. In the operation catalogue, locate the RunOperationsInParallel operation and drag onto your workflow.
  13. Select your nested workspace, and copy it (CTRL + C) and then paste it (CTRL + V) into your main workflow, connect both nested workspace's dependency output to RunOperationsInParallel's "Parallel operations" input.
    Nested workspace copy connected to the RunOperationsInParallel operation
    .
  14. In the operation catalogue, locate the LogText operation and drag this onto your workflow.
  15. Set the LogText's "Text to Log" input to the string "Tasks complete!" (excluding quotes).
  16. Connect the Dependencies output of RunOperationsInParallel to the LogText's Dependencies input.
  17. Create a workspace output for LogText's output.
    Our final workspace
    .
  18. Select your first nested workspace and set the input labelled "Input" value to 1.
  19. Select your second nested workspace and set the input labelled "Input" value to 2.
  20. Run the workspace.
  21. You should see both nested workspaces execute simultaneously and once both are complete (and ONLY when both are complete) the text "Tasks complete!" is logged to the workspace log window.
    Workflow output
  22. Try experimenting with different values for Pause in each of your nested workspaces and re-run. You can force a clean re-run of your workflow by selecting all the operations in your main workflow (CTRL + A), right clicking and choose the "Reset selected operation(s)" menu.
  23. Try also tagging a nested workflow with a feature as per the Directing the execution of parallel loops to specific machines tutorial. In doing so you can force your nested workflows to execute not only in parallel but also to run on specific remote machines and / or clusters.

Summary

This concludes the tutorial on running multiple workflow asynchronously.


[ Previous: Directing the execution of parallel loops to specific machines ] [ Up: Introductory Tutorials ] [ Next: Performing SSH operations in your workflows ]