The documentation you're currently reading is for version 3.0.0. Click here to view documentation for the latest stable version.
Orquesta Workflow Definition¶
The Orquesta workflow DSL (Domain Specific Language) is the native DSL for the Orquesta workflow engine. This DSL is derived from OpenStack Mistral.
The following is the list of attributes that makes up the workflow model. A workflow takes input,
perform a set of tasks in predefined order, and returns output. The workflow model here is a
directed graph where the tasks are the nodes and the transitions and their condition between tasks
form the edges. The tasks that compose a workflow will be defined in the DSL as a dictionary named
tasks where the key and value is the task name and task model respectively.
|version||Yes||The version of the spec being used in this workflow DSL.|
|description||No||The description of the workflow.|
|input||No||A list of input arguments for this workflow.|
|vars||No||A list of variables defined for the scope of this workflow.|
|tasks||Yes||A dictionary of tasks that defines the intent of this workflow.|
|output||No||A list of variables defined as output for the workflow.|
The following is a simple workflow example that illustrates the various sections of the model:
version: 1.0 description: A simple workflow. # A list of strings, assuming value will be provided at runtime or # key value pairs where value is the default value when value # is not provided at runtime. input: - arg1 - arg2: abc # A list of key value pairs. vars: - var1: 123 - var2: True - var3: null # A dictionary of task definition. The order of execution is # determined by inbound task transition and the condition of # the outbound transition. tasks: task1: action: core.noop next: - do: task2 task2: action: core.noop # A list of key value pairs to output. output: - var3: <% ctx().arg1 %> - var4: var41: 456 var42: def - var5: - 1.0 - 2.0 - 3.0
|delay||No||If specified, the number of seconds to delay the task execution.|
|join||No||If specified, sets up a barrier for a group of parallel branches.|
|with||No||When given a list, execute the action for each item.|
|action||No||The fully qualified name of the action to be executed.|
|input||No||A dictionary of input arguments for the action execution.|
|next||No||Define what happens after this task is completed.|
As described above, the workflow is a directed graph where the tasks are the nodes and the
transitions with their criteria between tasks form the edges. The starting set of tasks for
the workflow are tasks with no inbound edges in the graph. On completion of the task, the next
set of tasks defined in the list of task transitions under
next will be evaluated. Each task
transition will be represented as an outbound edge of the current task. When the criteria
when of the transition is met, the next set of tasks under
do will be invoked.
If there are no more outbound edges identified, then the workflow execution is complete.
Each task defines what StackStorm action to execute, the policies on action execution, and what happens after the task completes. All of the variables defined and published up to this point (aka context) are accessible to the task. At its simplest, the task executes the action and passes any required input from the context to the action execution. On successful completion of the action execution, the task is evaluated for completion. If criteria for transition is met, then the next set of tasks is invoked, sequentially in the order of the transitions and tasks that are listed.
If more than one tasks transition to the same task and
join is specified in the latter (i.e. the
barrier_task in the example below), then the task being transitioned into becomes a
barrier for the inbound task transitions. There will be only one instance of the barrier task. In
the workflow graph, there will be multiple inbound edges to the barrier node.
The following workflow definition illustrates the execution of parallel branches. The barrier task will be blocked until all the parallel branches complete and reach it.
version: 1.0 tasks: setup_task: # ... # Run tasks in parallel next: - do: - parallel_task_1 - parallel_task_2 - parallel_task_3 parallel_task_1: # ... # Wait to run barrier_task after this next: - do: - barrier_task parallel_task_2: # ... # Eventually run barrier_task next: - do: - intermediate_task intermediate_task: # ... # Wait to run barrier_task after this next: - do: - barrier_task barrier_task: # ... # Run after parallel_task_1, parallel_task_2, and intermediate_task have all finished join: all parallel_task_3: # ... # Run immediately after setup_task, do NOT wait for barrier_task
The following is the corresponding workflow execution graph.
=---- time (not to scale) ----> setup_task --+ | +------ parallel_task_1 --------------------------+ | | +-- parallel_task_2 --+ | | | | | +---- intermediate_task ----+ | | | +-- barrier_task --+ | | +-- parallel_task_3 -------------------------------------------------+ | +-- [finish]
Conversely, if more than one tasks transition to the same task and
join is not specified in
the latter, then the target task will be invoked immediately following the completion of the
previous task. There will be multiple instances of the target task. In the workflow graph, each
invocation of the target task will be its own branch with the inbound edge from the node of the
In other words, if
join: all was removed from the previous workflow, the
be run two different times, resulting in this execution graph:
=---- time (not to scale) ----> setup_task --+ | +------ parallel_task_1 ------+ | | | +-- barrier_task (1) ----------------------+ | | +-- parallel_task_2 --+ | | | | | +---- intermediate_task ----+ | | | | | +-- barrier_task (2) --+ | | +-- parallel_task_3 -----------------------------------------------------+ | +-- [finish]
With Items Model¶
with items section to process a list of items in a task. The task will iterate through
each item and request an action execution for each item. By default, all the items will be processed
at the same time. When
concurrency is specified, the number of items up to the concurrency value
will be processed and the remaining items will be queued. When the action execution for an item is
completed, the next item in the list will be processed.
The task result is a list of the action execution result in the same order as the items. All action executions must be completely successfully for the task to reach a succeeded state. If one ore more action executions abended, then the task will result in a failed state.
When there’s a request to cancel or pause the workflow, the task will be in a canceling or pausing state respectively until all action executions in the process of being executed are completed. Once these action executions are completed, the task will go to canceled or paused state respectively. If concurrency for the task is specified and there are remaining items, no new action executions will be requested. When a paused workflow resumes, the task will continue to process any remaining items.
|items||Yes||The list of items to execute the action with.|
|concurrency||No||The number of items being processed concurrently.|
The following is a simple example with a single list of items defined in a task. The task is given
a list of messages to echo. For an items list where no concurrency is required, there is a short
hand notation to pass just the list directly to the
with statement. The individual items can be
passed into the action as input for execution using the
version: 1.0 input: - messages tasks: task1: with: <% ctx(messages) %> action: core.echo message=<% item() %>
When concurrency is required, use the formal schema with
of the short hand notation for task definition.
version: 1.0 input: - messages tasks: task1: with: items: <% ctx(messages) %> concurrency: 2 action: core.echo message=<% item() %>
The item value can be named. The following example is the same workflow as the one above. Note
that the items are specified as
message in <% ctx(messages) %> where the value of the item
is named “message” and can be referenced with the
item function as
value returned from
item() in this case would be a dictionary like
The benefit is evident below when working with multiple lists of items.
version: 1.0 input: - messages tasks: task1: with: message in <% ctx(messages) %> action: core.echo message=<% item(message) %>
For multiple lists of items, the lists need to be zipped first with the
zip function and then
define the keys required to access the individual values in each item. In the example below, the
task will execute a specific command on a specific host. The hosts and commands are zipped via
<% zip(ctx(hosts), ctx(commands)) %> and then the keys to access the values in each item is
host, command in <% zip(ctx(hosts), ctx(commands)) %>. Finally, when specifying the
input parameters for the action execution, host value is accessed via
<% item(host) %> and the
command value is accessed via
<% item(command) %>.
version: 1.0 input: - hosts - commands tasks: task1: with: host, command in <% zip(ctx(hosts), ctx(commands)) %> action: core.remote hosts=<% item(host) %> cmd=<% item(command) %>
Task Transition Model¶
next section is a list of task transitions to be evaluated after a task completes. A task is
completed if it either succeeded, failed, or canceled. The list of transitions will be processed in
the order they are defined. In the workflow graph, each task transition is one or more outbound
edges from the current task node. For each task transition, the
when is the criteria that must
be met in order for transition. If
when is not defined, then the default criteria is task
completion. When criteria is met, then
publish can be defined to add new or update existing
variables from the result into the runtime workflow context. Finally, the list of tasks defined in
do will be invoked in the order they are specified.
|when||No||The criteria defined as an expression required for transition.|
|publish||No||A list of key value pairs to be published into the context.|
|do||No||A next set of tasks to invoke when transition criteria is met.|
The following is a more complex workflow with branches and join and various ways to define tasks and task transitions:
version: 1.0 description: Calculates (a + b) * (c + d) input: - a: 0 # Defaults to value of 0 if input is not provided. - b: 0 - c: 0 - d: 0 tasks: task1: # Fully qualified name (pack.name) for the action. action: math.add # Assign input arguments to the action from the context. input: operand1: <% ctx(a) %> operand2: <% ctx(b) %> # Specify what to run next after the task is completed. next: - # Specify the condition in YAQL or Jinja that is required # for this task to transition to the next set of tasks. when: <% succeeded() %> # Publish variables on task transition. This allows for # variables to be published based on the task state and # its result. publish: - msg: task1 done - ab: <% result() %> # List the tasks to run next. Each task will be invoked # sequentially. If more than one tasks transition to the # same task and a join is specified at the subsequent # task (i.e task1 and task2 transition to task3 in this # case), then the subsequent task becomes a barrier and # will be invoked when condition of prior tasks are met. do: - log - task3 task2: # Short hand is supported for input arguments. Arguments can be # delimited either by space, comma, or semicolon. action: math.add operand1=<% ctx("c") %> operand2=<% ctx("d") %> next: - when: <% succeeded() %> # Short hand is supported for publishing variables. Variables # can be delimited either by space, comma, or semicolon. publish: msg="task2 done", cd=<% result() %> # Short hand with comma delimited list is supported. do: log, task3 task3: # Join is specified for this task. This task will be invoked # when the condition of all inbound task transitions are met. join: all action: math.multiple operand1=<% ctx('ab') %> operand2=<% ctx('cd') %> next: - when: <% succeeded() %> publish: msg="task3 done" abcd=<% result() %> do: log # Define a reusable task to log progress. Although this task is # referenced by multiple tasks, since there is no join defined, # this task is not a barrier and will be invoked separately. log: action: core.log message=<% ctx(msg) %> output: - result: <% ctx().abcd %>
There are times when publish is required after a task completes but there are no more tasks
to execute next. In this case, a task transition can be defined without specifying the list
do. The following is a revision of the previous example:
version: 1.0 description: Calculates (a + b) * (c + d) input: - a: 0 # Defaults to value of 0 if input is not provided. - b: 0 - c: 0 - d: 0 tasks: task1: action: math.add operand1=<% ctx(a) %> operand2=<% ctx(b) %> next: - when: <% succeeded() %> publish: ab=<% result() %> do: task3 task2: action: math.add operand1=<% ctx("c") %> operand2=<% ctx("d") %> next: - when: <% succeeded() %> publish: cd=<% result() %> do: task3 task3: join: all action: math.multiple operand1=<% ctx('ab') %> operand2=<% ctx('cd') %> next: # After this task3 completes, it needs to publish the result # for output. Since there is no more tasks to execute afterward, # the do list is empty or not specified. - when: <% succeeded() %> publish: abcd=<% result() %> output: - result: <% ctx().abcd %>
The following example illustrates separate task transitions with different publishes on different condition:
version: 1.0 description: Send direct message to member input: - member - message tasks: task1: action: slack.post member=<% ctx(member) %> message=<% ctx(message) %> next: - when: <% succeeded() %> publish: msg="Successfully posted message." do: task2 - when: <% failed() %> publish: msg="Unable to post message due to error: <% result() %>" do: task2 task2: action: core.log message=<% ctx(msg) %>
The following is a list of engine commands with special meaning to the workflow engine.
When specified under
do in the task transition, the engine will act accordingly. These
commands are also reserved words that cannot be used for task name.
|noop||No operation or do not execute anything else.|
|fail||Fails the workflow execution.|
The following example illustrates the use of the
version: 1.0 description: > A workflow example that illustrates error handling. By default when any task fails, the notify_on_error task will be executed and the workflow will transition to the failed state. input: - cmd tasks: task1: action: core.local cmd=<% ctx(cmd) %> next: - when: <% succeeded() %> publish: stdout=<% result().stdout %> - when: <% failed() %> publish: stderr=<% result().stderr %> do: notify_on_error notify_on_error: action: core.echo message=<% ctx(stderr) %> next: # The fail specified here tells the workflow to go into # failed state on completion of the notify_on_error task. - do: fail output: - result: <% ctx(stdout) %>