Mistral + Jinja¶

Starting with StackStorm v2.2, Jinja expressions are also supported where YAQL expressions are accepted. Jinja expressions can be used for simple conditional evaluation and data transformation in Mistral workflows. There will be many cases where you did not author the actions but there’s a need to decide from the result of the action whether to continue or there’s a need to transform the result to another value or structure for the next action in the workflow.

Here are use cases where Jinja can be applied in Mistral workflows:

Define input values that are passed to tasks.
Define output values published from tasks and workflows.
Define conditions that determine transitions between tasks.

Knowing where Jinja can be applied in Mistral workflows, the following are some cool things that you can do with Jinja:

Access values from lists and dictionary.
Simple arithmetic.
Evaluation of boolean logic.
Use conditional logic and builtin filters to evaluate and transform data.

Note

Please refer to the official documentation for Mistral and Jinja. The documentation here is meant to help StackStorm users get started quickly, but is not a complete reference.

Basics¶

The following are statements in the workflow and task definition that accepts Jinja:

task action input
task concurrency
task on-complete
task on-error
task on-success
task pause-before
task publish
task retry break-on
task retry continue-on
task retry count
task retry delay
task timeout
task wait-before
task wait-after
task with-items
workflow output

Each of the statements can take a string with one or more Jinja expressions. Each expression in the string should be encapsulated with {{ }}. Code block using {% %} is also supported. Please note that the symbols { and } will conflict with JSON and Jinja expressions, and must always be encapsulated with quotes or double quotes in the workflow definition.

Note

Mixing of both YAQL and Jinja expressions in a single statement is not supported.

When evaluating a Jinja expression, Mistral also passes a JSON dictionary (aka context) to the Jinja templating engine. The context contains all the workflow inputs, published variables, and result of completed tasks up to this point of workflow execution, including the current task. The Jinja expression can refer to one or more variables in the context. The reserved symbol _ is used to reference the context. For example, given the context {"ip": "127.0.0.1", "port": 8080}, the string https://{{ _.ip }}:{{ _.port }}/api returns https://127.0.0.1:8080/api.

The following is the same example used in a workflow:

version: '2.0'

examples.yaql-basic:
    type: direct
    input:
        - ip
        - port
    tasks:
        task1:
            action: examples.call-api
            input:
                endpoint: https://{{ _.ip }}:{{ _.port }}/api

Note

The reserved symbol to reference the workflow context for Jinja is different than for YAQL expressions. This is due to the differences between the two engines. As mentioned above, the symbol _ is used to access context in Jinja whereas the symbol $ is used in YAQL.

The following is a more complex workbook example with a few more Jinja expressions. There are variables passed to input parameters and being published after task completion. Please take note of the install_apps task in the configure_vm workflow. The input parameter cmd is given the value formatted by the Jinja for loop. Unlike YAQL, a string in a Jinja expression must be explicitly encapsulated in quotes (i.e. {{ 'this is a string.' }}).

version: "2.0"
name: examples.mistral-jinja-workbook-complex
description: >
    A sample workflow that demonstrates nested workflows,
    forks, join, and use of jinja expressions.

workflows:

    main:
        type: direct
        input:
            - vm_name
            - cpu_cores
            - memory_mb
        output:
            vm_id: '{{ _.vm_id }}'
            ip: '{{ _.ip }}'
        tasks:
            register_dns:
                action: core.local
                input:
                    cmd: "sleep 1; printf 'Registering {{ _.vm_name }}...'"
                publish:
                    ip: "10.1.23.98"
                    status_message: "DNS for {{ _.vm_name }} is registered."
                on-success:
                    - configure_vm
                    - notify
            create_vm:
                wait-before: 1
                workflow: create_vm
                input:
                    name: '{{ _.vm_name }}'
                    cpu_cores: '{{ _.cpu_cores }}'
                    memory_mb: '{{ _.memory_mb }}'
                publish:
                    vm_id: "{{ task('create_vm').result.vm_id }}"
                    status_message: "VM {{ _.vm_name }} is created."
                on-success:
                    - configure_vm
                    - notify
            configure_vm:
                join: all
                workflow: configure_vm
                input:
                    vm_id: '{{ _.vm_id }}'
                    ip: '{{ _.ip }}'
                publish:
                    status_message: "VM {{ _.vm_name }} is reconfigured."
                on-success:
                    - close_request
                    - notify
            close_request:
                action: std.noop
                publish:
                    status_message: "VM request is fulfilled."
                on-success:
                    - notify
            notify:
                action: core.local
                input:
                    cmd: "printf '{{ _.status_message }}'"

    create_vm:
        type: direct
        input:
            - name
            - cpu_cores
            - memory_mb
        output:
            vm_id: '{{ _.vm_id }}'
        tasks:
            create:
                action: core.local
                input:
                    cmd: "printf 'vm6789'; sleep 5"
                publish:
                    vm_id: "{{ task('create').result.stdout }}"

    configure_vm:
        type: direct
        input:
            - vm_id
            - ip
        vars:
            apps:
                - 'super mario brothers'
                - 'zelda'
                - 'donkey kong'
        tasks:
            add_disks:
                action: core.local
                input:
                    cmd: "sleep 1; printf 'disks created'"
            add_nics:
                action: core.local
                input:
                    cmd: "sleep 1; printf 'nics created'"
            install_apps:
                action: core.local
                input:
                    cmd: "sleep 1; {% for app in _.apps %}printf '{{ app }} is installed.\n'; {% endfor %}"

Certain statements in Mistral such as on-success and on-error can evaluate boolean logic. The on-condition related statements are used for transition from one task to another. If a boolean logic is defined with these statements, it can be used to evaluate whether the transition should continue or not. Complex boolean logic using a combination of not, and, or, and parentheses is possible.

Take the following workflow as an example: Execution of a certain branch in the workflow depends on the value of _.path. If _.path == a, then task a is executed. If _.path == b, then task b. Finally task c is executed if neither matches.

version: '2.0'

examples.mistral-jinja-branching:
    description: >
        A sample workflow that demonstrates how to use conditions
        to determine which path in the workflow to take.
    type: direct
    input:
        - which
    tasks:
        t1:
            action: core.local
            input:
                cmd: "echo {{ _.which }}"
            publish:
                path: "{{ task('t1').result.stdout }}"
            on-success:
                - a: "{{ _.path == 'a' }}"
                - b: "{{ _.path == 'b' }}"
                - c: "{{ not _.path in ['a', 'b'] }}"
        a:
            action: core.local
            input:
                cmd: "echo 'Took path A.'"
        b:
            action: core.local
            input:
                cmd: "echo 'Took path B.'"
        c:
            action: core.local
            input:
                cmd: "echo 'Took path C.'"

The statement with-items in Mistral is used to execute an action over iteration of one or more lists of items. The following is a sample Mistral workflow that iterates over the list of given names to invoke the action to create an individual VM:

version: '2.0'

examples.create-vms:
    type: direct
    input:
        - names
    tasks:
        task1:
            with-items: "name in {{ _.names }}"
            action: examples.create-vm
            input:
                name: "{{ _.name }}"

with-items can take more than one list as the following example illustrates. In this case, a list of VMs and IP addresses are passed as inputs and then iterated through step by step together:

version: '2.0'

examples.create-vms:
    type: direct
    input:
        - names
        - ips
    tasks:
        task1:
            with-items:
                - "name in {{ _.names }}"
                - "ip in {{ _.ips }}"
            action: examples.create-vm
            input:
                name: "{{ _.name }}"
                ip: "{{ _.ip }}"

Note

The Jinja expression(s) passed to with-items is slightly different than YAQL expression(s). If using Jinja, the entire statement (i.e. "name in {{ _.names }}") require quotation because the symbols { and } for the delimiters conflict with JSON. Whereas if using YAQL, the statement does not necessarily require quotation.

Jinja Filters¶

Jinja has a list of built-in filters to work with strings, dictionaries, lists, etc. Please refer to the Jinja documentation for the list of available filters.

A number of Mistral and StackStorm specific custom functions (aka filters in Jinja) such as st2kv, task, env, and execution that are available in YAQL are also made available in Jinja.

Mistral¶

env() returns the environment variables passed to the workflow execution on invocation such as the StackStorm Action Execution ID st2_execution_id.

For example, the expression {{ env().st2_action_api_url }}/actionexecutions/{{ env().st2_execution_id }} returns the API endpoint for the current workflow execution in StackStorm as something like https://127.0.0.1:9101/v1/actionexecutions/874d3d5b3f024c1aa93225ef0bcfcf3a.
To access information about the parent action, the following expressions can be used {{ env()['__actions']['st2.action']['st2_context']['parent']['api_user'] }}, {{ env()['__actions']['st2.action']['st2_context']['parent']['source_channel'] }} or {{ env()['__actions']['st2.action']['st2_context']['parent']['user'] }}.

Note that this similar to the ActionChain expressions {{action_context.parent.source_channel}}, {{action_context.parent.user}} or {{action_context.parent.api_user}}.
task('task_name') returns the state, state_info, and the result of the given task_name.

StackStorm¶

st2kv('st2_key_id') queries StackStorm’s datastore and returns the value for the given key. For example, the expression {{ st2kv('system.shared_key_x') }} returns the value for a system scoped key named shared_key_x while the expression {{ st2kv('my_key_y') }} returns the value for the user scoped key named my_key_y.

Please note that the key name should be in quotes otherwise Jinja treats a key name with a dot like system.shared_key_x as a dict access.

Note

If the retrieved value was stored encrypted, st2kv no longer attempts decryption by default (as of version 2.4). To decrypt the retrieved value, you must explicitly enable it through the decrypt parameter: st2kv('st2_key_id', decrypt=true).

Testing Expressions¶

Sometimes Jinja expressions can become complex and the need to verify the expression outside of StackStorm is necessary. To accomplish this there are a few options:

Jinja2 online evaluator

Write a small test script:

#!/usr/bin/env python
import jinja2
import os


class JinjaUtils:

    @staticmethod
    def render_file(filename, context):
        path, filename = os.path.split(filename)
        env = jinja2.Environment(loader=jinja2.FileSystemLoader(path or './'))
        tmpl = env.get_template(filename)
        return tmpl.render(context)

    @staticmethod
    def render_str(jinja_template_str, context):
        env = jinja2.Environment()
        tmpl = env.from_string(jinja_template_str)
        return tmpl.render(context)


if __name__ == "__main__":
    context = {'results': {'name': 'Stanley'}}
    template = "Hello {{ results.name }}"
    print JinjaUtils.render_str(template, context)

Note

The test script does NOT include the custom StackStorm Jinja filters or functions such as st2kv.

More Examples¶

More workflow examples using Jinja expressions can be found at /usr/share/doc/st2/examples. The examples are prefixed with mistral-jinja.