Sensors and Triggers

Sensors

Sensors are essentially adapters that are a way to integrate StackStorm with an external system so that triggers can be injected into StackStorm before rule matching results in potential actions. Sensors are pieces of Python code and have to follow the StackStorm defined sensor interface requirements to run successfully.

Triggers

Triggers are StackStorm constructs that identify the incoming events to StackStorm. A trigger is a tuple of type (string) and optional parameters (object). Rules are written to work with triggers. Sensors typically register triggers though this is not strictly the case. For example, webhook triggers are just registered independently. You don’t have to write a sensor.

Internal Triggers

By default StackStorm emits some internal triggers which you can leverage in the rules. Those triggers can be distinguished by non-system triggers since they are prefixed with st2..

A list of available triggers for each resource is included below:

Action

Reference Description Properties
st2.generic.actiontrigger Trigger encapsulating the completion of an action execution. status, start_timestamp, result, parameters, action_ref, runner_ref, execution_id, action_name
st2.generic.notifytrigger Notification trigger. status, data, start_timestamp, channel, route, action_ref, message, runner_ref, execution_id, end_timestamp
st2.action.file_writen Trigger encapsulating action file being written on disk. host_info, ref, file_path

Key Value Pair

Reference Description Properties
st2.key_value_pair.create Trigger encapsulating datastore item creation. object
st2.key_value_pair.update Trigger encapsulating datastore set action. object
st2.key_value_pair.value_change Trigger encapsulating a change of datastore item value. new_object, old_object
st2.key_value_pair.delete Trigger encapsulating datastore item deletion. object

Sensor

Reference Description Properties
st2.sensor.process_spawn Trigger indicating sensor process is started up. object
st2.sensor.process_exit Trigger indicating sensor process is stopped. object

Creating a Sensor

Creating a sensor involves writing a Python file and a YAML meta file that defines the sensor. An example meta file is shown below:

---
  class_name: "SampleSensor"
  entry_point: "sample_sensor.py"
  description: "Sample sensor that emits triggers."
  trigger_types:
    -
      name: "event"
      description: "An example trigger."
      payload_schema:
        type: "object"
        properties:
          executed_at:
            type: "string"
            format: "date-time"
            default: "2014-07-30 05:04:24.578325"

The corresponding simple sensor Python implementation is shown below.

from st2reactor.sensor.base import Sensor


class SampleSensor(Sensor):
    """
    * self.sensor_service
        - provides utilities like
            - get_logger() - returns logger instance specific to this sensor.
            - dispatch() for dispatching triggers into the system.
    * self._config
        - contains parsed configuration that was specified as
          config.yaml in the pack.
    """

    def setup(self):
        # Setup stuff goes here. For example, you might establish connections
        # to external system once and reuse it. This is called only once by the system.
        pass

    def run(self):
        # This is where the crux of the sensor work goes.
        # This is called once by the system.
        # (If you want to sleep for regular intervals and keep
        # interacting with your external system, you'd inherit from PollingSensor.)
        # For example, let's consider a simple flask app. You'd run the flask app here.
        # You can dispatch triggers using sensor_service like so:
        # self.sensor_service(trigger, payload, trace_tag)
        #   # You can refer to the trigger as dict
        #   # { "name": ${trigger_name}, "pack": ${trigger_pack} }
        #   # or just simply by reference as string.
        #   # i.e. dispatch(${trigger_pack}.${trigger_name}, payload)
        #   # E.g.: dispatch('examples.foo_sensor', {'k1': 'stuff', 'k2': 'foo'})
        #   # trace_tag is a tag you would like to associate with the dispacthed TriggerInstance
        #   # Typically the trace_tag is unique and a reference to an external event.
        pass

    def cleanup(self):
        # This is called when the st2 system goes down. You can perform cleanup operations like
        # closing the connections to external system here.
        pass

    def add_trigger(self, trigger):
        # This method is called when trigger is created
        pass

    def update_trigger(self, trigger):
        # This method is called when trigger is updated
        pass

    def remove_trigger(self, trigger):
        # This method is called when trigger is deleted
        pass

It shows a bare minimum version of how a sensor would look like. Your sensor should generate triggers of the form (Python dict):

trigger = 'pack.name'
payload = {
    'executed_at': '2014-08-01T00:00:00.000000Z'
}
trace_tag = external_event_id

The sensor would inject such triggers by using the sensor_service passed into the sensor on instantiation.

self.sensor_service.dispatch(trigger=trigger, payload=payload, trace_tag=trace_tag)

If you want a sensor that polls an external system at regular intervals, you would use a PollingSensor instead of Sensor as the base class.

from st2reactor.sensor.base import PollingSensor


class SamplePollingSensor(PollingSensor):
    """
    * self.sensor_service
        - provides utilities like
            get_logger() for writing to logs.
            dispatch() for dispatching triggers into the system.
    * self._config
        - contains configuration that was specified as
          config.yaml in the pack.
    * self._poll_interval
        - indicates the interval between two successive poll() calls.
    """

    def setup(self):
        # Setup stuff goes here. For example, you might establish connections
        # to external system once and reuse it. This is called only once by the system.
        pass

    def poll(self):
        # This is where the crux of the sensor work goes.
        # This is called every self._poll_interval.
        # For example, let's assume you want to query ec2 and get
        # health information about your instances:
        #   some_data = aws_client.get('')
        #   payload = self._to_payload(some_data)
        #   # _to_triggers is something you'd write to convert the data format you have
        #   # into a standard python dictionary. This should follow the payload schema
        #   # registered for the trigger.
        #   self.sensor_service.dispatch(trigger, payload)
        #   # You can refer to the trigger as dict
        #   # { "name": ${trigger_name}, "pack": ${trigger_pack} }
        #   # or just simply by reference as string.
        #   # i.e. dispatch(${trigger_pack}.${trigger_name}, payload)
        #   # E.g.: dispatch('examples.foo_sensor', {'k1': 'stuff', 'k2': 'foo'})
        #   # trace_tag is a tag you would like to associate with the dispacthed TriggerInstance
        #   # Typically the trace_tag is unique and a reference to an external event.
        pass

    def cleanup(self):
        # This is called when the st2 system goes down. You can perform cleanup operations like
        # closing the connections to external system here.
        pass

    def add_trigger(self, trigger):
        # This method is called when trigger is created
        pass

    def update_trigger(self, trigger):
        # This method is called when trigger is updated
        pass

    def remove_trigger(self, trigger):
        # This method is called when trigger is deleted
        pass

For a complete implementation of a sensor that actually injects triggers into the system, look at the examples section.

Sensor Service

As you can see in the example above, a sensor_service is passed to each sensor class constructor on instantiation.

The Sensor service provides different services to the sensor via public methods. The most important one is the dispatch method which allows sensors to inject triggers into the system.

All public methods are described below.

Common Operations

1. dispatch(trigger, payload, trace_tag)

This method allows the sensor to inject triggers into the system.

For example:

trigger = 'pack.name'
payload = {
    'executed_at': '2014-08-01T00:00:00.000000Z'
}
trace_tag = uuid.uuid4().hex

self.sensor_service.dispatch(trigger=trigger, payload=payload, trace_tag=trace_tag)

2. get_logger(name)

This method allows the sensor instance to retrieve the logger instance which is specific to that sensor.

For example:

self._logger = self.sensor_service.get_logger(name=self.__class__.__name__)
self._logger.debug('Polling 3rd party system for information')

Datastore Management Operations

In addition to the trigger injection, the sensor service also provides functionality for reading and manipulating the datastore.

Each sensor has a namespace which is local to it and by default, all the datastore operations operate on the keys in that sensor-local namespace. If you want to operate on a global namespace, you need to pass local=False argument to the datastore manipulation method.

Among other reasons, this functionality is useful if you want to persist temporary data between sensor runs.

A good example of this functionality in action is TwitterSensor. The Twitter sensor persists the ID of the last processed tweet after every poll in the datastore. This way if the sensor is restarted or if it crashes, the sensor can resume from where it left off without injecting duplicate triggers into the system.

For the implementation, see twitter_search_sensor.py in StackStorm Exchange

1. list_values(local=True, prefix=None)

This method allows you to list the values in the datastore. You can also filter by key name prefix (key name starts with) by passing prefix argument to the method.

For example:

kvps = self.sensor_service.list_values(local=False, prefix='cmdb.')

for kvp in kvps:
    print(kvp.name)
    print(kvp.value)

2. get_value(name, local=True)

This method allows you to retrieve a single value from the datastore.

For example:

kvp = self.sensor_service.get_value('cmdb.api_host')
print(kvp.name)

3. set_value(name, value, ttl=None, local=True)

This method allows you to store (set) a value in the datastore. Optionally you can also specify time to live (TTL) for the stored value.

last_id = 12345
self.sensor_service.set_value(name='last_id', value=str(last_id))

4. delete_value(name, local=True)

This method allows you to delete an existing value from a datastore. If a value is not found this method will return False, True otherwise.

self.sensor_service.delete_value(name='my_key')

API Docs

class st2reactor.container.sensor_wrapper.SensorService(sensor_wrapper)[source]

Instance of this class is passed to the sensor instance and exposes “public” methods which can be called by the sensor.

dispatch(trigger, payload=None, trace_tag=None)[source]

Method which dispatches the trigger.

Parameters:
  • trigger (str) – Full name / reference of the trigger.
  • payload (dict) – Trigger payload.
  • trace_tag – Tracer to track the triggerinstance.
dispatch_with_context(trigger, payload=None, trace_context=None)[source]

Method which dispatches the trigger.

Parameters:
  • trigger (str) – Full name / reference of the trigger.
  • payload (dict) – Trigger payload.
  • trace_context (st2common.api.models.api.trace.TraceContext) – Trace context to associate with Trigger.
get_logger(name)[source]

Retrieve an instance of a logger to be used by the sensor class.

Running Your First Sensor

Once you write your own sensor, the following steps can be used to run your sensor for the first time:

1. Place the sensor Python file and YAML metadata in the ‘default’ pack in /opt/stackstorm/packs/default/sensors/. Alternatively, you can create a custom pack in /opt/stackstorm/packs/ with appropriate pack structure (see Create and Contribute a Pack) and place the sensor artifacts there.

  1. Register the sensor by using the st2ctl tool. Look out for any errors in sensor registration.
st2ctl reload

If there are errors in registration, fix the errors and re-register them using st2ctl reload.

  1. If registration is successful, the sensor will run automatically.

Once you like your sensor, you can promote it to a pack (if required) by creating a pack in /opt/stackstorm/packs/${pack_name} and moving the sensor artifacts (YAML and Python) to /opt/stackstorm/packs/${pack_name}/sensors/. See Create and Contribute a Pack for how to create a pack.

Debugging a Sensor From a Pack

If you just want to run a single sensor from a pack and the sensor is already registered, you can use the st2sensorcontainer to run just that single sensor.

/opt/stackstorm/st2/bin/st2sensorcontainer --config-file=/etc/st2/st2.conf --sensor-ref=pack.SensorClassName

For example:

/opt/stackstorm/st2/bin/st2sensorcontainer --config-file=/etc/st2/st2.conf --sensor-ref=git.GitCommitSensor

Examples

For more examples, please reference packs in StackStorm Exchange.