ChatOps Troubleshooting Guide
This guide will help you troubleshoot
st2chatops, up to the point where Hubot should work,
reply to your commands and send manually triggered messages via
The recommended way to install
st2chatops is using the steps mentioned
in Installation Section for your OS.
Troubleshooting Using Hubot Self-check Script:
We have a self-check script to help you debug ChatOps issues.
Just copy the script and run it on your server. It will run a few essential tests that will provide you basic troubleshooting steps in case of a failure.
Manual Troubleshooting Steps:
These are the issues that users usually face with ChatOps:
Your bot is not Online:
You have installed StackStorm, however Hubot did not come online in your ChatOps client (Slack, Mattermost, etc.)
Incorrect adapter settings. Check the settings in
/opt/stackstorm/chatops/st2chatops.envto verify if correct. For example, for the Slack adapter, we need to uncomment the following lines and update the Slack token:
# Slack settings (https://github.com/slackhq/hubot-slack): # export HUBOT_ADAPTER=slack # export HUBOT_SLACK_TOKEN=xoxb-CHANGE-ME-PLEASE
After changing the adapter settings don’t forget to restart the service using this command:
$ sudo service st2chatops restart
Make sure the login credentials and
After you made sure
st2chatops.envsettings are correct, check if the service is running:
$ service st2chatops status
In case the installation is outdated or became corrupted, reinstall the
st2chatops package with
yum depending on your distro.
Note that in most chat services you have to manually invite the bot into your chatroom first: for example, in Slack.
There are no StackStorm Commands in
Hubot is online and present in your room, however when you say
!help, there are no commands listed except
!help itself. (The
st2 sets of commands should be installed by default).
Hubot can not connect to StackStorm API. Look in the st2chatops logs for errors:
/var/log/st2/st2chatops.logor for systemd based distros
Actions and/or ChatOps aliases aren’t registered. Try running
st2ctl reload --register-all.
It is also possible you may have changed the bot’s name or you have not invited your bot to a channel in the chat client.
If you are sending the bot a private message instead of messaging it in a channel, do not prepend
!(a known Hubot limitation). In private messages,
helpwill work, however
!helpwill not; in channels it works the other way around.
StackStorm Commands Throw Errors:
Hubot is online, you can see
st2 commands, but entering something like
!st2 list actions
either throws an error, or gives an acknowledgement message without result, or no response at all.
- No response at all:
Usually the command you’re trying to run has a typo, or the syntax is incorrect. Double-check it, and glance over the alias yaml definition to check what the command syntax is exactly.
- Throws an error (no acknowledgement):
Should be debugged according to the error, could be both client-side and server-side, or an alias problem. Normally, a look into an error or logs will provide you ideas on how to proceed.
- Gives an acknowledgement message without result:
If you get an acknowledgement message, but then nothing happens, Hubot is most likely disconnected from the StackStorm stream. The main reason for it is a wrong networking setup (Hubot can’t connect to your StackStorm instance); check nginx configuration and the parameters in
ST2_HOSTNAME). Another reason is that either the StackStorm action you’re trying to launch or your alias fails with an unexpected error that the bot can’t process. This can be checked in StackStorm execution history through CLI or Web UI.
- Result message is delayed:
Seeing really long delays with your result messages? Check that all services are running correctly, especially
- Gives an acknowledgement message, then an error:
If the default commands (like
!st2 list actions) run fine, but your own aliases throw errors, the format of your alias or the underlying action is most likely the problem. Debug according to the error.
- Bonus: have you tried turning StackStorm off and on again?
sudo st2ctl restartor
sudo st2ctl reload --register-allsometimes seem to magically fix problems, often quite unexpectedly. Restarting just the
st2chatopsservice also works sometimes:
sudo service st2chatops restart.
If the problem persists, there’s likely a back-end problem. Make sure other parts of StackStorm are working properly. Try Step 6 and Step 7 of the self-check script :
StackStorm Commands are Fine but no Manual Messages:
You can run StackStorm commands (and your own aliases) via your bot,
but you can’t trigger
chatops.post_message action manually from CLI or Web UI.
Some of your action parameters (route, channel, etc) are incorrect. Take a look at
chatops.post_resultworkflow execution from any chat command you issued before, and repeat every parameter in
chatops.post_message(the last step of the workflow) as is.
st2 run chatops.post_message channel=<channel_name>to post on a channel. This step assumes that a bot was created and is invited it to the channel on ChatOps application.
st2 run chatops.post_message channel=<username> whisper=Trueto post to a user. Note that some chat services have limitations on private messages from bots to users (e.g. in Slack a bot can’t initiate a new conversation).
By now you should have your bot up and running. If not, then just ask for help!