SysGenius-e-Log: Ansible

I spent several hours last week-end doing some research and putting together an Ansible callback plugin that posts messages to Microsoft Teams when specific event(s) occurs in Ansible playbook. I could not find a real good documentation or example to follow. Don't get me wrong, yes, there are documentation/blog for Slack or some even related to sending messages to Teams, but not the way, I wanted. I wanted to send custom messages using Office 365 connector card written in Jinja2 template, which could be customized using the value(s) of extra-vars, goup_vars/host_vars for both success and failure events.

Finally, I've put together a fully functional callback plugin and wanted to share it with the community, so that people will not have to pull out their hair for the same. The plugin source code can be found in the GitHub (see the links below), but here I'm explaining the details.

Why callback plugin? You don't really need to use callback plugin to post a message to any end point from Ansible. Ansible even has an Office 365 connector card plugin. But the problem starts when you want to capture the failure event and post message accordingly. From Ansible playbook it's not easy to capture the failure event, unless you put your entire playbook in 'try...' block or put each task in 'try..' block because you can't possibly know which task will fail in next run. With callback plugin, it becomes easy, as the corresponding method is automatically invoked by a particular event in Ansible playbook.

Why Jinja2 template? It adds flexibility in creating custom messages. Also, it helps to make your callback plugin more universal, as message customization is externalized to Jinja2 template. To demonstrate all this, I've included a simple playbook that fakes the application deployment and posts success or failure deployment messages accordingly.

Now, let's dive little into the details, starting with callback plugins. Ansible documentation describes callback plugins as "Callback plugins enable adding new behaviors to Ansible when responding to events...". Refer to the Callback Plugins page for general information. In this blog post, I'm not going to explain how to develop your own plugin, but only provide specific information on how this msteam plugin has been developed. If you have not previously written Ansible plugin, I'd suggest looking into Developing plugins section of the Ansible documentation for general guidelines.

Class Import section:


from __future__ import (absolute_import, division, print_function)

from ansible.plugins.callback import CallbackBase

from jinja2 import Template
...

The 1st line above is required for any plugin and 2nd line is required for Callback plugins. 3rd line above is to work the Jinja2 template.

Class body section:

As you can see in the lines below, I'm creating msteam with the CallbackModule(CallbackBase) as parent, that means, methods defined in the parent class are available to override. Refer to _init__.py to see what methods are available. For msteam plugin, I've overriden only specific version 2.0 methods as the intention is to use it with Ansible version 2.0 or later. Note: CallbackBase class defines regular as well as corresponding 'v2_*' methods.


class CallbackModule(CallbackBase):

    CALLBACK_VERSION = 2.0

    CALLBACK_TYPE = 'notification'

    CALLBACK_NAME = 'msteam'

    CALLBACK_NEEDS_WHITELIST = True

__init__ section:

See the comment in the code for details.


self.playbook_name = None

# Record the playbook start time. In this case I'm using Canada/Eastern

self.tz = timezone('Canada/Eastern')

self.dt_format = "%Y-%m-%d %H:%M:%S"

self.start_time = datetime.datetime.now(self.tz)

# Placeholder for extra-vars variable

self.extra_vars = None


# If you are executing your playbook from AWX/Tower

# Replace with your Ansible Tower/AWX base url

#self.v_at_base_url = "https://<Ansible tower host>:<port>


# To record whether the playbook variables are retrieved, so that we retrieve them just once.

self.pb_vars_retrieved = False


# Here you can assign your default MS Teams webhook url

self.v_msteam_channel_url = "<replace with your own MS Team webhook URL>"


# default MS Teams message card template. Here I'm assigning the one included in the example playbook

self.v_message_template = "templates/msteam_default_msg.json.j2"



# default job status in the beginning

self.job_status = "successful"



# If you need to post through proxies, uncomment the following and replace with your proxy URLs.

# self.proxies = {

#     "http": "<http-proxy-url>",

#     "https": "<https-proxy-url>",

# }

v2_playbook_on_start:


def v2_playbook_on_start(self, playbook):

    display.vvv(u"v2_playbook_on_start method is being called")

    self.playbook = playbook

    self.playbook_name = playbook._file_name

v2_playbook_on_play_start:


def v2_playbook_on_play_start(self, play):

            display.vvv(u"v2_playbook_on_play_start method is being called")

            self.play = play

            # get variable manager and retrieve extra-vars

            vm = play.get_variable_manager()

            self.extra_vars = vm.extra_vars

            self.play_vars = vm.get_vars(self.play)

            # The following is used to retrieve variables defined under group_vars or host_vars.

            # If the same variable is defined under both with the same scope, 
            # the one defined under host_vars takes precedence.

            self.host_vars = vm.get_vars()['hostvars']

            if not self.pb_vars_retrieved:

                self.get_pb_vars()

As you have noticed above, you have to obtain the variable manager from play to get the extra-vars object and use 'get_vars' to get the general playbook variables, and get_vars()['hostvars'] to get the group_vars/host_vars. Refer to the get_pb_vars() method to see how I have obtained the extra-vars, and playbook variables.

v2_playbook_on_stats:


def v2_playbook_on_stats(self, stats):

    		display.vvv(u"v2_playbook_on_stats method is being called")

    		if not self.pb_vars_retrieved:
          self.get_pb_vars()

    		hosts = sorted(stats.processed.keys())

    		self.hosts = hosts

    		self.summary = {}

    		self.end_time = datetime.datetime.now(self.tz)

    		self.duration_time = int((self.end_time - self.start_time).total_seconds())

    		# Iterate trough all hosts to check for failures

    		for host in hosts:
          summary = stats.summarize(host)
          self.summary = summary
          if summary['failures'] > 0:
              self.job_status = "failed"

    
          if summary['unreachable'] > 0:
              self.job_status = "failed"

    
          display.vvv(u"summary for host %s :" % host)
          display.vvv(str(summary))

    

    				# Add code here if you want to post to MS Teams per host

    

    		# Just send a single notification whether it is a failure or success

    		# Post message to MS Teams

    		if(not self.disable_msteam_post):
          self.notify_msteam()

    		else:
          display.vvv(u"Posting to MS Team has been disabled.")

As you have noticed above, I'm calling notify_msteam() method to post to MS Teams. I'm posting a single message at the end of the playbook execution. However, if you like to post for each host, see how to do that in the code (you have to call the notify_msteam() within the 'for' loop).

notify_msteam:

I'm not going to post the entire code here, you can see it in the GitHub repository. Here are few important lines. The basic idea here is first to load the Jinja2 template from the given file, then render the template with values retrieved from extra-vars, playbook variable and group_vars/host_vars and finally post the message (see the commented section if you are using the proxy)

try:

    		with open(self.v_message_template) as j2_file:

    				template_obj = Template(j2_file.read())

except Exception as e:

    		print("ERROR: Exception occurred while reading MS Teams message template %s. Exiting... %s" % (

    				self.v_message_template, str(e)))

    		sys.exit(1)

    

rendered_template = template_obj.render(

    		v_ansible_job_status=self.job_status,

    		v_ansible_job_id=self.tower_job_id,

    		v_ansible_scm_revision=self.scm_revision,

    		v_ansible_job_name=self.tower_job_template_name,

    		v_ansible_job_started=self.start_time.strftime(self.dt_format),

    		v_ansible_job_finished=self.end_time.strftime(self.dt_format),

    		v_ansible_job_elapsed_time=self.duration_time,

    		v_ansible_host_list=self.hosts,

    		v_ansible_web_url=web_url,

    		v_ansible_app_file=self.v_app_file,

    		v_ansible_deployment_action=self.v_deployment_action,

    		v_ansible_environment=self.v_environment,

    		v_ansible_instance_name=self.v_instance_name,

    		v_ansible_executed_from_tower=self.executed_from_tower

)



try:

    		with SpooledTemporaryFile(max_size=0, mode='r+w') as tmpfile:
          tmpfile.write(rendered_template)
          tmpfile.seek(0)
          json_payload = json.load(tmpfile)
          display.vvv(json.dumps(json_payload))

except Exception as e:

    		print("ERROR: Exception occurred while reading rendered template or writing rendered MS Teams message template. Exiting... %s" % str(e))

    		sys.exit(1)

    

try:
 
    		# using proxy

    		# response = requests.post(url=self.v_msteam_channel_url,

    		#                          data=json.dumps(json_payload), headers={'Content-Type': 'application/json'}, timeout=10, proxies=self.proxies)

    

    		# without proxy

    		response = requests.post(url=self.v_msteam_channel_url,

    														 data=json.dumps(json_payload), headers={'Content-Type': 'application/json'}, timeout=10)

    

    		if response.status_code != 200:
          raise ValueError('Request to msteam returned an error %s, the response is:\n%s' % (

    						response.status_code, response.text))

except Exception as e:

    		print(

    				"WARN: Exception occurred while sending notification to MS Teams. %s" % str(e))

Message card as Jinja2 template:

{
    "@type": "MessageCard",
    "@context": "http://schema.org/extensions",
    "themeColor": "{{ '#008000' if(v_ansible_job_status != 'failed') else '#FF0000' }}",
    "title": "Deployment of {{v_ansible_app_file}} on {{ v_ansible_environment }} environment {{'completed successfully' if(v_ansible_job_status == 'successful') else 'failed.' }}",
    "summary": "Ansible Job Summary",
    "sections": [{
        "activityTitle": "Job {{ v_ansible_job_id }} summary: ",
        "facts": [
        {% if v_ansible_executed_from_tower is sameas true %}
        {
            "name": "Playbook revision",
            "value": "{{ v_ansible_scm_revision }}"
        }, {
            "name": "Job name",
            "value": "{{ v_ansible_job_name }}"
        },
        {% endif %}
        {
            "name": "Job status",
            "value": "{{ v_ansible_job_status }}"
        }, {
            "name": "Job started at",
            "value": "{{ v_ansible_job_started }}"
        }, {
            "name": "Job finished at",
            "value": "{{ v_ansible_job_finished }}"
        }, {
            "name": "Job elapsed time (sec)",
            "value": "{{ v_ansible_job_elapsed_time }}"
        }, {
            "name": "Application (v_app_file)",
            "value": "{{ v_ansible_app_file }}"
        }, {
            "name": "Action (v_deployment_action)",
            "value": "{{ v_ansible_deployment_action }}"
        }, {
            "name": "Environment (v_environment)",
            "value": "{{ v_ansible_environment }}"
        }, {
            "name": "Hosts",
            "value": "{{ v_ansible_host_list | join(',') }}"
        },{
            "name": "Instance name(v_instance_name)",
            "value": "{{ v_ansible_instance_name | default('na') }}"
        }],
        "markdown": false
    }]
    {% if v_ansible_executed_from_tower is sameas true %}
    ,"potentialAction": [{
        "@context": "http://schema.org",
        "@type": "ViewAction",
        "name": "View on Ansible Tower",
        "target": [
            "{{ v_ansible_web_url }}"
        ]        
    }]
  {% endif %}
}

Note: In the example above, it adds the "View on Ansible Tower" button if the playbook is executed from Ansible Tower/AWX as shown below.