Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective

What is it about?

Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures.

Featured Image

Photo by Luca Bravo on Unsplash

Photo by Luca Bravo on Unsplash

Why is it important?

We have conducted an empirical study to assess the executability of publicly available Ansible playbooks. From our empirical study, we have found that 28.5% of 2,952 Ansible playbooks can be executed without generating failures. We have also observed that using four fix strategies derived from crowdsourced knowledge sources, 82.2% of the playbooks can be executed. Based on our findings, we recommend practitioners to use our identified set of fix strategies as heuristics to resolve playbook-related failures.

Perspectives