This page looks best with JavaScript enabled

[My Ansible Journey] Collections

 ·  ☕ 5 min read  ·  🦆 Jeremy

Part 1 : Presentation and prerequisites
Part 2 : Basics and first Playbook
Part 3 : Variables
Part 4 : Collections
––

Here is the fourth part of the series and it is about Collections! A new way to deal with modules, roles and plugins. It is a recent feature which is part of a larger change in Ansible’s releases.

The code regarding the examples is on GitHub

Collections

Introduction

I had already introduced the subject in a previous article, now it’s time to go deeper.
The Collections allow us to group together Playbook, roles, modules and plug-ins. This is useful for editors to group their Ansible content into one project.
Starting with Ansible 2.10, a wild range of modules has been removed from the Ansible project (called ansible-base for 2.10 and ansible-core for following versions) and are now a dedicated Collection project.

Why this choice? It’s rather simple actually, the Ansible project started to be too large and therefore too hard to maintain. When you had a look on the github project issues, it was mainly about modules. Those modules were, for the vast majority, maintained by dedicated teams, so it makes sense to have them split in different projects. You can find a detailed explanation here: https://www.ansible.com/blog/announcing-the-community-ansible-3.0.0-package

This new way of working can provide the following benefits:

  • A lightened Ansible Project
  • Independent life cylce (for both Ansible and Collections)
  • Dedicated teams to work for Collections

These changes will mostly impact core devs as there is now the Ansible community package (also called Ansible Project) for end-users. This bundle includes a defined version of ansible-core and a freezed version of collections. The 5.0 version should be released at the end of this month (November 2021).

Changes

There are three types of collection:

  • ansible.builtin.: still in the core project
  • community.: maintain by the community in dedicated project
  • cisco., vmware. etc.: maintain by editors

Exhaustive list here: https://github.com/ansible-collections

FQCN

For Fully Qualified Collection Name, it indicates the full name of a Module, Role, Plug-in or Playbook in a Collection. Let’s take the Grafana collection, the module that allows to manage Datasources is named: community.grafana.grafana_datasource

It consists of a namespace : community, a collection : grafana and a content_name : grafana_datasource.
The namespace list is managed by RedHat. Some editors have their namespace reserved like Cisco or VMware

Note: Even if ansible content tends to be split among collections, there are still lot in the general purpose collections:

  • community.general
  • community.network

Most of the collections are hosted on GitHub. Ansible Galaxy has been updated to be able to manage collections.

Update from 2.9: New VirtualEnv

If you have installed Ansible with pip, you won’t be able to upgrade in 2.10+, you will have to uninstall it.
If you have created a VirtualEnv to install Ansible, then you can create a new one for the newer versions. You can still keep the old one, it may be useful for testing purposes.

Creation of a newVirtualEnv:

1
virtualenv -p /usr/bin/python3 ansible4

Activation:

1
source ansible4/bin/activate

Installation of Ansible 4.7.0 and jmespath (for JSON filters):

1
2
pip install ansible
pip install jmespath

While displaying the Ansible version, you can see the associated ansible-core version:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
 (ansible210) jlg@DESKTOP-N97SPSR:~$ ansible --version
ansible [core 2.11.6] 
  config file = None
  configured module search path = ['/home/jlg/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jlg/venv/ans/lib/python3.6/site-packages/ansible
  ansible collection location = /home/jlg/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jlg/venv/ans/bin/ansible
  python version = 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
  jinja version = 3.0.2
  libyaml = True

Using collection

We can try to install the collection, if it’s already installed we will be notified:

1
ansible-galaxy collection install community.grafana

In a Play, we will specify we want to collection to be loaded:

1
2
3
4
5
6
7
8
- name: Grafana
  hosts: tig
  remote_user: jlg
  become: true
  gather_facts: false

  collections:
    - community.grafana

Grafana installation

Now we can continue our Playbook. Before installation Grafana, we will add some prerequisites:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
---
- name: Prerequisites
  hosts: tig
  gather_facts: yes
  remote_user: jlg
  become: true
  tasks:
    - name: Set timezone to Europe/Paris
      timezone:
        name: Europe/Paris
[...]
    - name: Add Grafana Apt key
      apt_key:
        url: https://packages.grafana.com/gpg.key
        state: present

    - name: Add Grafana stable Apt repository
      apt_repository:
        repo: deb https://packages.grafana.com/oss/deb stable main
        state: present

Then we add a new play with the installation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
- name: Grafana
  hosts: tig
  remote_user: jlg
  become: true
  gather_facts: false

  collections:
    - community.grafana

  tasks:
    - name: Install Grafana OSS
      apt:
        force_apt_get: yes
        update_cache: yes
        pkg: grafana

    - name : Start Grafana service
      service:
        name: grafana-server
        enabled: yes
        state: started

Add a Datasource

The first step after the installation of Grafana is adding a Datasource. To be able to create dashboards, we need a datasource. A datasource can be a relational DB (MySQL, Postgres …) or a TSDB like InfluxDB. There are other choices, here is a list: https://grafana.com/docs/grafana/latest/datasources/

To add a datasource we will use the Grafana collection. We add this task after the installation part:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    - name: Ensure Influxdb datasource exists
      grafana_datasource:
        name: "influxdb"
        grafana_url: "http://127.0.0.1:3000"
        grafana_user: "admin"
        grafana_password: "admin"
        org_id: "1"
        ds_type: "influxdb"
        ds_url: "http://127.0.0.1:8086"
        database: "{{ influxdb_database_name }}"
        time_interval: ">10s"

For the database name we will use the variable which has been defined in the previous article.

In Grafana we can now see the newly added InfluxDB Datasource:

In the next article we will see the rest of the configuration as well as the dashboard import and the data gathering with Telegraf!

Share on