An Introductory Guide on Deploying Cassandra Database using Ansible

Creating Cassandra_installation playbookDeploying a Cassandra database manually on hundreds of machines might be a complicated and exhausting task. Fortunately, we have automation tools that can install all the needed packages on all nodes with minimal configuration. In this case, Ansible is a great tool to get the job done. Throughout this guide, I’ll provide a detailed overview of a simple example of how to install and configure the Ansible environment and then install, configure and run Cassandra databases using Ansible on multiple nodes. The real-life scripts are quite more sophisticated, using roles for each task, but I'll try to avoid complexities and show an introductory step-by-step case.

What is Ansible?

To keep things short, Ansible is a software tool that helps administrators install and manage packages, execute commands and manage files on multiple machines using just their hostnames or IPs. Ansible needs to be installed on one machine called the Control Node (it can be your laptop). The nodes that are going to be pinged by the control node are called target nodes and do not need any installation since Ansible is a push-based configuration management system. Below you can see a simple visualization of how Ansible works:

Picture 1: Illustration of how Ansible sends requests on multiple nodes

By “request”, we mean installing and managing packages, performing commands, changing file permissions, and so on using appropriate Ansible modules. To group related tasks and run multiple requests at the same time, use Ansible Playbook. We will be working with them throughout this blog. Moreover, keep in mind that Ansible is an inconsistent tool, which means that if we try to install a dependency on 2 nodes and a request is successful on one node but fails on another, the changes will be applied on the node that succeeded. To better understand Ansible, what modules are, and how they work, please refer to the official Ansible documentation. Now, let’s go step by step through each key aspect until we have a successfully configured Cassandra cluster running.

Pre-requisites 

Step 1: Establish SSH connection between Control and Target nodes - We are not going to focus on this step here. I create an SSH keypair on the Control node, then send the public key to the Target node using the ssh-copy-id command. 

 Step 2: Install Ansible - The most obvious one, right? There are 2 ways of installing Ansible: using package installation or pip installation. I prefer the first option. Here are the installation steps for Ubuntu OS: 

$ sudo apt update
$ sudo apt install software-properties-common 
$ sudo add-apt-repository --yes --update ppa:ansible/ansible
$ sudo apt install ansible
  

If you prefer the pip installation or other OS, please refer to the official documentation page: https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html 

Step 3: Create configuration files ping Target Nodes through Ansible

1. Create ansible directory, then create 2 files inside this directory: ansible.cfg and inventory

mkdir ansible && cd ansible && touch inventory && touch ansible.cfg

2. Inside the inventory file, insert the IPs of the target nodes, as simple as that. In my case, it is the following: 

10.128.0.21
10.128.0.22
10.128.0.23

Inventory file content

3. Inside ansible.cfg file, insert the following piece of code: 

[defaults]

inventory = inventory
private_key_file = ~/.ssh/ansible_key

ansible.cfg  file content

Where inventory parameter refers to the inventory file, private_key_file parameter refers to the private SSH key path. Configure the path of private_key_file on your own to the location where your key is stored.  

4. After completing the following steps, we should be able to use the Ansible ping module to test access to the target nodes. The ping module tries to establish an SSH connection to the target nodes and report SUCESS if it works. To confirm, insert the following command: 

ansible all -m ping

The result should look like this: 

Picture 2: Successful result of Pinging Target Nodes through Ansible 

After establishing the connection, we can finally start working on our playbooks, where we will include all tasks required to install and configure Cassandra. 

How to use variables in Ansible

You may ask: “Can we use variables in Ansible, and how can we configure and store them?” In Ansible, variables are configured and stored in a separate file in a different directory. This helps us configure the parameters of our playbook without modifying its code. Variables are retrieved using double curly brackets before and after the variable. Variables in curly brackets should be located inside the double quotes. Below, you can find an example of storing variables in one of our playbooks: 

  - name: Install Python and Java
    apt:
      name:
      - ""
      - ""

Example of using variables in playbooks

There are a few ways to store variables in ansible, but we will focus on two most common for beginners: using vars or group_vars directories. The difference between these two methods is that while storing variables in group_vars you don't need to include or import them inside the playbook; just create your file here: 

dimonkosta1801@ansible:~/ansible/group_vars$ ls
all.yaml  
dimonkosta1801@ansible:~/ansible/group_vars$ pwd
/home/dimonkosta1801/ansible/group_vars

Path to all.yaml file 

But why do we store our variables inside the all.yaml file, instead of, for example, pre_task_vars.yaml? It's because all.yaml stands for “all hosts”, so that we can store variables that need to be executed on all Target Nodes in one place for multiple playbooks. Throughout this guide, we will have to configure some parameters in each individual node, so this feature will become extremely helpful. 

If you prefer to store variables in different files for each playbook, you will need to add an additional include_vars parameter, to which you will assign your vars file. In our case, it's pre_task_vars.yaml.

Picture 3: Adding additional include_vars parameter to our playbook

Here is where this  pre_task_vars.yaml file should be stored.

dimonkosta1801@ansible:~/ansible/vars$ ls
pre_task_vars.yaml
dimonkosta1801@ansible:~/ansible/vars$ pwd
/home/dimonkosta1801/ansible/vars

path to pre_task_vars.yaml file 

Be aware that if you try to store your pre_task_vars.yaml inside group_vars, your playbook will abort, as it accepts only file names with hosts or host group values. 

I'm going to use group_vars during this tutorial. 

Creating Pre-tasks Playbook 

For easier navigation, let’s create a separate playbook for installing all necessary packages that are not directly related to Cassandra. 

Before proceeding with Cassandra, we need to install Python 2.7 to access the cql shell and java jdk 1.8.0. Unfortunately, Ubuntu does not support the python package, only python2 or python3, depending on the version you want to install, so we will need to create a placeholder for python to work correctly. 

Inside the ansible directory, create a file called pre-tasks.yaml. Insert the following code inside: 

- name: Install and configure supporting packages for Cassandra
  hosts: all
  become: true
  tasks:

  - name: Install Python and Java
    apt:
      name:
      - ""
      - ""

  - name: Assigning python alias to python2
    community.general.alternatives:
      name: python3
      link: /usr/bin/python
      path: /usr/bin/python3


  - name: Printing python and java versions to make sure installation was correct
    shell: 'python -V && java -version'
    register: command_output
   
   # Check java version
  - debug:
      var: command_output.stderr_lines

   # Check python version
  - debug:
      var: command_output.stdout_lines

Pre-tasks .yaml file content

Here’s how my all.yaml file looks like this after creating the first playbook: 

java_version: openjdk-8-jdk
python_version: python3

all.yaml file content after creating pre_tasks.yaml playbook

At the beginning of the playbook, we identify which hosts we are going to target in this playbook. In our case, we are targeting all hosts. The next line, become: true, will perform a play as a sudo command. If sudo requires a password in the target nodes, you can use -K to ansible ask it for you. You will see a prompt "BECOME password".  

The first task in our playbook is responsible for installing appropriate packages of python and java. In the second task, we assign an alternative link to the python2 package so that we can use python keyword instead of python2. Finally, in our last task, we will use the ansible.builtin.shell module to execute a command python -V && java –version to confirm the successful execution of the previous two tasks. 
Let's run our playbook to ensure that it is configured correctly. To do so, execute the following command: 

ansible-playbook -i inventory pre-tasks.yaml -K

As mentioned before, -K will prompt you for a sudo password. Please note that without the -K option our playbook will abort if sudo requires password, and we specified become: true in our playbook.

The result should be similar to the screenshot below. The green "ok" result means your task is done, but Ansible didn't change anything (something like a package was already installed), and the yellow "changed" means your task was successful and Ansible was effectively acting on your system (something like ansible installed a new package). 

 Picture 4: Successful execution of pre_task.yaml playbook

 Now, let's proceed with the installation of our Cassandra database.

Creating Cassandra_installation playbook

Now, let's create a playbook for installing Cassandra. The screenshot below shows the tasks that I used to install Cassandra as a tarball: 

- name: Cassandra Installation Playbook
  hosts: all
  become: true
  tasks:

  - name: "Ensure group  exists"
    ansible.builtin.group:
      name: ""
      state: present

  - name: "Create  user"
    ansible.builtin.user:
      name: ""
      group: ""
      shell: /bin/bash
      append: yes
      comment: "Cassandra User"
      state: present

  - name: Installing cassandra tarball
    ansible.builtin.get_url:
      url: https://archive.apache.org/dist/cassandra//apache-cassandra--bin.tar.gz
      dest: ""
      mode: 0755
      owner: ""
      group: ""


  - name: Extract cassandra tarball
    unarchive:
      src: "/apache-cassandra--bin.tar.gz"
      dest: ""
      remote_src: true
      owner: ""
      group: ""


  - name: Change ownership of the installation folder
    become: true
    file:
      path: "/apache-cassandra-/"
      owner: ""
      group: ""
      recurse: true
      state: directory

 Cassandra_installation.yaml Playbook

Firstly, we are going to check if the cassandra group exists. If not, we are creating one. Then, we are creating the cassandra user, as in the future, we are going to make this user the Cassandra directory owner. Next, we are installing Cassandra using tarball and assigning a path where it should be located on Target Nodes. As you can see, we are also assigning this tarball to the newly created user and group. 

After that, we extracted the tarball and placed it into the cassandra_destination directory. You may also have the same destination for both tarball and extracted tarball files. Remember that you should extract tarball under the /usr  folder while downloading to wherever destination you want. Last but not least, we need to make sure that the Cassandra user and Cassandra group are assigned to our extracted file, as without it our Cassandra will not start up. Now, let's review all our updates.yaml file:

java_version: openjdk-8-jdk
python_version: python3

cassandra_version: 3.11.15
cassandra_destination: /usr/local
tarball_destination: /home/dimonkosta1801
cassandra_user: cassandra
cassandra_group: cassandra
 

all.yaml file after adding vars from Cassandra_installation.yaml Playbook

Let's run our second playbook and install Cassandra. 

Picture 5: Result of running Cassandra_installation.yaml Playbook

After accessing our Target Nodes, we will see tarball downloaded and extracted with the right permissions and in the appropriate directories: 

Picture 15: Allocation of Cassandra packages on Target Nodes

Next, we are going to configure our Cassandra database. 

Configuring Cassandra Database using Jinja2 Template

Jinja2 template allows us to configure files with variable placeholders on our Control Node and then pass these files to the Target Nodes with proper configuration. On your Control Node, in the ansible directory, clone the following git repository: 

git clone git@github.com:d1mkko/cassandra_configuration_files_template.git. 

Picture 16: cloning repository 

The new cassandra_configuration_files_template directory should appear: 

Picture 17: cassandra_configuration_files_template content

After introducing the cloned github repo, it might be difficult to understand the location of so many files. Below is a diagram for easier navigation:

.
└── ansible
    ├── ansible.cfg
    ├── cassandra_configuration_files_template
    │   ├── README.md
    │   ├── cassandra-rackdc.properties.j2
    │   ├── dc1.yaml
    │   ├── dc2.yaml
    │   ├── dc3.yaml
    │   ├── cassandra.yaml.j2
    │   └── cassandra_config_vars.yaml
    ├── cassandra_installation.yaml
    ├── group_vars
    │   ├── all.yaml
    ├── inventory
    └── pre-tasks.yaml

Ansible folder file tree 

• cassandra_installation.yaml and pre-tasks.yaml - our playbooks
• cassandra_configuration_files_template - cloned github repository
• group_vars - directory to store our variables
• cassandra-rackdc.properties.j2 and cassandra.yaml.j2 - jinja2 templates
• dc1.yaml, dc2.yaml, dc3.yaml - variable files to target specific host group

Inside this directory, we have 2 key configuration files, which are configured with the jinja2 template, and 4 variable files: dc1.yaml, dc2.yaml, dc3.yaml and cassandra_config_vars.yaml. Inside dcN.yaml files, we are configuring variables for cassandra_rackdc.properties for each host separately. For example, I decided to have a 3-node cluster which will be distributed across 3 DCs. If I want to configure each data center individually, I can input the configuration inside the variable file with the appropriate dc name and store it in group_vars directory. Now, let's separate our hosts in the inventory file into different groups:

[dc1]
10.128.0.21

[dc2]
10.128.0.22

[dc3]
10.128.0.23

Splitting our hosts into different groups

As you can see, the group names are now corresponding to the variable file names. It means that variables from dc1.yaml will be applied in the dc1 group, and so on.  Next, let's move our dcN.yaml files to the group_vars directory. Use the following command: 

mv cassandra_configuration_files_template/dc*.yaml group_vars/ 

Inside the cassandra_config_vars.yaml file you will find variables that are stored in cassandra.yaml file: 

Picture 20:  cassandra_config_vars.yaml file  

Here is a sample of how variables in cassandra.yaml.j2 file looks like. Note that for Jinja templates, double quotes are not required.

Picture 21:  cassandra.yaml.j2 file

Copy and paste parameters from cassandra_config_vars.yaml to all.yaml, then configure them based on your needs. Don't forget to specify your own seed nodes, as Cassandra will not be able to bootstrap. Here is how our all.yaml looks after inserting the variables.

Picture 22: Final version of all.yaml file

After having all our files configured, we can start writing our playbook to configure and start Cassandra:  

- name: Playbook for Configuring Cassandra
  hosts: all
  become: true
  tasks:

  - name: Send cassandra-rackdc.properties template file to hosts
    template:
      src: cassandra_configuration_files_template/cassandra-rackdc.properties.j2
      dest: "/apache-cassandra-/conf/cassandra-rackdc.properties"
      backup: yes
      owner: cassandra
      group: cassandra

  - name: Send cassandra.yaml template file to hosts
    template:
      src: cassandra_configuration_files_template/cassandra.yaml.j2
      dest: "/apache-cassandra-/conf/cassandra.yaml"
      backup: yes
      owner: cassandra
      group: cassandra

- name: Starting Cassandra 
  hosts: all
  serial: 1
  become: true
  tasks:

  - name: Starting Cassandra
    shell:
      cmd: "su - cassandra -c '/apache-cassandra-/bin/cassandra'"
      chdir: "/apache-cassandra-"

  - name: Wait for port 9042 to become open on the host, don't start checking for 30 seconds
    ansible.builtin.wait_for:
      port: 9042
      timeout: 300
      delay: 30

 cassandra.configuration.yaml playbook 

In the first and second tasks, we are sending jinja2 templates to the appropriate Cassandra configuration files using a template module. This module allows sending template files to the Target Node, replacing variable placeholders by their values. Last but not least, we execute the command using the shell module to start Cassandra under Cassandra user on all nodes. For this purpose, we added a new parameter serial with value 1. It means we will perform tasks on 1 node at a time. The last task, with the name:  Wait for port 9042 to become open on the host, don't start checking for 30 seconds is for Ansible to wait for 30 seconds after the current node is up and running to bootstrap the next node. After completing a playbook and ensuring all the variables are configured, let's run our playbook.

Picture 24: cassandra.configuration.yaml playbook successfully executed 

After that, access target nodes and check the logs to ensure Cassandra started bootstrapping:

tail -1000f /path/to/cassandra/logs/system.log

Once bootstrapped, use the below ansible ad-hoc command to ensure nodes are up and running:

ansible <<choose_one_node_ip_address>> -m ansible.builtin.shell -a "/usr/local/apache-cassandra-3.11.15/bin/nodetool status"

The output should be similar to the screenshot below: 

Picture 25: running nodetool status on all nodes

As you can see, we now have Cassandra configured and installed without even touching the target nodes. Knowing the basics of managing Cassandra using Ansible, you can do it simultaneously on hundreds of nodes with different settings and requirements of your choice. 
 

It was an introductory Ansible playbook. I hope it is helpful for you to get started. I also advise you to read more about the recommended directory structure, including group vars, and how to reuse code using Ansible roles. All this information and other best practices can be found in the Ansible Documentation - Best Practices.