The magic explained
In the previous episode, we cloned the code from Github, modified some variables and provisioned and deployed two servers on AWS. In this final episode we will explain what was going on behind the scenes and discuss the Terraform and Ansible part.
1. The Terraform part
The most important files of the Terraform code can be broken down into two files.
a. The main.tf file.
In this file, all things that need to be done by Terraform are configured.
First, we need to declare all the variable used in this file.
variable "aws_region" {} variable "profile" {} variable "server_port" {} variable "key_name" {} variable "public_key" {} variable "private_key" {} variable "aws_access_key_id" {} variable "aws_secret_access_key" {} variable "aws_availability_zone" {} variable "aws_ami_id" {}
Let Terraform know we will use the aws-provider.
provider "aws" { profile = var.profile region = var.aws_region access_key = var.aws_access_key_id secret_key = var.aws_secret_access_key }
Add a public key to aws for ssh access.
resource "aws_key_pair" "deployer" { key_name = var.key_name public_key = var.public_key }
Configure the firewall with some inbound(ingress) and outbound(egress) rules.
resource "aws_security_group" "devotest" { name = "terraform-example-instance" ingress { from_port = var.server_port (A variable used as port from the world accessable) to_port = var.server_port protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } ingress { from_port = 22 (Port 22 ingress from the world accessable)
to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } egress { from_port = 80 (Port 80 to egress) to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } egress { from_port = 443 (Port 443 to egress) to_port = 443 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } }
We will create two centos 7.7 machine instance type t2.micro. T2.micro is a machine with 1 vCPU, 6CPU credits/hour and 1GiB memory. The var aws_ami_id gives us the centos 7.7 machines. One is called devotest_web, the other devotest_db.
resource "aws_instance" "devotest_web" { ami = var.aws_ami_id instance_type = "t2.micro" key_name = var.key_name vpc_security_group_ids = [aws_security_group.devotest.id] tags = { Name = "terraform-devotest" } root_block_device { delete_on_termination = true (Delete also the volume when deleting the instance) } } resource "aws_instance" "devotest_db" {
ami = var.aws_ami_id instance_type = "t2.micro" key_name = var.key_name vpc_security_group_ids = [aws_security_group.devotest.id] tags = { Name = "terraform-devotest" } root_block_device { delete_on_termination = true (Delete also the volume when deleting the instance) } }
After the deployment of the machines, we want to get back some information as variables.
public_ip_web, the public ip of the webserver. output "public_ip_web" { value = aws_instance.devotest_web.public_ip description = "The public IP of the web server" }
name_web, the name of the webserver output "name_web" { value = aws_instance.devotest_web.tags.Name description = "The Name of the web server" }
state_web, the state of the webserver instance output "state_web" { value = aws_instance.devotest_web.instance_state description = "The state of the web server" } public_ip_db, the ip of the database instance output "public_ip_db" { value = aws_instance.devotest_db.public_ip description = "The public IP of the db server" } name_db, the name of the database instance output "name_db" { value = aws_instance.devotest_db.tags.Name description = "The Name of the db server" } state_de, the state of the database instance output "state_db" { value = aws_instance.devotest_db.instance_state description = "The state of the db server" }
b. Theterraform.tfvarsfile.
This file will be created from an Ansible template which makes it possible to pass Ansible variables to the Terraform part.
This is the content of the roles/terra-provision/templates/terraform.tfvars.j2 file, the template used to create the terraform/terraform.tfvars file. The variables between “{{ }}” are Ansible variables and will be replaced with the values from group_vars/all/vars.yml and group_vars/all/vault.yml.
# the location of the private key for connection to the servers private_key ="~/ansible-terraform-keys/id_rsa" server_port = 80 key_name = "aws_deployer" # public key, aws_access_key_id, aws_secret_access_key will be overwritten from ansible vault var public_key = "{{ public_key }}" aws_access_key_id = "{{ aws_access_key_id }}" aws_secret_access_key = "{{ aws_secret_access_key }}" profile = "{{ aws_profile }}" # aws_ami_id, aws_region will be overwritten from ansible var aws_ami_id = "{{ aws_ami_id }}" aws_region = "eu-west-1" aws_availability_zone = "eu-west-1a"
2. The Ansible part.
The job of Ansible playbook is divided into 3 plays.
a. First play use Terraform to deploy the machines.
- hosts: localhost (we run this playbook on our local machine) connection: local gather_facts: no tasks: The different tasks follow after this parameter.
First task: we create the variable file for Terraform from a template (see above). This can be done via the terra-provision role.
- name: create teraform.tfvars include_role: name: terra-provision
If the Terraform does not exist, we run the Terrafrom init command to initialise Terraform.
- name: init the terraform if .terraform is not there shell: terraform init args: chdir: "{{ playbook_dir }}/terraform/" creates: "{{ playbook_dir }}/terraform/.terraform/"
We run the Terraform script. This will read the terraform/main.tf file and deploys whatever is configured.
- name: run the terraform script terraform: project_path: "{{ playbook_dir }}/terraform/" state: "{{ aws_instance_state }}" variables: aws_region: "{{ aws_region }}" aws_access_key_id: "{{ aws_access_key_id }}" aws_secret_access_key: "{{ aws_secret_access_key }}" aws_ami_id: "{{ aws_ami_id }}" public_key: "{{ public_key }}" register: terra_result (The output will be added to the terra_result variable)
For debug we show the result from the previous Terraform command.
- name: show terra_result debug: var: terra_result
Out of the result, we retrieve the ip address form the web- and database server.
- name: set vm_ip / name set_fact: vm_ip_web: "{{ terra_result.outputs.public_ip_web.value }}" vm_ip_db: "{{ terra_result.outputs.public_ip_db.value }}" when: - terra_result.outputs.state_web is defined - terra_result.outputs.state_db is defined
In the next block, we create a dynamic inventory from the Terraform result, the ip’s will be stored in the Ansible inventory so it can be used in the next plays. We create an inventory for the webserver and another for the database server.
- name: create the dynamic inventory block: - name: remove old dynamic group_vars file file: path: "{{ item }}" state: absent with_items: - group_vars/dynamic_web.yml - group_vars/dynamic_db.yml - name: create new centos group_vars file file: path: "{{ item }}" state: touch with_items: - group_vars/dynamic_web.yml - group_vars/dynamic_db.yml - name: create the inventory directory file: path: inventory/ state: directory
- name: remove old dynamic host file file: path: inventory/hosts state: absent - name: create new dynamic host file file: path: inventory/hosts
state: touch - name: add retrieved IP to file blockinfile: path: group_vars/dynamic_web.yml marker: "" block: | ---
ansible_host: {{ vm_ip_web }} ansible_user: {{ remote_user[hypervisor] }} become_user: {{ remote_user[hypervisor] }} remote_user: {{ remote_user[hypervisor] }} become: true
- name: add retrieved IP to file blockinfile: path: group_vars/dynamic_db.yml marker: "" block: | --- ansible_host: {{ vm_ip_db }} ansible_user: {{ remote_user[hypervisor] }} become_user: {{ remote_user[hypervisor] }} remote_user: {{ remote_user[hypervisor] }} become: true ... - name: add retrieved IP to file blockinfile: path: "inventory/hosts" marker: "" block: | [dynamic_web] {{ vm_ip_web }} [dynamic_db] {{ vm_ip_db }}
- name: Add host add_host: hostname: "{{ vm_ip_web }}" groupname: dynamic_web remote_user: "{{ remote_user[hypervisor] }}" - name: Add host add_host: hostname: "{{ vm_ip_db }}" groupname: dynamic_db remote_user: "{{ remote_user[hypervisor] }}" when:
- terra_result.outputs.state_web is defined - terra_result.outputs.state_db is defined - name: Collect facts again setup:
As a last step, we will check if we can access the webserver via ssh.
################################ # pause # ################################ - name: Wait 300 seconds for port 22 to become open and contains the string "OpenSSH" wait_for: port: 22 host: '{{ vm_ip_web }}' search_regex: OpenSSH delay: 10 vars: ansible_connection: local when: vm_ip_web is defined
b. A second play installs the software on the webserver, configure it and start it.
Use the dynamic_web inventory
- hosts: dynamic_web
Load the role webserver.
tasks: - name: create a website include_role: name: webserver
The content of the webserver role can be found in roles/webserver/tasks/main.yml
You will see that this role does a yum update (which can take some time), installs the apache webserver (httpd), creates a website configuration file in ‘/etc/httpd/conf.d/’, creates a html page to serve, and finally restarts the webserver.
This was the last episode of the Ansible Terraform series. You can try to extend the code by trying the following:
- database server:
- add a password for the root user.
- create some tables with content in the Devoteam database.
- open ports for connection from webserver to database server on port 3306.
- webserver:
- create a connection to the database server and show some content of it on the webpage.