Blog

Terraform the Cloud

Category
Software development
Terraform the Cloud

Are you still creating cloud infrastructure by clicking on the web UI? That’s old. A new approach for provisioning cloud infrastructure (or any other platform infrastructure) has emerged in the past few years, based on Infrastructure as Code principle (or IaC shorter).

What are the advantages over the manual provision, you might ask? The obvious one is that we can version the code; therefore, everything is easily manageable and observable. Also, IaC tracks each change, and everyone can see the changes made when they need to.

Third, an even more important consequence is that in a couple of clicks, we can now provision either an entirely new environment (for example, a new test or staging environment) or, even more importantly, in case of any catastrophic event, quickly and simply recover the whole infrastructure.

Currently, with IaC approach usage growing, dozens of solutions exist. The most common ones are, Terraform, Ansible, Plumi, CloudFormation, Chef, Puppet, and many more. 

Our point of interest in this blog is Terraform, developed by HashiCorp. Terraform is a great and widely used IaC tool because of its modularity, simplicity, and how lightweight it s. And it’s free. Of course, the tool has a freemium Cloud version with expanded features.

How Will We Terraform Today?

In this blog, I will demonstrate an example of cloud provisioning of some infrastructure objects on the Google Cloud Platform.

That implies:

  • creating Kubernetes deployment and secrets
  • LoadBalancer on a Kubernetes GKE cluster
  • initializing the VM (Compute Engine)
  • installing software

Software initialized on VM is the Keycloak server, whose password will be generated and then stored in Kubernetes secret. Of course, we can provide the GKE cluster via Terraform, but talking about it would seriously expand this blog. Moreover, Hashicorp already made their own guide.

Let’s start off then.

Authentication to GCP

The first “must” thing you must do is install and log in to gcloud. Invoking the command gcloud auth application-default login in the terminal will redirect you to the browsers Google Auth Library login. With successful login, the gcloud tool will create Application Default Credentials, which can authorize API calls that Terraform will create to provision objects on GCP.

Another approach to this solution is to use a predefined Service Account with enough rights. With a key file containing Service Account information saved on the local machine, we have to write its path in the environment variable GOOGLE_APPLICATION_CREDENTIALS.

I went with the first approach since it is faster and better solution when running Terraform on a local machine.

After you sort the authentication, it’s time to write some code.

Setting Up Providers for Terraform

Before creating objects themself, Terraform needs a provider. Provider in Terraform is a plugin that enables API communication with a targeted platform. We need a Google provider since GCP is a targeted platform for this demo.

The second one is the one for Kubernetes. While we use Google driver to create and provision GKE clusters, nodes, and other related things, we also need a Kubernetes provider to manage objects in the Kubernetes Cluster. Also, as I will install the Keycloak server using Terraform to manage it, I also need a Keycloak provider.

Defined providers in main.tf file look like this:

# env/workshop
terraform {
 required_version = ">=1"
 required_providers {
   google = {
 	source = "google"
 	version = ">=4.4.0"
   }
   kubernetes = {
 	source = "hashicorp/kubernetes"
 	version = ">= 2.7"
   }
   keycloak = {
 	source = "mrparkers/keycloak"
 	version = "~> 3.9.0"
   }
 }
}Code language: PHP (php)

The next thing we need to configure is Terraform backend. The backend is the storage of Terraforms state files. Default storage is on a local machine from which Terraform runs, but if someone else would like you to use or modify the same Terraform code, the state of already deployed items wouldn’t be available.

Also, since that state file contains everything in plain text, even passwords and other secrets, storing it in the repository is not an option. To elude this situation, you can use Backends, such as Consul, GCS, S3, Kubernetes, and others. In this approach, since we will be using Google Cloud, the backend option will be GCS. That means the state will be stored in a bucket.

# env/workshop
terraform {
  required_version = ">=1"
  backend "gcs" {
	bucket = "tf-state-learn-at-lunch"
	prefix = "very-secret-folder/env/workshop"
  }
  required_providers {
	...
  }
}Code language: PHP (php)

With this configuration, each user of the same Terraform code with the correct credentials, has access to the state of already deployed elements, and, therefore, can’t destroy them while applying the code again.

Creating GCP VM

With Terraform configured, the next stop is creating infrastructure objects. Terraform configuration will consist of a couple of modules. The first one we’re gong to create is GCP, and the second one is Kubernetes. 

File Structure Terraform Configuration

The first and the most common object created is VM, or as Google calls it, Compute Engine

# gcp/main.tf
resource "google_compute_instance" "test" {
  name         = "latl-test"
  machine_type = "e2-medium"
  zone         = "europe-west3-c"
  boot_disk {
	initialize_params {
  	image = "debian-cloud/debian-11"
  	size  = 10
	}
  }
  network_interface {
	network = "default"
	access_config {
  	network_tier = "STANDARD"
	}
  }
  tags = ["latl"]
}Code language: PHP (php)

This block will create an e2-medium Compute Engine named latl-test with Debian 11 OS. Also, it will provide an external IP for accessing the VM from the internet. The tag I named latl, we will use later to assign VM to a firewall rule which we will create next. It will allow external access (or ingress) to ports 22 (for SSH), 8080, and 8443.

# gcp/main.tf
resource "google_compute_firewall" "default" {
  depends_on = [
	google_compute_instance.test
  ]
  name	   = "latl-firewall"
  network = "default"
  allow {
	protocol = "tcp"
	ports    = ["22", "8080", "8443"]
  }
  target_tags   = [ "latl" ]
  source_ranges = [ "0.0.0.0/0" ]
  direction     = "INGRESS"
}
output "out_extIp" {
  value 	= google_compute_instance.test.network_interface.0.access_config.0.nat_ip
  description = "External IP address of VM"
}Code language: PHP (php)

In this block, you can see the first appearance of the depends_on argument. It explicitly says to Terraform that block (meaning infrastructure element) relays other blocks and needs to be created after that other one.

Also, as an output result, an external IP address of the VM we need to store for later use.

With these three blocks, we can create a VM instance and a firewall rule. But where is the fun in that? The instance is empty and doesn’t have any functionality. 

Installing Keycloak Server

To tackle the lack of content, the instance will be filled with the instance of the Keycloak server. VM image containing Keycloak we could create for example, with Packer or by using GCPs metadata_startup_script (like in this example). But, for this demo, I will use Terraforms remote-exec provisioner. It’s a last-resort tool and you should use it with caution because Terraform’s descriptive model doesn’t control its behavior but depends on provided scripts that might not always work.

Remote-exec provisioner executes a remote connection after the created and executes a provided shell script from a file.

# gcp/main.tf
provisioner "remote-exec" {
	script = "${path.module}/script.sh"
 }Code language: PHP (php)
# gcp/script.sh
sudo apt-get update
sudo apt-get install --yes wget software-properties-common
sudo wget -qO - https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | sudo apt-key add -
sudo add-apt-repository --yes https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/
sudo apt-get update && sudo apt-get --yes install adoptopenjdk-11-hotspot
sudo curl -LO https://github.com/keycloak/keycloak/releases/download/16.0.0/keycloak-16.0.0.tar.gz
sudo tar -xvzf keycloak-16.0.0.tar.gzCode language: PHP (php)

The script will download and install Keycloak and also all required dependencies. To run Keycloak, the remote-exec provisioner executes the next shell code, but as an inline code.

# gcp/main.tf
  provisioner "remote-exec" {
	inline = [
  	"cd /home/terraform/keycloak-16.0.0/bin/",
  	"sudo ./add-user-keycloak.sh --realm master --user admin --password ${local.randomPwd}",
  	"sudo nohup ./standalone.sh -b 0.0.0.0 > /dev/null 2>&1 &",
  	"sleep 20",
  	"sudo ./kcadm.sh config credentials --server http://localhost:8080/auth --realm master --user admin --password ${local.randomPwd}",
  	"sudo ./kcadm.sh update realms/master -s sslRequired=NONE"
	]
  }
}Code language: PHP (php)

You may have noticed that there is a string interpolation of randomPwd inside the script. That is, as the name suggests, a randomly generated password, later used as an admin password. The generated password is 36 characters long with some added rules. After it’s created, the password is stored as a local variable and then set as secret output, which will be saved to Kubernetes secret.

# gcp/main.tf
resource "random_password" "random_pwd" {
  length      = 36
  special     = false
  min_upper   = 6
  min_numeric = 15
}
locals {
  randomPwd = sensitive(random_password.random_pwd.result)
}
output "out_randomPwd" {
  value 	= local.randomPwd
  sensitive 	= true
}Code language: PHP (php)

To be able to use remote-exec provisioner, we need to add an SSH key. Of course, the key is also generated by Terraform using the tls_private_key resource, and its matching public key is added to VMs approved SSH keys. To configure the provisioner connected, we use a generated private key.

# gcp/main.tf
resource "tls_private_key" "ssh_key" {
	algorithm = "RSA"
	rsa_bits  = 2048
}
resource "google_compute_instance" "test" {
...
  metadata = {
	ssh-keys = "terraform:${trimspace(tls_private_key.ssh_key.public_key_openssh)}"
  }
...
  connection {
  	host        = google_compute_instance.test.network_interface.0.access_config.0.nat_ip
  	type        = "ssh"
  	user        = "terraform"
  	agent       = "false"
  	private_key = trimspace(tls_private_key.ssh_key.private_key_openssh)
  }Code language: PHP (php)

With that, we completed the GCP module.

Creating Variables Files for Keycloak Server

But before creating objects, I’ll create a variables file that will, as the name suggests, contain variables and some values. Those variables we’re going to use multiple times across configuration blocks, so it’s good practice to place them as part of the variables file. The variable file in Terraform we will name variables.tf. There are more naming conventions and ways to use variables.

variable "gke_namespace" {
  default = "learn-at-lunch"
  type = string
  description = "Kubernetes namespace"
}
variable "keycloak_password" {
  default = ""
  type = string
  description = "Password for keycloak admin user"
}
variable "keycloak_url" {
  default = "localhost"
  type = string
}Code language: JavaScript (javascript)

We must create the Keycloak URL and password variables to pass them from the previously created GCP module.

Deploying to the GKE Cluster

The next stop is a Kubernetes module. It consists of provisioning a namespace, Flask application deployment, some secret, and a LoadBalancer service on the existing GKE cluster.

# kubernetes/main.tf
resource "kubernetes_namespace" "namespace" {
  metadata {
	name = "learn-at-lunch"
  }
}
resource "random_pet" "random_animal" {
  length = 1
}
resource "kubernetes_secret" "secrets" {
	depends_on = [
  		kubernetes_namespace.namespace
	]
	metadata {
  		name      = "web-secret"
  		namespace = var.gke_namespace
	}
	data = {
  	"very_secret_animal" = random_pet.random_animal.id
  	"keycloak_password"  = var.keycloak_password
  	"keycloak_url"   	 = var.keycloak_url
	}
}

The preceding code block creates a namespace named learn-at-lunch, generates a random animal, and creates a secret that consists of three key-value pairs. Key very_secret_animal takes data from the randomly generated animal name, and the other two will inherit some Keycloak values from the GCP module by output blocks. A more on that in a bit.

Flask Application deployment

The next great thing is Flask application deployment. The deployment block consists of a lot of configuration elements. The most important one is image ​​moreskovic/simple-web-notch, a Docker image that contains the Flask application.

Next up is setting environment variables from a secret so that the application could have the needed data, then opening HTTP port 80 on which the application will listen for incoming connections, and finally, limit pod resources.

# kubernetes/main.tf
resource "kubernetes_deployment" "web" {
  depends_on = [
    kubernetes_secret.secrets
  ]
  timeouts {
    create = "2m"
    update = "1m"
  }
  metadata {
    namespace  = var.gke_namespace
    name       = "simple-web-notch"
    labels     = {
   		    "used_for": "learn-at-lunch"
    	}
  }
  spec {
    replicas = 1
    strategy {
     	 type = "RollingUpdate"
    }
    selector {
     	 match_labels = {
   			 "used_for": "learn-at-lunch"
     	 }
    }
    template {
   		 metadata {
   			 labels = {
 				 "used_for": "learn-at-lunch"
   			 }
     	 }
     	 spec {
   			 restart_policy = "Always"
   			 node_selector = {
 				 "cloud.google.com/gke-nodepool": "default-pool"
   			 }
   			 container {
 				 image = "moreskovic/simple-web-notch:latest"
 				 name  = "simple-web-notch"
 				 env {
   					 name = "SECRET_VARIABLE"
   						 value_from {
     							 secret_key_ref {
       							 name = "web-secret"
       							 key  = "very_secret_animal"
     							 }
   						 }
 				 }
  				 env {
   					 name = "KEYCLOAK_URL"
   						 value_from {
     							 secret_key_ref {
       					 		name = "web-secret"
       					 		key  = "keycloak_url"
     							 }
   						 }
 				 }
 		 port {
   			 container_port = 80
   			 name           = "http"
   			 protocol       = "TCP"
 		 }
 		 resources {
   			 limits = {
     				 memory = "128Mi"
   			 }
   			 requests = {
     				 cpu    = "50m"
     				 memory = "64Mi"
   			 }
 		}
   	   }
     	}
     }
   }
}

The application will use provided environment variables from secret for displaying Keycloak URL and randomly generated animal. Since the Flask application needs just a few resoures, the limit is 128 MB of memory.

LoadBalancer

And last, but not least object is LoadBalancer which will enable reaching applications from the internet. LoadBalance may not be the best choice for such a small application, but for this demo, the purpose is more than good enough.

# kubernetes/main.tf
resource "kubernetes_service" "web_lb" {
  depends_on = [
	kubernetes_namespace.namespace,
	kubernetes_deployment.web
  ]
  timeouts {
	create = "1m"
  }
  metadata {
	namespace = var.gke_namespace
	name  	   = "simple-web-notch-lb"
  }
  spec {
	selector = {
  	   "used_for": "learn-at-lunch"
	}
	port {
  	   port         = 80
  	   target_port  = 80
	}
	type = "LoadBalancer"
  }
}
output "lb-ip" {
  value = kubernetes_service.web_lb.status.0.load_balancer.0.ingress.0.ip
}

The creation of LoadBalancer depends on the namespace and application itself, so we’re going to create it afterwards. The Port used is the same as the application, 80 or HTTP, and via selector specification, deployment is targeted. If creating the LoadBalancer is successful, external IP will pass as a module output.

After we create that module, we’re almost at the finish line. All it remains is to use created modules from Terraform configuration and configure Keycloak.

To use modules, we need to describe them in the main.tf file in the workshop folder. Besides modules, providers Google and Kubernetes need some configuring. Google needs at least the project’s name and region, while Kubernetes needs a path to the kubeconfig file.

# workshop/main.tf
provider "google" {
  project = "wearenotch"
  region  = "europe-west3"
}
provider "kubernetes" {
  config_path = "~/.kube/config"
}
module "gcp" {
  source = "../../mod/gcp"
}
module "kuberentes" {
  source = "../../mod/kubernetes"
  depends_on = [
	module.gcp
  ]
  keycloak_password = module.gcp.out_randomPwd
  keycloak_url  	= "http://${module.gcp.out_extIp}:8080"
}Code language: PHP (php)

The GCP module will execute first because Kubernetes depends on it. In the execution of the Kubernetes module, outputs, generated passwords, and assigned IP addresses from the GCP module, will be passed as variables and therefore added to the Kubernetes secret. Upon execution of the Kubernetes module and using another output, this time from a parent configuration file, LoadBalancer IP will be displayed on the CLI.

# workshop/main.tf
...
output "lb-access-ip" {
  value = module.kuberentes.lb-ip
}Code language: PHP (php)

The application should open by entering the given IP address into the web browser.

Example of given output Terraform
Keycloak URL Terraform Cloud

Random animal and Keycloak URLs are written. 

Keycloak Management

Since Keycloak has no users or realms except admin, it’s about the right time to add some. To add users and realms again, we will be using Terraform. This time configuration will be separate, independent of any modules and modules parent. 

Adding Users to Keycloak

The structure is the same as before, the main.tf contains configuration, variables.tf contains variables with the addition of users.csv. The next step is to create the file that consists the list of users. 

Keycloak Management Terraform
# keycloak/variables.tf
variable "gke_namespace" {
  default = "learn-at-lunch"
  type = string
  description = "Kubernetes namespace"
}
variable "keycloak_password" {
  default = ""
  type = string
  description = "Password fo keycloak admin user"
}
variable "keycloak_url" {
  default = "localhost"
  type = string
}Code language: PHP (php)
# keycloak/users.csv
id,username,first_name,last_name,email
1,test1,Test1,Testy,test1@wearenotch.com
2,test2,Test2,Testy,test2@wearenotch.comCode language: PHP (php)

This list can be long as needed and could even be a JSON. To create all users, Terraform will iterate over the list. But, to get access to Keycloak, the provider is first needed. Yes, again, since now this configuration is independent of previous ones.

# keycloak/main.tf
terraform {
  required_version = ">=1"
  backend "gcs" {
	bucket = "tf-state-learn-at-lunch"
	prefix = "very-secret-folder/keycloak"
  }
  required_providers {
	kubernetes = {
  	source  = "hashicorp/kubernetes"
  	version = ">= 2.7"
	}
	keycloak = {
  	source  = "mrparkers/keycloak"
  	version = "~> 3.9.0"
	}
  }
}
provider "kubernetes" {
  config_path = "~/.kube/config"
}Code language: PHP (php)

The next step is to fetch the stored admin password from the Kubernetes secret to authenticate to the Keycloak server for management purposes. To do so, use Terraform data block. Since Terraform now, using a Kubernetes provider and Kubeconfig, has access to the GKE cluster, it can use its data by defining the namespace and name of the secret. With this approach, a user won’t be able to see the generated password, nor would they need it.

# keycloak/main.tf
data "kubernetes_secret" "keycloak-credentials" {
  metadata {
	name  	= "web-secret"
	namespace = var.gke_namespace
  }
}

While data from the secret would now be available, we still needs to be access it and extract it. I’ll use local variables to do so. Local variables are available only inside this configuration file and are going to mostly store values from some longer syntaxes. Besides extracted values from the secret, they store the initial user password and user list. Processing provided CSV file creates a user list and using function csvdecode, will transform the csv file into a list of maps, which we can iterate or use later.

# keycloak/main.tf
locals {
  keycloak_url  	= data.kubernetes_secret.keycloak-credentials.data.keycloak_url
  keycloak_password = data.kubernetes_secret.keycloak-credentials.data.keycloak_password
 
  init_password 	= "InicijalniPassword123!"
  users             = csvdecode(file("${path.module}/users.csv"))
}Code language: PHP (php)

With local variables done, you should initialize the access to the Keycloak server. That is, of course, done via a provider. The provider uses a couple of parameters, two of which are the Keycloak server URL and administrator password.

# keycloak/main.tf
provider "keycloak" {
  client_id      = "admin-cli"
  username       = "admin"
  password       = local.keycloak_password
  url            = local.keycloak_url
  initial_login  = false
}Code language: PHP (php)

After configuriring provider, Terraform is good to go. It has access to Keycloak and can manage it. 

Adding Realms to Keycloak

Now, the next thing that we need is a realm. There is actually one realm by default, the master realm. We need to make only a few changes here, such as setting a password policy, locales, and security defenses.

After the realm, we can create users. To do so, the first thing needed is the master realm id. Id is a randomly generated UUID, so we need to extract it from the system. The best way to do it is to use the data block “keylocak_realm” which can provide such information.

Realm id is then fetched from it inside the block that creates users. Using the for_each looping function, each map is read from the list (generated from a CSV file), and the user is created.

# keycloak/main.tf
resource "keycloak_realm" "realm_master" {
  realm           = "master"
  password_policy = "upperCase(1) and lowerCase(1) and digits(1) and specialChars(1) and length(12)"
  security_defenses {
	headers {
  	x_frame_options                     = "SAMEORIGIN"
  	content_security_policy             = "frame-src 'self'; frame-ancestors 'self'; object-src 'none';"
  	content_security_policy_report_only = ""
  	x_content_type_options          	   = "nosniff"
  	x_robots_tag                    	   = "none"
  	x_xss_protection                	   = "1; mode=block"
  	strict_transport_security       	   = "max-age=31536000; includeSubDomains"
	}
  }
  internationalization {
	supported_locales = [
  	   "en",
  	   "de"
	]
	default_locale = "en"
  }
}

data "keycloak_realm" "keycloak-realm-info" {
  realm = "master"
}

resource "keycloak_user" "test-users" {
  for_each = { for user in local.users : user.id => user}
  realm_id   = data.keycloak_realm.keycloak-realm-info.id
  username   = each.value.username
  first_name = each.value.first_name
  last_name  = each.value.last_name
  enabled	= true
  email  	= each.value.email
  attributes = {
	"locale"	= "en"
  }
  initial_password {
	value     = local.init_password
	temporary = false
  }
}Code language: PHP (php)

Terraforming the Conclusion 

Terraform, or any IaC tool on the market today, are great for many reasons, and I think their usage will grow even more extensively in the future.

As demonstrated in this demo, the biggest perks are security and scalability.

With all configuration code executed, I’ve set up a Keycloak server with multiple user accounts, all created via machine, without my need to know the master password nor the need for generating and adding SSH keys manually. This approach of using Terraform is convenient from a teamwork perspective also. Any change that we might need to achieve on Keycloak, or for example, user-added, it’s an additional block to a configuration file.

You can find the full code in this demo, as well as the Flask application my GitHub

CONTACT US

Exceptional ideas need experienced partners.