HOW TO CREATE AN AWS ELASTIC KUBERNETES SERVICE(EKS) CLUSTER USING TERRAFORM

COMPANY BACKGROUND
XXX Enterprise Solutions is a technology firm on a mission to reinvent the world of technology. As the range of their products grew, they recognized the importance of having an efficient and scalable container orchestration solution. The decision was to use Amazon Elastic Kubernetes Service (EKS) to manage containerized applications efficiently.

CHALLENGE
XXX Enterprise Solutions encountered an obstacle in deploying and managing an AWS EKS cluster efficiently. The conventional manual approach was time-consuming and error-prone, slowing the rate of innovation. The team needed a scalable solution that could adapt to their changing needs while maintaining reliability and ease of maintenance.

SOLUTION: Using Terraform for Infrastructure as Code:
Recognizing the importance of a systematic approach, XXX Enterprise Solutions selected Terraform as their Infrastructure as Code (IaC) solution. Terraform declarative syntax appealed to the team since it provides a straightforward, concise, and human-readable approach to defining AWS resources. This decision formed the groundwork for a more standardized and consistent infrastructure architecture.

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service that eliminates the need to install, operate, and maintain your own Kubernetes control plane on Amazon Web Services (AWS). Kubernetes is an open-source system that automates the management, scaling, and deployment of containerized applications.

PREREQUISITE

• AWS Account
• AWS cli configured.
• Terraform installed.
• Kubectl installed.
• An existing AWS VPC that meets the requirement for EKS setup. Find the GitHub link to the repository to deploy an AWS VPC.

Terraform Configuration file for EKS cluster with managed node groups.

1. To create an Amazon EKS cluster.

• To create an Amazon EKS cluster IAM role and policy.

## master-role.tf

data "aws_iam_policy_document" "master_assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "master_role" {
  name               = "eks-masternode-role"
  assume_role_policy = data.aws_iam_policy_document.master_assume_role.json
}

resource "aws_iam_role_policy_attachment" "masternode_AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.master_role.name
}

The above terraform configuration file defines an IAM role for the EKS control plane with a specific assumed role policy, which attaches the AmazonEKSClusterPolicy to the role, granting the EKS control plane the necessary permissions for managing the cluster.

The data “aws_iam_policy_document” “master_assume_role block defines the aws_iam_policy_document data source and specifies the permissions to allow AWS Elastic Kubernetes Service (EKS) service (eks.amazonaws.com) to assume sts:AssumeRole action.
Resource “aws_iam_role” “master_role” block creates an IAM role “eks-masternode-role,” and the resource “aws_iam_role_policy_attachment” “masternode_AmazonEKSClusterPolicy” block attaches the AmazonEKSClusterPolicy managed policy to the IAM Role which provides the EKS control plane the necessary permissions for managing the cluster.

• Create the security group.

# sg.tf
# EKS Cluster security group
resource "aws_security_group" "eks_cluster" {
  name        = var.cluster_name
  description = "Cluster communication with worker nodes"
  vpc_id      = var.vpc_id

  tags = {
    Name = "EKS-project"
  }
}

resource "aws_security_group_rule" "cluster_inbound" {
  description              = "Allow worker nodes to communicate with the cluster API Server"
  from_port                = 443
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eks_cluster.id
  source_security_group_id = aws_security_group.eks_nodes.id
  to_port                  = 443
  type                     = "ingress"
}

resource "aws_security_group_rule" "cluster_outbound" {
  description              = "Allow cluster API Server to communicate with the worker nodes"
  from_port                = 1024
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eks_cluster.id
  source_security_group_id = aws_security_group.eks_nodes.id
  to_port                  = 65535
  type                     = "egress"
}

This sg.tf file creates an AWS security group attached to the EKS cluster, enabling secure communication between the cluster API server and worker nodes. It allows inbound traffic on port 443 from worker nodes and outbound traffic on dynamic ports (1024–65535) from the API server to worker nodes.

• Create the EKS Cluster

# main.tf
# Defining Terraform Provider
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.32.1"
    }
  }
}

## eks-cluster.tf

resource "aws_eks_cluster" "eks_project" {
  name     = "AWS_EKS_Project"
  role_arn = aws_iam_role.master_role.arn

  vpc_config {
    security_group_ids      = [aws_security_group.eks_cluster.id, aws_security_group.eks_nodes.id]
    endpoint_private_access = true
    endpoint_public_access  = true
    subnet_ids = var.subnet_ids
  }
  depends_on = [
    aws_iam_role_policy_attachment.masternode_AmazonEKSClusterPolicy
  ]
}

This block of the configuration files creates an AWS EKS cluster named “AWS_EKS_Project” with the IAM role, VPC ID, security group, and subnet attached to the cluster.

2. Add nodes to your cluster.

A Kubernetes node is a machine that runs containerized applications. Each node has the following components: a container runtime, kubelet, and kube-proxy.
Amazon EKS Cluster can schedule pods on any combination of self-managed nodes, Amazon EKS managed node groups, and AWS farmgate. But we will be working with AWS EKS-managed node groups. Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters. With Amazon EKS managed node groups, you don’t need to separately provision or register the Amazon EC2 instances that provide compute capacity to run your Kubernetes applications.

• Create an IAM role for the nodes.

## worker-role.tf
resource "aws_iam_role" "worker_role" {
  name = "Worker-node-role"

  assume_role_policy = jsonencode({
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
    Version = "2012-10-17"
  })
}

resource "aws_iam_role_policy_attachment" "workernode_AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.worker_role.name
}

resource "aws_iam_role_policy_attachment" "workernode_AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.worker_role.name
}
resource "aws_iam_role_policy_attachment" "aws_eks_cni_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.worker_role.name
}

The above .tf file creates an IAM role and policies that allows EC2 managed instances to assume the role. some of the permissions that was attached include;

• “AmazonEKSWorkerNodePolicy”: Permissions for the kubelet to describe Amazon EC2 resources in the VPC.
• “AmazonEC2ContainerRegistryReadOnly”: Permissions for the kubelet to use container images from Amazon Elastic Container Registry (Amazon ECR).
• “AmazonEKS_CNI_Policy”: provide permissions for the VPC CNI on the instance role.
• Create a security group for the node.

# sg.tf
# Worker node security group

resource "aws_security_group" "eks_nodes" {
  name        = var.nodes_name
  description = "Security group for all nodes in the cluster"
  vpc_id      = var.vpc_id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
        Name                                        = "node-sg"
        "kubernetes.io/cluster/${var.eks_cluster_name}" = "owned"
  }
}

resource "aws_security_group_rule" "nodes" {
  description              = "Allow nodes to communicate with each other"
  from_port                = 0
  protocol                 = "-1"
  security_group_id        = aws_security_group.eks_nodes.id
  source_security_group_id = aws_security_group.eks_nodes.id
  to_port                  = 65535
  type                     = "ingress"
}

resource "aws_security_group_rule" "nodes_inbound" {
  description              = "Allow worker Kubelets and pods to receive communication from the cluster control plane"
  from_port                = 1025
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eks_nodes.id
  source_security_group_id = aws_security_group.eks_cluster.id
  to_port                  = 65535
  type                     = "ingress"
}

This .tf file creates an AWS security group for EKS worker nodes, allowing internal communication among nodes and inbound communication from the EKS cluster control plane on specified ports (1025–65535). The security group also has an egress rule allowing all outbound traffic.

• Create the managed node group.

# eks-nodegroup.tf
# Nodes in private subnets
resource "aws_eks_node_group" "private" {
  cluster_name    = aws_eks_cluster.eks_project.name
  node_group_name = "eks_private_node"
  node_role_arn   = aws_iam_role.worker_role.arn
  subnet_ids      = var.private_subnet_ids

  ami_type       = var.ami_type
  disk_size      = var.disk_size
  instance_types = var.instance_types

  scaling_config {
    desired_size = var.private_desired_size
    max_size     = var.private_max_size
    min_size     = var.private_min_size
  }

  tags = {
    Name = "eks_private_node"
  }
  depends_on = [
    aws_iam_role_policy_attachment.workernode_AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.aws_eks_cni_policy,
    aws_iam_role_policy_attachment.workernode_AmazonEC2ContainerRegistryReadOnly,
  ]
}

# Nodes in public subnet
resource "aws_eks_node_group" "public" {
  cluster_name    = aws_eks_cluster.eks_project.name
  node_group_name = "eks_public_node"
  node_role_arn   = aws_iam_role.worker_role.arn
  subnet_ids      = var.public_subnet_ids

  ami_type       = var.ami_type
  disk_size      = var.disk_size
  instance_types = var.instance_types

  scaling_config {  
    desired_size = var.public_desired_size
    max_size     = var.public_max_size
    min_size     = var.public_min_size
  }

  tags = {
    Name = "eks_public_node"
  }
  depends_on = [
    aws_iam_role_policy_attachment.workernode_AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.aws_eks_cni_policy,
    aws_iam_role_policy_attachment.workernode_AmazonEC2ContainerRegistryReadOnly,
  ]
}

The above terraform script defines two AWS EKS node groups, “eks_private_node” and “eks_public_node,” each specifying node configurations, scaling config, and dependencies on IAM role policies that were created.

# var.tf
# Defined variables
variable "eks_cluster_name" {
  description = "EKS Project"
  type = string
  default = "EKS_project"
}
variable "cluster_name" {
   description = "cluster SG"
   type = string
   default = "EKS_project"
}

variable "nodes_name" {
    description = "node SG"
    type = string
    default = "EKS_nodegroup"
}

variable "subnet_ids" {
    description = "list of subnet"
    type = list(string)
# Input the subnets ID 
    default = [ "subnet-0239e12834f028fcc", "subnet-0744d7de015a3ff50", "subnet-043f798c11f22305d", "subnet-0239e12834f028fcc", "subnet-0744d7de015a3ff50", "subnet-043f798c11f22305d"]
}
variable "vpc_id" {
    description = "vpc id"
# Existing VPC ID 
    default     =  "vpc-056d89704eafe3921"
    type = string
}
# Public subnet ID
variable "public_subnet_ids" {
    description = "List of public subnet IDs"
    type = list(string)
    default  = [ "subnet-0239e12834f028fcc", "subnet-0744d7de015a3ff50", "subnet-043f798c11f22305d" ]
}
# Private subnet ID
variable "private_subnet_ids" {
    description = "List of private subnet IDS"
    type = list(string)
    default  = [ "subnet-0239e12834f028fcc", "subnet-0744d7de015a3ff50", "subnet-043f798c11f22305d"]
}
variable "ami_type" {
  description = "Type of Amazon Machine Image (AMI) associated with the EKS Node Group. Defaults to AL2_x86_64. Valid values: AL2_x86_64, AL2_x86_64_GPU."
  type = string 
  default = "AL2_x86_64"
}

variable "disk_size" {
  description = "Disk size in GiB for worker nodes. Defaults to 20."
  type = number
  default = 20
}

variable "instance_types" {
  type = list(string)
  default = ["t3.medium"]
  description = "Set of instance types associated with the EKS Node Group."
}

variable "private_desired_size" {
  description = "Desired number of worker nodes in private subnet"
  default = 1
  type = number
}

variable "private_max_size" {
  description = "Maximum number of worker nodes in private subnet."
  default = 1
  type = number
}

variable "private_min_size" {
  description = "Minimum number of worker nodes in private subnet."
  default = 1
  type = number
}

variable "public_desired_size" {
  description = "Desired number of worker nodes in public subnet"
  default = 1
  type = number
}

variable "public_max_size" {
  description = "Maximum number of worker nodes in public subnet."
  default = 1
  type = number
}

variable "public_min_size" {
  description = "Minimum number of worker nodes in public subnet."
  default = 1
  type = number
}

output "endpoint" {
  value = aws_eks_cluster.eks_project.endpoint
}

Use these commands to check syntax validations and to provision the EKS cluster.

terraform init

terraform validate

terraform plan

terraform apply

kindly note that this step will take around 15–20 minutes to complete.

To interact with your cluster, run this command in your terminal:

aws eks --region us-east-2 update-kubeconfig --name AWS_EKS_Project

Next, run kubectl get node to confirm if the nodes are ready

Compiled terraform configuration files are in this GitHub repo.

That’s it for our series on setting up AWS EKS Cluster on AWS using Terraform. If your cluster isn’t required, run Terraform destroy.

I hope this helps!

UP NEXT

• How to configure and manage authentication and authorization in AWS Elastic Kubernetes Service.
• How to set up prometheus, Grafana and alert manager on EKS cluster using Helm chart