Its always Wednesday ma dudes !

Junaid Subhani


# Karpenter !

[Karpenter](https://karpenter.sh/) is an opensource tool that allows you to auto-scale your kubernetes cluster. Initially I was using cluster-autoscaler to provision nodes on demand but I encountered the following pain points with it

1. Cluster autoscaler relies on underlying node groups. I need to create a node group, associate the relevant IAM policies to it, dimension it accordingly. I manage an EKS cluster for multiple customers, each having their own node group. Everytime I onboard a new clustomer or offboard an old one, i need to update my Terraform code.
2. Everytime a customer requirements are updated, I need to update certain configurations in the terrafrom code i.e update instance type or tags
3. If a nodegroup is being updated, certain issues were found while spawning up new nodes.

Karpenter takes away all of these issues.

1. No need to create nodegroup. You only need to have one nodegroup which acts as your core node group with a minimum of 2 nodes for HA. This nodegroup hosts all system crititcal pods like CNI, kube-proxy and ofcourse Karpenter pods.
2. No need to do homework on how to dimension the nodes. Based on the workload requirment, Karpenter will find the appropriate instance type and launch it to host your workload.
3. If Karpenter finds a cheaper node to host you workload, it will switch your workload there hence reducing costs

## Deployment

Lets get to deploying Karpenter. My use case is simple. I have an EKS cluster on which I want to run resources for 2 customers. Each customer has their own setup of resource requirments, each set of IAM policies to work with. I'll call them customer Alpha and customer Beta.

For the sake of this tutorial, here are some of the values used


| Option | Description                                                               |
| ------ | ------------------------------------------------------------------------- |
| EKS cluster name  | acme |
| AWS account ID | `1234567890`    |
| OIDC URL    | `oidc.eks.ca-central-1.amazonaws.com/id/C99F50C4A2A9B6C3516F9BB73F3237D7`                                     |
| Default InstanceProfle | `default-acme`|



### EKS configurations

Certain configs needs to be done on the EKS cluster. If you are using the EKS module, the following configs need to part of your Terraform module

#### Security Group

The below config will create a Security Group rule to allow traffic so that Karpenter can talk to the cluster API
```json
  node_security_group_additional_rules = {
    ingress_nodes_karpenter_port = {
      description                   = "Cluster API to Node group for Karpenter webhook"
      protocol                      = "tcp"
      from_port                     = 8443
      to_port                       = 8443
      type                          = "ingress"
      source_cluster_security_group = true
    }
  }
```

The below config adds a tag to the security group. This is how Karpenter will discover the security group

```json
node_security_group_tags = {
    "karpenter.sh/discovery" = "acme"
  }
```

We will be leveraging [IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) so that the kubernetes service account deployed in an EKS cluster can assume the permissions of an AWS IAM role.

### IAM 
```
module "karpenter_irsa" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version = "5.3.1"
  providers = {
    aws = aws.dts-dev
  }

  role_name                          = "karpenter-controller-acme"
  attach_karpenter_controller_policy = true

  karpenter_tag_key               = "karpenter.sh/discovery/acme"
  karpenter_controller_cluster_id = module.eks.cluster_id
  karpenter_controller_node_iam_role_arns = [
    module.eks.eks_managed_node_groups["core"].iam_role_arn
  ]

  oidc_providers = {
    ex = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["karpenter:karpenter"]
    }
  }
}

```

The above module does the following:

1. Create an IAM role for Karpenter
2. Associate permissions to the IAM role

```json
{
    "Statement": [
        {
            "Action": [
                "pricing:GetProducts",
                "ec2:DescribeSubnets",
                "ec2:DescribeSpotPriceHistory",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeLaunchTemplates",
                "ec2:DescribeInstances",
                "ec2:DescribeInstanceTypes",
                "ec2:DescribeInstanceTypeOfferings",
                "ec2:DescribeImages",
                "ec2:DescribeAvailabilityZones",
                "ec2:CreateTags",
                "ec2:CreateLaunchTemplate",
                "ec2:CreateFleet"
            ],
            "Effect": "Allow",
            "Resource": "*",
            "Sid": ""
        },
        {
            "Action": [
                "ec2:TerminateInstances",
                "ec2:DeleteLaunchTemplate"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:ResourceTag/karpenter.sh/discovery/acme": "acme"
                }
            },
            "Effect": "Allow",
            "Resource": "*",
            "Sid": ""
        },
        {
            "Action": "ec2:RunInstances",
            "Condition": {
                "StringEquals": {
                    "ec2:ResourceTag/karpenter.sh/discovery/acme": "acme"
                }
            },
            "Effect": "Allow",
            "Resource": [
                "arn:aws:ec2:*:1234567890:security-group/*",
                "arn:aws:ec2:*:1234567890:launch-template/*"
            ],
            "Sid": ""
        },
        {
            "Action": "ec2:RunInstances",
            "Effect": "Allow",
            "Resource": [
                "arn:aws:ec2:*::image/*",
                "arn:aws:ec2:*:1234567890:volume/*",
                "arn:aws:ec2:*:1234567890:network-interface/*",
                "arn:aws:ec2:*:1234567890:instance/*",
                "arn:aws:ec2:*:1234567890:subnet/*"
            ],
            "Sid": ""
        },
        {
            "Action": "ssm:GetParameter",
            "Effect": "Allow",
            "Resource": "arn:aws:ssm:*:*:parameter/aws/service/*",
            "Sid": ""
        },
        {
            "Action": "iam:PassRole",
            "Effect": "Allow",
            "Resource": [
                "arn:aws:iam::1234567890:role/core"
            ],
            "Sid": ""
        }
    ],
    "Version": "2012-10-17"
}
```

3. Build a Trust Relationship i.e. configure a Kubernetes service account to assume an IAM role.

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::1234567890:oidc-provider/oidc.eks.ca-central-1.amazonaws.com/id/C99F50C4A2A9B6C3516F9BB73F3237D7"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.ca-central-1.amazonaws.com/id/C99F50C4A2A9B6C3516F9BB73F3237D7:sub": "system:serviceaccount:karpenter:karpenter",
                    "oidc.eks.ca-central-1.amazonaws.com/id/C99F50C4A2A9B6C3516F9BB73F3237D7:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}
```

We also need to create the default InstanceProfile. This is what is specific in the Karpenter config. An InstanceProfile maps to one IAM role. We will map it to the IAM role associated with the `core` node group.

```json
resource "aws_iam_instance_profile" "default" {
  name = "default-acme"
  role = module.eks.eks_managed_node_groups["core"].iam_role_name
}
```

where `oidc.eks.ca-central-1.amazonaws.com/id/C99F50C4A2A9B6C3516F9BB73F3237D7` is the OpenID connect URL of the EKS cluster.

By this point we have the following done

1. EKS cluster created with a default `core` nodegroup with 3 nodes
2. A security group allowing traffic on port `8443` from cluster to the nodegroup
3. Security group tagged properly
4. Relevant IAM roles/policies for Karepneter created and trust relationship between IAM role and kubernetes service account crerated.
5. Default instance profile created


This tutorial walks you through on how to install Karpenter in AWS. The AWS IAM permissions will be handled using Terraform while the Karpenter app itself will be deployed using Helm

Deploying Karpenter in AWS using Terraform and Helm


Something has always existed. According to physics, there can never be true physical nothingness—though there can be times when existence resembles nothing, such as a vacuum (the state of minimum possible energy) (Phys.org). Creating a space where there are no quantum fluctuations requires an enormous amount of energy, and there would be a remnant of that energy in that space afterwards if the fluctuations were flushed out, plus an unstable environment (1veritasium). Even on computers, deleted data is not actually tossed away, by rather written over.

# h1 Heading

## h2 Heading

### h3 Heading

#### h4 Heading

##### h5 Heading

###### h6 Heading

## Horizontal Rules

---


## Emphasis

**This is bold text**

**This is bold text**

_This is italic text_

_This is italic text_

~~Strikethrough~~

## Blockquotes

> Blockquotes can also be nested...
>
> > ...by using additional greater-than signs right next to each other...
> >
> > > ...or with spaces between arrows.

## Lists

Unordered

- Create a list by starting a line with `+`, `-`, or `*`
- Sub-lists are made by indenting 2 spaces:
  - Marker character change forces new list start:
    - Ac tristique libero volutpat at
    * Facilisis in pretium nisl aliquet
    - Nulla volutpat aliquam velit
- Very easy!

Ordered

1. Lorem ipsum dolor sit amet
2. Consectetur adipiscing elit
3. Integer molestie lorem at massa

4. You can use sequential numbers...
5. ...or keep all the numbers as `1.`

Start numbering with offset:

57. foo
1. bar

## Code

Inline `code`

Indented code

    // Some comments
    line 1 of code
    line 2 of code
    line 3 of code

Block code "fences"

```
Sample text here...
```

Syntax highlighting

```js
var foo = function (bar) {
  return bar++;
};

console.log(foo(5));
```

## Tables

| Option | Description                                                               |
| ------ | ------------------------------------------------------------------------- |
| data   | path to data files to supply the data that will be passed into templates. |
| engine | engine to be used for processing templates. Handlebars is the default.    |
| ext    | extension to be used for dest files.                                      |

Right aligned columns

| Option |                                                               Description |
| -----: | ------------------------------------------------------------------------: |
|   data | path to data files to supply the data that will be passed into templates. |
| engine |    engine to be used for processing templates. Handlebars is the default. |
|    ext |                                      extension to be used for dest files. |

## Links

[link text](http://dev.nodeca.com)

[link with title](http://nodeca.github.io/pica/demo/ "title text!")

Autoconverted link https://github.com/nodeca/pica (enable linkify to see)


Something has always existed. According to physics, there can never be true physical nothingness—though there can be times when existence resembles nothing, such as a vacuum (the state of minimum possible energy) (Phys.org). Creating a space where there are no quantum.

Its always Wednesday ma dudes !

Deploying Karpenter in AWS using Terraform and Helm

How to create SVG-ready icon symbols in Sketch PART 2