Kubernetes secret management using KMS

AWS Key management service (KMS) helps to easily create and manage cryptographic keys and control their use across a wide range of AWS services and in your applications. In this blog we'll walk through using KMS for Kubernetes secret management.

Advantages of KMS :

1. It is a managed service which is integrated with various other AWS Services.
2. It provides high availability, as keys are stored in multiple availability zones within a region.
3. If someone is writing his application or using any AWS services, he can control who can access your master keys and gain access to his data.
4. KMS is a global service but the keys are regional which means one can't send keys outside the region in which they are created.
5. One can audit, for what purpose, by whom and when a key was used, which helps to meet compliance and regulatory needs.

How does KMS work KMS uses envelope encryption, which has two different keys for protecting data. Generated by AWS or imported, the data key encrypts each piece of data and resources. Data and resources are then encrypted under a customer master key (CMK) defined in KMS and stored in AWS. When a user needs to decrypt data, the encrypted key is sent to KMS and decrypted with the CMK.Data KeysData keys are encryption keys that we can use to encrypt data, including large amounts of data and other data encryption keys. AWS KMS does not store, manage, or track our data keys, or perform cryptographic operations with data keys. We must use and manage data keys outside of AWS KMS. We can use AWS KMS customer master keys (CMKs) to generate, encrypt, and decrypt data keys or import keys from your own key management infrastructure.Envelope EncryptionWhen we encrypt our data, our data is protected, but we have to protect our encryption key. One strategy is to encrypt it. Envelope encryption is the practice of encrypting plaintext data with a data key, and then encrypting the data key under another key.We can even encrypt the data encryption key under another encryption key, and encrypt that encryption key under another encryption key. But, eventually, one key must remain in plaintext so that we can decrypt the keys and our data. This top-level plaintext key encryption key is known as the master key.

Difference between CMK(by user) and CMK created by AWSCustomer managed CMKs are CMKs in our AWS account that we create, own, and manage. We have full control over these CMKs, including establishing and maintaining their key policies, IAM policies, and grants, enabling and disabling them, rotating their cryptographic material, adding tags, creating aliases that refer to the CMK, and scheduling the CMKs for deletion.AWS managed CMKs are CMKs in our account that are created, managed, and used on your behalf by an AWS service that is integrated with AWS KMS. Some AWS services support only an AWS managed CMK. However, we cannot manage these CMKs, rotate them, or change their key policies.A common way to store secrets in kubernetes is to store a base64 encoded configmap. But, in that case, anyone can access the secrets from the configmap. As a way to restrict access to all the users, we can store the secrets in a file in an S3 bucket where only the admin can access it.Configuring the KMS with S3Encrypted data is finally stored in a storage that can be anything, e.g. EBS, EFS but the most used is Amazon S3

1. Create a KMS key from KEY MANAGEMENT SERVICE. Keys can be accessed from AWS console — IAM Role- keys
2. Choose the roles and users who can administer this key.
3. Tick the check box if you want administer to allow to delete these keys.
4. Now when uploading any file to your bucket choose the KMS key.

Accessing the S3 to get the secrets All the users who have been granted the permissions to access the S3 file in the IAM role can read the configs using aws sdk calls.

But in this scenario, we will have to use the AWS access and secret keys in the call and storing this secretly is a new challenge. One solution is to store the AWS secrets in the config map.IAM access to pods to access Amazon S3 Another solution to the above problem is to give the pods the access to read the configs from S3, using the IAM. Kube2iam and Kiam solves the IAM issue in kubernetes. Kube2iam works by deploying a kube2iam pod to every node in the cluster as a DaemonSet. It runs in the host network with privileged security. The pods on each node intercept calls to the API and instead of authenticating directly, will assume roles that are assigned to pods in the cluster via annotations, and respond with the temporary credentials from assuming that role.

How to setup Kube2iamThe setup process for kube2iam is very simple:

1. Create roles for your pods to assume
2. Add permission to your Kubernetes nodes to assume these roles
3. Set up RBAC for kube2iam
4. Deploy the kube2iam DaemonSet
5. Annotate your pods with iam.amazonaws.com/role: <role arn> and apply changes
6. Clean up your previous IAM access method once you have verified access via kube2iam

The detailed comparison of kube2iam and kiam can be accessed at :-https://www.bluematador.com/blog/iam-access-in-kubernetes-kube2iam-vs-kiam .With this, you should have a better understanding of how to use AWS's KMS for Kubernetes secret management. Keep your secrets secret!