Getting started with Kubernetes Cluster API (CAPI)

In our previous blog post, we delved into the concept of Cluster API (CAPI) and its potential to simplify Kubernetes cluster management. We explored its principles, benefits, and how it streamlines the lifecycle of Kubernetes clusters.

In this sequel, we'll embark on a hands-on journey to harness the power of Cluster API for real-world Kubernetes cluster management. We'll provide step-by-step guidance, practical examples, and best practices for putting Cluster API to work in your environments.

Understanding the basics of Cluster API

Let’s recap some basics. CAPI is a project maintained by the CNCF special interest group (SIG) Cluster Lifecycle Management. 

Just as Kubernetes uses declarative APIs to manage containers, Cluster API introduces declarative Kubernetes-style APIs enabling you to create and manage clusters themselves. 

It enables consistent and repeatable cluster deployments across a wide variety of infrastructure environments.

Setting up the environment 

Before using Cluster API, you need to set up your environment. You must ensure that you have the necessary environments, such as GCP or AWS accounts and Kubernetes clusters, and the credentials required to access these resources. 

Follow the steps below to set up your environment for Cluster API.

Install Kubernetes on your machine

If you do not already have Kubernetes installed on your machine, you can install it by following the guide on the official documentation for Kubernetes.

A Kubernetes cluster is a prerequisite for using Cluster API. This Kubernetes cluster is called the management cluster, and it does not need to be production-ready. Some of the popular ways to create and deploy a management cluster are by creating Kind or Minikube clusters. 

Install Clusterctl and Kubectl

Clusterctl installs necessary CLI tools, Kubernetes manifests, and the required CRDs on the management cluster.

Kubectl is a command-line tool used for interacting with Kubernetes clusters. It is the primary interface for managing and controlling Kubernetes resources and workloads. 

For MacOS, you can use Homebrew to install Kubectl and Clusterctl; for instance, run the command `brew install clusterctl` to install Clusterctl or `brew install kubectl` to install Kubectl.

Alternatively, you can download Clusterctl and Kubectl directly as a binary for other platforms.

To install Kubectl, use the guide on the Kubernetes documentation. Similarly, install Clusterctl using the steps provided in the official documentation.

Choosing your providers 

Cluster API supports three types of providers: infrastructure, bootstrap, and control plane providers.

Infrastructure providers

Infrastructure providers provision and manage the underlying infrastructure required for running Kubernetes clusters. 

They handle tasks such as creating infrastructure like virtual machines, managing networking resources, and configuring storage. 

The Cluster API project supports multiple infrastructure providers, allowing users to choose their preferred cloud or virtualization platform. Some examples of infrastructure providers include AWS, Azure, OpenStack and vSphere.

Bootstrap providers

Bootstrap providers are responsible for bootstrapping the control plane nodes of the Kubernetes cluster. 

They handle tasks such as installing the required software components and configuring the initial cluster state. 

Cluster API supports multiple bootstrap providers to cater to different use cases. Some examples of bootstrap providers include Kubeadm, Kubeadm-bootstrap, etc.

Control plane providers

Control plane providers manage the Kubernetes control plane components, such as the API server, scheduler, and controller manager. They handle tasks such as scaling the control plane, upgrading components, and handling control plane failures. 

Cluster API supports different control plane providers. Some examples of control plane providers include KubeadmControlPlane and ClusterAPI Provider for Docker (CAPD).

By combining different infrastructure, bootstrap, and control plane providers, Cluster API offers flexibility and choice to users in managing their Kubernetes clusters across various cloud and virtualization platforms. 

The infrastructure provider is a compulsory component of the Cluster API setup, whereas bootstrap and control plane providers are optional. The bootstrap and control plane providers default to Kubeadm when not explicitly specified at setup. 

Kubeadm (Kubernetes Administration) is the most popular tool that helps you bootstrap, deploy, and manage a Kubernetes cluster. It focuses on the initial deployment and basic cluster management tasks; hence, more advanced configurations and operations may require other Kubernetes tools and resources.

Setting up your infrastructure 

For this post, we will set up a Cluster API with GCP and AWS providers. We therefore need to generate or collect necessary credentials such as:

• GCP Project ID
• GCE JSON key
• AWS Access Key ID
• Secret Access Key

You’ll need these to interact with Google and Amazon through the Kubernetes operator and Cluster API controller manager. 

Configuring the Cluster API Provider for Google Cloud (CAPG) or Cluster API Provider for AWS (CAPA) typically involves providing specific environment variables or configuration settings. 

While there isn't a standard "env file" that covers all CAPG or CAPA configurations, you can set environment variables in various ways, such as through a shell script or Kubernetes manifest. 

Here's a general example of how you might configure CAPG or CAPA using environment variables in a shell script:

Create a shell script containing the CAPG or CAPA configuration

For GCP, create a shell file capg-config.sh, list your CAPG configuration variables. An example is shown below:

 
#!/bin/bash

#CAPG configuration
export GCP_PROJECT="MY_PROJECT_ID"
export SERVICE_ACCOUNT_KEY_PATH="/path/to/service-account.json"
export GOOGLE_APPLICATION_CREDENTIALS "/path/to/google-application-credentials.json"
export CLUSTER_NAME="my-cluster"
export ZONE="us-central1-a"
export NODE_COUNT=3
export GCP_REGION "us-central1"
export KUBERNETES_VERSION "[Kubernetes version]"


# Optionally, other configuration variables

# Run your CAPG commands using the above environment variables

If your preferred cloud provider is AWS, create a shell file capa-config.sh and add the environment variables for your AWS environment using the example below as a guide:

 
#!/bin/bash

export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY"
export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_KEY"
export AWS_REGION="us-west-2"
export CONTROL_PLANE_MACHINE_COUNT=1
export WORKER_MACHINE_COUNT=3
export KUBERNETES_VERSION="v1.21.0"
export CLUSTER_NAME="my-cluster"
export NODE_INSTANCE_TYPE="t3.medium"

# Optionally, other configuration variables
# Run your CAPA commands using the above environment variables

Run the script

Make the script executable and run the script as shown below.

 
# chmod -x [config-file-name.sh] for example
chmod -x capg-config.sh

# ./[config-file-name.sh] for example
./capg-config.sh

Note: The example above is for illustrative purposes. Actual CAPG or CAPA configurations and environment variable names may vary based on the version of CAPG or CAPA and the specifics of your setup. 

Always check the official documentation and resources for accurate and up-to-date information on how to configure CAPG or CAPA for your cloud environment.

Setting up provider components

This typically involves installing CRDs, controllers, and a set of custom resources that drive the provider. 

Run the following commands to install the provider components for AWS and GCP, respectively.

For AWS: 

 
clusterctl init --infrastructure aws

For GCP: 

 
clusterctl init --infrastructure gcp

Creating and deploying the cluster

Once you have set up your environment and installed the providers, you can move on to cluster creation. 

To create a workload cluster, you need to define a cluster object by specifying the desired state of the cluster. 

This includes specifications for the number of nodes, Virtual Machines, networks, load balancers and other required resources. 

Once created, the controller will ensure that the actual state matches the desired state.

You can now deploy your first Kubernetes cluster using Cluster API and the desired infrastructure provider. 

The command below deploys a Kubernetes cluster named `my-cluster` in the `us-west1` region and creates its corresponding manifest.

For AWS: 

 
clusterctl config cluster my-cluster --kubernetes-version v1.26.0 --control-plane-machine-count=1 --worker-machine-count=1 --region=us-west-1 > my-cluster.yaml; 
kubectl apply -f my-cluster.yaml

For GCP: 

 
clusterctl config cluster my-cluster --kubernetes-version v1.26.0 --control-plane-machine-count=1 --worker-machine-count=1 --zone=us-west1-b > my-cluster.yaml; kubectl apply -f my-cluster.yaml

Verifying that the cluster was created

Once the deployment is successful, you will need to verify whether the cluster exists or not.

Here you have to choose your approach. 

You can check that the cluster has been created by monitoring the status of the cluster and machines custom resources from the management cluster. An example of how to do this is shown below:

 
kubectl get clusters
kubectl get machinedeployments
kubectl get crd

You can also check the nodes with `kubectl get nodes`, but for that, you first need to download the kubeconfig file for the target cluster using `clusterctl get kubeconfig <name_of_the_target_cluster>` and switch context to it:

 
# Get the kubeconfig for the new cluster
clusterctl get kubeconfig my-cluster > my-cluster-kubeconfig.yaml

# Set KUBECONFIG to the new kubeconfig
export KUBECONFIG=my-cluster-kubeconfig.yaml

# Verify connectivity by checking the nodes
kubectl get nodes

Note: If you're running a larger multi-environment or multi-cloud setup, you will create one manifest per cluster, or setup. You will end up with a lot of yaml files and a lot of scripts, and maintaining these efficiently is something you need to take seriously.

Managing the cluster

After creating a cluster object, you can manage your cluster like any other Kubernetes resource. This includes scaling up or down your cluster, updating configurations, and creating new resources like Nodes and Storage. 

You can also use Kubernetes tools like Kubectl and Helm to manage the resources. Cluster API includes many tools, such as Clusterctl, that make it easier to manage your cluster.

If you're just starting with the concept of containerization, managing your cluster can prove difficult. Cluster API provides a unified way of defining, deploying, and managing Kubernetes clusters, making it easier to deploy and scale your containerized applications. 

Troubleshooting and best practices

Like any other Kubernetes resource, troubleshooting Cluster API can be challenging. 

There are many resources and documentation available online, including the official Cluster API documentation and Kubernetes Slack. 

If you're investigating Cluster API because you need to create and manage multiple clusters, there is an easier way. 

Instead of having to create bootstrap clusters and understand operators, you can use a tool like Palette that is based on Cluster API and can build your clusters (and all the integrations) for you.  Spectro Cloud, the organization behind Palette, actively contributes to CAPI providers, such as the Canonical MAAS provider for creating bare metal clusters. 

Conclusion

As the CAPI docs say, “Cluster API is a Kubernetes sub-project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters.”

By simplifying the management of Kubernetes resources, organizations like yours can focus more on development and less on cluster management.

In this post, we went through the basics of Cluster API, including setting up the environment, creating a cluster, managing the cluster, and troubleshooting. We also mentioned some management platforms, such as Palette, that help you use CAPI more efficiently at scale.

The foundation of Palette is Cluster API, andWe hope that this guide has been helpful to you. Happy clustering!