Simplify Kubernetes day 2 ops with Palette Cluster Profiles

Like many organizations, you’ve gone cloud native, adopted Kubernetes and deployed your workloads into production clusters. Now what? Managing, updating, and securing those Kubernetes clusters — in other words, day 2 operations — is naturally your top priority. 

In this blog we’ll discuss the differences between day-0, day-1, and day 2, shed more light on what happens is day-2 with Kubernetes and explore how Palette’s concept of Cluster Profiles can help you simplify and automate some essential day-to-day operations activities, such as Kubernetes cluster updates. 

What are difference between Day-0, Day-1, and Day-2 operations?

In the cloud-native world a “day” refers to the phases of the IT operations lifecycle. The main differences between Day-0, Day-1, and Day-2 of operations lie in their respective objectives and activities:

• Day 0 is the design phase and the this first stage of your application lifecycle. It is here where the development of the architecture and technologies (commercial or open source) you need to support the functional requirements of the applications you’re deploying takes place.
• Day 1 is the deploy phase and the second stage of your application lifecycle. In this stage, you actually stand up the environment, build the ci/cd pipeline, test and implement the application stack.
• Day 2 is the operate phase where the application shifts from a development project to a production environment, becoming available to its users. Changing an application or its environment once it’s live is typically more complex because you need to maintain availability. On day two of operations, we focus on scaling, optimizing performance, enhancing security measures, and fine-tuning system configurations based on initial feedback and observations.

Take a look at the following diagram to better understand some of the typical activities you might undertake in each phase, from Day 0 to Day 2. 

Check out our vintage blog post Kubernetes Lifecycle Management! So Important! (What Does It Mean?) to learn more about the different phases, and out blog posts that cover other Day-2 management aspects such as cost management, security practices, and more.

How can you make Kubernetes Day-2 ops easier?

As the complexity of your Kubernetes environments grows, Day-2 management becomes not just time-consuming, but challenging. 

According to the 2023 State of Production Kubernetes report, 75% of respondents say they suffer issues affecting the running of their clusters and 40% say they lack the skills and headcount to manage Kubernetes. 

If you don’t take control, you can risk the availability, performance and security of your application workloads. 

At Spectro Cloud we’ve built our Palette management platform to support you with many of the most important Day-2 activities. Take a look at the table for details.

Deploy and update a cluster with Palette for easy Day-2 ops

This “Deploy Cluster Profile Updates” tutorial gives you a hands-on insight into how Palette’s architecture supports Day-2 operations on Kubernetes clusters. It walks you through deploying and updating a cluster, highlighting how this critical Day-2 operation works with Palette. 

Let’s take a look at a real scenario you might encounter. 

You’ve just started a new job on the platform team of a midsize manufacturer, running multiple clusters globally. Your team is fully remote, distributed around the world and you find yourself working alone for a couple of hours. Suddenly, the application team pings you. They have rolled out a new version of the application and this has caused an outage. They need your help to roll back to get their clusters to a healthy state, without any downtime. 

This is just one example where you will need to know how to simplify your day 2 Kubernetes and learn how Palette’s Cluster Profiles can support.

Cluster Profiles are a central concept in Palette, as they allow you to specify layers for your workloads using packs, Helm charts, Zarf packages, or cluster manifests. Cluster Profiles are the declarative, full-stack models that Palette follows when it provisions, scales, and maintains your clusters, providing you with consistency across environments. 
Once you create your Cluster Profile, you can deploy it to a new cluster in various ways: using the Palette UI, the Spectro Cloud Terraform Provider, or the Palette CLI. You can learn how to create your first cluster in our “Deploy a Cluster” tutorial. Once created, clusters are maintained through Palette’s Day-2 operations capabilities. 

Cluster Profile layers give you the flexibility to easily make changes by removing, swapping or adding a new layer, either from the Spectro Cloud registries or your own custom ones. They can also be imported and exported in JSON format, making it easy to reuse them. 

Palette Cluster Profiles can be updated through different methods, as summarized in the table below.

Updating with Cluster Profile versions

The recommended way to update Cluster Profiles is by creating Cluster Profile versions. They are commonly used for adding, removing, or swapping layers, as well as pack configuration updates. Cluster Profile versions provide better change visibility and control over the layers in your host clusters. 

One of the key advantages of using Cluster Profile versions is that they make it possible to maintain a copy of previously known working states, serving as a library for rollback and audit purposes. The ability to roll back to a previously verified, working cluster profile in one action shortens the time to recovery in the event of an incident.

In the scenario we described above, you can roll back to the previous Cluster Profile version in one click, and Palette will bring your cluster to that state. This is possible through the declarative approach of Cluster Profiles, which ensures that your cluster is brought back to a consistent, known state once the reconciliation process is completed. 

The application developers can then fix their code, while your cluster returns to a previously known working state. Happy days! 

Once the fix is applied and you have verified it, you can then release the newer cluster profile version to all your clusters, regardless of the complexity of your infrastructure. Palette will monitor and maintain the state of your clusters, removing the need for error-prone manual effort.

Updating profiles in place

Cluster profiles can also be updated in place, without creating a new cluster profile version. The previous state of the cluster profile will not be saved once it is overwritten, so you do not have the same capability to roll back changes as previously.

Palette monitors the state of your clusters and notifies you when updates are available for your host clusters. This feature is beneficial to managing production environments that may contain many clusters, making it difficult to know which clusters are linked to a particular cluster profile. Clusters with pending updates that have not been applied yet are marked in the list of clusters. You may then choose to apply your changes at a convenient time.

Applying cluster-specific overrides

Finally, if you want to make cluster-specific changes, Palette provides cluster overrides. They allow you to apply cluster-level changes, but they do not offer the same audit trail as Cluster Profile versions. These changes persist even if you apply another cluster profile version. When applying updates, Palette highlights your overrides in the cluster profile differential viewer. This feature is beneficial for keeping track of and reviewing dynamic overrides in complex production environments. 

Looking back at our scenario, you could use cluster overrides to manually edit a manifest to point to another version of an application or set an environment variable that disables a particular feature flag. 

Conclusion

Kubernetes Day-2 operations can pose exciting engineering challenges. In this blog post, we've walked you through the differences of each stage in the operations cycle from da2-0, to day-1, to day-2 and we also introduced a powerful concept of a Palette Cluster Profile to make day two operations much easier.

Make sure to check out the “Deploy Cluster Profile Updates” tutorial for more details and hands-on steps on how to use the key features highlighted in this blog post. You will need a Palette account to successfully complete it — arrange a demo to get started.

We'd love feedback on this tutorial or any ideas about future tutorials you'd like to see. You can leave feedback directly in the tutorial through our feedback button or head to our Slack community to discuss directly with the Spectro Cloud team and other users.

Thanks for reading!