vSphere with Tanzu: The fastest route to a modern infrastructure

Update note, Nov. 8, 2021 – This blog post was originally written for vSphere update 1. In more recent versions of vSphere with Tanzu, the HAProxy functionality has been replaced by the NSX Advanced Load Balancer essentials edition included with Tanzu Basic licensing.

Using Octant to observe Tanzu Kubernetes Grid Clusters on vSphere with Tanzu

by Billy Downing

Overview

VMware has made some big announcements recently — specifically, the addition of vSphere with Tanzu released with the vSphere 7 update 1. This new capability will finally afford customers the ability to spin up upstream-conformant Kubernetes clusters natively in vSphere alongside their traditional virtual-machine workloads. This will support gradual application modernization without all the external technical requirements. How does this translate? Tanzu Basic licensing will give you the option to deploy Kubernetes clusters on top of vSphere in a native and vendor-supported way, eliminating the need for NSX-T, vSAN, or VMware Cloud Foundation as a whole. VMware has deemed this the fastest route to a modern infrastructure initiative within an organization, without having to redesign, re-architect, or relearn your existing environment.

In our environment, we won’t be using any of the VCF components outside of vSphere itself. Our goal is to hand power to the developers and give them a Kubernetes-as-a-service on-premises that relies on the existing vSphere investments.

How is this accomplished? vSphere with Tanzu can now provide built-in functionality to deploy and manage Tanzu Kubernetes clusters in a declarative manner. Developers can provision their clusters on top of vSphere by feeding the Kubernetes Application Programming Interface (API) the appropriate manifest to realize the creation of Tanzu Kubernetes clusters. These clusters will use a pre-deployed and included in the basic license NSX Advanced Load Balancer (ALB) control-plane (Avi networks) to front-end load all balancing requirements, and Antrea as node-to-node networking within the cluster.

When the clusters are deployed, they are complete Cloud Native Computing Foundation (CNCF) qualified clusters that can be inspected using Sonobuoy, or similar tools, for upstream conformance. Developers can write their applications once, and deploy anywhere, including in the existing vSphere environment.

How does this appear in the vSphere User Interface (UI)? Figure 1 displays the environment with the Tanzu Kubernetes service enabled, a namespace called ‘DevOps,’ which houses a three-node Tanzu Kubernetes Cluster deployed through a typical Kubernetes manifest, using custom resource definitions. Towards the bottom of the right inventory pane, there is also the HAProxy (nb-Haproxy) virtual machine (VM) deployed to front-end the Kubernetes API for the clusters deployed, which has been replaced as the default load balancer by the NSX advanced Load Balancer in later version of vSphere with Tanzu.

This provides the service of type LoadBalancer for applications. We will look at this further down the line. Finally, we see the three virtual machines, SupervisorControlPlaneVM(1-3), that make up our Kubernetes control-plane to facilitate the creation of the Tanzu Kubernetes clusters as first-class objects.

figure 1: Environment with Tanzu Kubernetes service enabled

Using this control plane for Kubernetes clusters is the same as the way vCenter manages ESXi clusters. vCenter to a cluster of ESXi nodes is very similar to the relationship the SupervisorControlPlaneVM’s have with each Kubernetes cluster. vCenter manages the underlying cloud platform that supports the raw aggregate of resources (compute, storage, network) while the Supervisor cluster supports the applications platform in Kubernetes. This is a similar design practice to how several public cloud-offerings supply Kubernetes as a service. Essentially, there are two control-planes: one for the platform (vCenter) and another for applications (Kubernetes). Figure 2 shows the cluster is functioning using the standard Kuberctl Command Line Interface (CLI) tool.

figure 2: Kubectl view of the Tanzu Kubernetes Cluster

Problem

A caveat to running Tanzu Basic in this way is that NSX-T is required to run vSphere Pods, or, Kubernetes native pods directly on top of ESXi. For most applications, bare-metal pods are not a requirement. This inability does limit our visibility into the application running on the containers within the vSphere interface. All of the information about the clusters themselves is shown (Figure 3), however, as seen in Figure 4, we cannot see the actual objects.

figure 3: Cluster visibility within vSphere

figure 4: No deployment visibility

Solution

The issue presented has a myriad of solutions. This discussion is about the quickest way to realize the benefits of Kubernetes using the tools we have at our disposal today, without deploying new services such as the vRealize suite. For this, we will use the open-source (VMware is a large contributor) dashboard Octant. Octant is a simple tool that allows us to run a web-based dashboard for in-depth visibility into our clusters directly from our local machines. Deployment of new virtual machines is not required, nor is configuring your clusters in such a way that they must be compliant with Octant. Rather, Octant can be downloaded and ran on your local desktop in minutes and use the already existing Kubeconfig files to authenticate and visualize your cluster. Figure 5 shows the basic overview of our cluster provided by Octant. We can see the deployments that exist in our cluster that were not present in the native vSphere UI. Additionally, near the left of Figure 4, there’s a drop-down of all the various clusters and contexts pulled from the kubeconfig file on the desktop, allowing you to view the metrics and health of all clusters we currently have access to.

figure 5: Octant visibility into our Tanzu Kubernetes Cluster

Usage

From here, upgrading the vSphere environment to version 7 update 1, and consuming the Tanzu Basic licenses, we have allowed our developers to self-provision a Tanzu Kubernetes Cluster that is lifecycle-managed by the Supervisor cluster. This, in turn, is managed by our vCenter deployment. Now that we have downloaded and utilized Octant for visualizing our Tanzu Kubernetes cluster and the application it is hosting, we can dig a little deeper into Octant as a tool. We want to not only observe but also troubleshoot and deploy.

Towards the bottom of Figure 5, we see a deployment that indicates health issues. From this view alone, we might not be familiar with all the objects associated with this specific deployment, and therefore would have to parse through CLI output to understand what components could be failing. Octant can provide us a diagram-based visualization of the service to better understand the sprawl of each of our services. Figure 6 shows the diagram produced by Octant to help understand the service.

figure 6: Diagram of a micro-service within our Tanzu Kubernetes Cluster

Now that all the various objects interacting with our micro-service have been identified, we can be sure of the issue and know where to find it. Since we have the pods tied to the deployment here, using Octant will dig into logs to figure out the issue. Figure 7 displays how easy it is to comb the logs of our pods and containers running in the Tanzu Kubernetes Cluster to understand our root cause. This is seen here because of the demo application that is the standard Google provided ‘hipster shop’. There seems to be a problem with that container missing environment variables specific to the Google Cloud Platform.

figure 7: logs of impacted Pods

While this single micro-service is showing errors, all others seem to work fine. We can still view our application, even if it is in a limited state with problems in the service, again, using the Octant to discover the front-ended IP address that is allocated within HAProxy for our application. Figure 8 shows the IP allocated and confirms that in Figure 9 by cross-referencing the actual configuration file in our HAProxy virtual machine to match the IP address of the service frontend-external-2, 10.150.165.134. (this was automatically built when we instantiated the service of type LoadBalancer for our application). Since HAProxy is being replaced by the NSX ALB, in later releases you will be able to cross-reference IP information directly using the NSX ALB interface and IPAM features.

figure 8: Services view from Octant

figure 9: Services built within the HAProxy VM by the Tanzu Kubernetes Cluster

Finally, we can be sure the application is still running by accessing it through the HAProxy service of type LoadBalancer, as shown in Figure 10.

figure 10: Hipster Shop Demo Application in our Tanzu Kubernetes Cluster

Next Steps

By now, we have successfully given our developers or applications teams the ability to:

Provision a Tanzu Kubernetes cluster for testing/deploying container-based modern applications
Deploy a modern, cloud-native application to their cluster
Visualize and troubleshoot their application using Octant

Next, we start containerizing applications and migrating them to Tanzu Kubernetes clusters and beyond.

All of this was accomplished by adding the Tanzu Basic licensing to our existing vSphere investments once they have reached version 7 update 1. Providing developers, the option to test and deploy bleeding-edge applications in tandem with existing virtual machine infrastructure, without impacting production workloads, ensures the flexibility required to initiate your cloud-native, application modernization journey. This can all be done on your own schedule without having to retrain your infrastructure team using their existing vSphere expertise.

Summary

Kubernetes has become the de-facto standard for managing and orchestrating container-based applications while providing a common API and infrastructure abstraction for application portability. vSphere with Tanzu is the fastest way for existing vSphere customers to build and deploy Kubernetes as a Service in a vSphere native manner. Kubernetes is a complex engine that provides countless benefits to applications running within. However, VMware has taken much of that complexity and integrated it into their existing industry-tested platform to help ease organizations into the upcoming transition to cloud-native applications. Sterling is here to help along the way. Deploying Kubernetes clusters is only a small part of the overall application modernization journey. The Sterling Cloud engineering team has invested extensive time devoted to education to ensure we can provide to our customers with the smoothest transition into cloud-native architectures. This has allowed businesses to realize the benefits of modern platforms as quickly as possible.

References:

Octant Github: https://github.com/vmware-tanzu/octant