Governance for Platform Teams
Projects in Nutanix Cloud Manager are used by enterprises to provide self-service capabilities to consumers of IT infrastructure. With Projects, you can logically group infrastructure resources, say, for a team of engineers collaborating on a project. The administrator can then manage RBAC by assigning specific roles to the project users. In other words, projects can be considered logical entities enabling multi-tenancy by intelligently segmenting resources and encapsulating users and infrastructure.
One of the key values Projects provides is the ability to set quotas and enforce governance policies. This allows Nutanix administrators to have better visibility into resource consumption, ensuring project members do not exceed the defined limits. This is especially critical in Kubernetes environments where consumption can skyrocket due to container proliferation.
Further, the usage of Prism Central categories allows us to assign entities to various categories based on some criteria and apply storage and security policies to them. Resources within the same project can be assigned to different categories, allowing fine-grained policy mapping to these entities.
Projects and categories are supported for Red Hat OpenShift Container Platform deployments starting with OCP 4.13. Let’s delve further to find out how this works.
Prism Central Configuration
The screenshot below is from a Prism Central instance. We see that there are two projects created, “stranger-things” and “squid-game”. The DevOps team working on the “squid-game” project would like to spin up an OCP cluster on Nutanix for last-minute load testing of their media ranking application. (because, obviously, Season 2 will break the Internet).
The project is currently empty with no resources; however, there have been quotas pre-defined. The project has 40 vCPU, 100 GiB memory, and 800 GiB of storage allocated. Note that in the below example, we have only defined resource quota limits at the project level without defining limits for the associated Nutanix account or provider level.
Switching to Categories within the Infrastructure view, we can see there is a category key called OpenShift created. It has two values, squid-game and stranger-things. There are six VMs already assigned to the stranger-things category from the stranger-things project, while none are assigned to squid-game.
With this configuration in place, the team is now ready to create the cluster!
OpenShift Install Configuration
Use the familiar installer-provisioned infrastructure (IPI) method of OCP installation which automates the entire provisioning process. We must ensure that the latest OCP 4.13 installation program is used which has support for projects and categories.
Once the install-config.yaml file has been generated by the installer, we will have to add parameters to the file to specify the Project name.
platform:
nutanix:
defaultMachinePlatform:
categories:
- key: OpenShift
Value: squid-game
project:
type: name
name: squid-gameBelow is the YAML file that was used for this cluster. Note the usage of the “project” and “categories” parameters in the highlighted section.
For a complete list of all the possible parameters, please refer to the official OpenShift documentation.
After the cluster installation has finished, you can see additional details from the Projects view in Prism Central. The six VMs, three control planes, and three worker nodes have been added to the Project by the installer. Further, the installer would have also updated the quota consumption with above 90% usage for compute, memory, and storage.
If we filter out the ocp-nl VMs within the VM view, we see that all six of them have been assigned to the expected category and project. The DevOps team can only view the VMs created in their target project.
Note that at the time of writing this blog, the Nutanix Marketplace method of installation does not support projects and categories. The administrator will have to assign the existing OCP VMs to projects and categories manually if required to provide sufficient levels of isolation to the individual tenants.
Quota Violation
While testing, the DevOps team realized that three worker nodes are insufficient and require an additional node to scale their tests. All they have to do is to increase the MachineSet count to four, via UI or CLI, and OpenShift automatically creates the new VM using Nutanix APIs. There is built-in support for autoscaling, too. The MachineSet can automatically add or remove machines depending on the load.
However, by performing this action, the tenant would violate the quota limit set for the ‘squid-game’ Project.
The task fails, and a message is displayed on the Prism console. The request exceeded the quota by 12 GiB memory and 40 GiB storage space!
Everyone Wins!
This means that the IT administrators can monitor resources and manage Kubernetes/OpenShift infrastructure just the way they used to for traditional VMs. They continue to specify governance policies to adhere to compliance regulations with the Project construct. Developer and DevOps teams can self-service their infrastructure requests using their oc, kubectl, or other tools of choice without involving IT administration while IT still has visibility and control. The Nutanix platform’s infrastructure as code (IaC) capabilities and rich APIs make this possible.
In the above example, on receiving notifications or configurable alerts, the administrator can increase the quota limits for the project, allowing the additional worker node to be created. The DevOps team may continue working with minimal intervention.
By assigning the VMs to specific categories, policies are enforced immediately upon VM creation. For example, using Flow Network Security, we can create an isolation policy that can block all traffic between the OCP VMs in the category OpenShift: squid-game and VMs in the category OpenShift: stranger-things, while all the squid-game VMs can communicate with each other. Categories can also be extended to storage policies, where attributes like compression, encryption, and QoS can be applied to these VMs.
And more importantly, Operation Squid Game proceeds forward!
To learn more about OpenShift on Nutanix, take a free Test Drive here.