Well, you might be able to improve it, but how do you know you’ve improved it? And how do you know the improvement was worth the effort if you can’t quantify it? When it comes to power consumption the first step to any reduction programme has to be to measure. This isn’t just true for Nutanix customers or even just IT, it’s true for just about anything in our modern data driven world and technology advances now mean that it should be easier than ever. But that’s not to say that it’s trivial. Furthermore, depending on your geographical location you could find yourself required to measure power consumption for compliance purposes.
Power consumption in the data center can broadly be measured in 3 ways:
- Data Center or Data Hall: For the whole facility. Can be via electricity bill or meter.
- Within “in rack” Devices: The IT Platform
- From the OOB (out of band management or IPMI). Via the user interface, programmatically using the RedFish API.
- “In-band” via the hypervisor or device (e.g. server) operating system (not VMs, physical devices only).
- Data Center facility equipment: At the PDUs, rack PDU (if available), UPS and other supporting equipment.
Here is a handy diagram to help visualize the above:
You can see that (1) is the sum of (2) and (3). Remember that the purpose of all of that data center facility equipment (3) is to deliver power to the devices in the racks that actually (hopefully?) add value to your organization. Achieving this efficiently is referred to as Power Usage Effectiveness (PUE) and is discussed in depth in this article.
The Monitoring and Measuring Power Consumption technote describes how (2) can currently be achieved in some detail for Nutanix Cloud Infrastructure (NCI) and has some general points to make around (1) and (3).
Top Tip: If your organization uses a co-location provider such as Equinix or Cyxtera, check with them to understand what power measurement or carbon reporting services they can provide. Some data center locations have advanced per device power monitoring services that utilize real time metric feeds from Smart PDUs.
Measure Twice, Cut Once
Is often the mantra when it comes to carpentry, but it’s worth keeping in mind for lots of other things (environment sizing!) including power.
Measuring more than once across your entire data center is preferable so you can be sure of your data and detect any anomalies or inefficiencies. It will help you understand which areas or specific pieces of hardware are drawing the most power and where improvements might be made.
Top-Tip: If your infrastructure is running within your own location, it’s possible the facility could already have a BMS (Building Management System) or PMS (Power Management System) that you could get useful information from. That information may go beyond power metrics for certain circuits / rooms / departments and may include temperature, footfall or other information that you can feed into your data analysis platform to gain sustainability driving insights. Ask your facilities staff for help, but be prepared to find out that the system sits on an old desktop PC under someone’s desk and is ripe for modernisation!
Get the Baseline
Baselining ahead of any improvement activity is essential so you can measure what has been achieved and justify future action. More granularity is always preferable, but you might need to compromise and make estimations in some cases, or use point in time measurements to estimate overall usage. Automated measurement and data normalization is always preferable. Automating the data collection process by feeding power consumption and performance information into a monitoring or analysis tool such as Splunk or nZero can allow you to demonstrate the results of your actions and better justify future optimisations. The more this can be done in real time, the more users will connect with it and drive optimization.
It’s important to get some kind of baseline sooner rather than later and balance this rather than spending a long time getting the perfect real time data stream.
Top Tip: As a starting point, if you can obtain your facility’s electricity bills you might be able to better understand the tariff and how much your organization is using and paying per kWh. Using that information in conjunction with usage data can help you demonstrate savings made and can be useful for building the case for future changes. Most utilities companies can email bills to a list of addresses so why not ask to be included?
Within “in rack” Devices: The IT Platform
The Monitoring and Measuring Power Consumption technote describes how Nutanix customers can make use of existing tools to track power consumption of their NX nodes and other HW platforms. It also discusses use of 3rd party tools from Nutanix’s hardware vendor partners.
Another approach (for 2 in the diagram above) that works with most Nutanix NX hardware is using the container based IPMI Exporter and its associated Grafana dashboards. You will need a basic understanding of containers but you could use the Nutanix Kubernetes Engine (NKE) or another Kubernetes flavor.
VM Level Metrics
Increasingly discussed is attributing power consumption and by extension carbon emissions to individual VMs or services, both in the data center and in the public cloud (for CO2e anyway). By feeding back individual workload consumption to application owners / IT consumers this can put the responsibility (and costs!) into their hands and drive improvement in much the same way as FinOps processes reduce cost through showback/chargeback/shameback.
Attributing power consumption to VMs can sound simple at a high level, but it is actually more complex the more granular you get. For example, if a VM created a lot of disk IO, but didn’t use too much of its allocated CPUs, how can that be balanced against a VM that created only a little IO, but had lots of memory and pushed its CPUs to their limits? There are various approaches and methodologies to this challenge all with their various pros and cons. When looking at an approach, it is wise to ask yourself a few questions:
- Does this methodology and the way the data is presented drive the right kind of behavior?
- What will happen to the baseline if we wish to improve this methodology?
- Does it cover all or most of the power consumed by the system?
- If being used for GHG (Greenhouse Gas) or CO2 (Carbon Dioxide) emissions does it:
- Cover all GHG Protocol Scopes 1, 2 and 3?
- Use credibly sourced carbon intensity grid factors approved by your organization
- Provide suitably granular results (time, location based) for your organization’s requirements
- Could the information fed back be misleading or misinterpreted when taken out of context and result in an unintended use?
- Can the results be replicated and provide an audit trail?
The gotcha of numbers 4, 5 and 6 are that the metrics may end up being used for compliance or disclosure use cases, rather than just driving internal optimisation which could lead to regulatory complications. Awareness of IT and the users of the metrics generated should be considered when
Conclusion
The important thing is to make a start, and hopefully by using Nutanix and reading this blog you’ve made that start and now have some ideas of how to progress, or at least what to further research. Also, reach out to your wider community on social platforms and talk to other IT practitioners about power consumption, reduction and sustainability. You could well find you have a lot of allies out there all looking at the same issues! This is the first, and probably most important post in the blog series: 7 Strategies for reducing power consumption and improving sustainability with the Nutanix Cloud Platform that aims to help customers reduce their consumption. We’re aiming to release a new part in this series every few weeks over the summer of 2023, but also keep them updated and relevant. As such please give us feedback via the Nutanix Community if you have any comments, suggestions or ideas!
See 7 Strategies for Reducing Consumption and Improving Sustainability: Introduction for a full list of content in this series.