Edge computing with Nutanix Karbon Platform Services (KPS) frees developers from the complexity of deploying edge devices and data streams and instead allows them to focus on creating useful data analysis and transformation logic through easy to use Kubernetes applications, reusable data pipelines, serverless functions, and pluggable machine learning architecture.
In this article, we will be discussing a major component of Karbon Platform Services: data interfaces which enable you to configure and deploy data sources in order to connect your IoT sensors, PLCs, and historians to our platform.
Karbon Platform Services Data Interface Model
KPS is a multi-cloud PaaS platform that allows you to easily create solutions for gathering data from remote sensors and edge appliances and perform real-time data processing on that data. A typical IoT solution starts with edge hardware, connected to sensors, generating data. The data streams (routed through protocol gateways as needed) are configured as data sources, which can be connected as input/output to applications or data pipelines for analysis and transformation. This is the component of KPS that will be focused on today.
Data Interface Basics
The KPS edge environment supports running custom data sources called data interfaces. On KPS, data interfaces are simply Docker containers running in a hosted Kubernetes environment. Data interfaces deployed as part of a KPS solution are primarily used to bridge the gap between IoT devices, services running on the platform, and external services on-prem or in the cloud like AWS Kinesis.
If you’re not already familiar with the concept of application containers, here is a quick primer.
Docker is a popular open-source software platform for creating lightweight, containerized applications that can be deployed and run independently in a shared execution environment. Docker containers are software packages that include all of the software dependencies and settings required to run an application.
Kubernetes is an open-source platform for managing — or “orchestrating” — containers and their related networking and management services. Tools such as Docker Compose and Kompose are used to create, convert, and manage the YAML configuration files used to define and run multi-container applications in Docker and Kubernetes environments. KPS accepts these YAML files directly as input for configuration.
If you’re already familiar with using Docker and Kubernetes, you’ll find that deploying a data interface to the edge with Karbon Platform Services is just as easy as deploying to the datacenter.
Data Interfaces in Karbon Platform Services
To create and manage data interfaces in Karbon Platform Services, you can use the command line interface or REST APIs.
The command line interface, kps, allows for creation, deletion, and modification of Karbon Platform Services’ resources including data sources. You can create data interfaces via the command line interface with a yaml file similar to the following:
kind: dataSource name: ex_nats svcDomain: <Service Domain> protocol: DATAINTERFACE type: Sensor authType: CERTIFICATE ifcInfo: class: DATAINTERFACE img: registry.hub.docker.com/xiiot/nats:v1.0 kind: IN protocol: nats ports: - name: nats port: 4222 edge: myprovideredge fields: - name: topic topic: "mock-topic" - name: host topic: "natshost-10.0.0.1" - name: port topic: "natsport-4222"
For more information regarding how to create data interfaces, look through our various recipes on how to create specific data interfaces: Data Interface Recipes
I’ve had the pleasure of developing data interfaces as part of many projects during my time with Karbon Platform Services. Below, I’ll take you through a few examples.
Customer Story #1 – IoT/AI: PLC
We were tasked with streaming data to generate insights from an Omron PLC running on the customer’s factory floor. The requirement was to increase machine up-time and provide operation visibility by reading temperature and pressure data from a motor that was part of an injection molding machine.
The factory manager needs to be able to monitor this data centrally for increased visibility and reduced reaction time. Note that a minor change in temperature and pressure parameters generates a heat profile that can cause part defects.
Though many protocols such as MQTT, RTSP, etc are natively supported in the stack, we have chosen to leverage Litmus Automation in order to ingest data from PLCs. The goal was to ingest data from the Omron PLC, filter and transform it, then push the data to an S3 bucket running in AWS.
The question became how can we pass data from Litmus Edge to Karbon Platform Services (named Xi IoT at the time). The solution we found was providing a Containers-as-a-Service to the customer to connect their Litmus Edge to Karbon Platform Services. This CaaS solution we used became the KPS Data Interfaces you see today. Today we have a litmus container we provide customers to connect their Litmus Edge to Karbon Platform Services for various PLC oriented use cases.
We were able to deploy the solution in a matter of two weeks given the success of the data interface, and the customer was able to drive up the OEE (Overall Equipment Effectiveness) and reduce the raw material wastage by 15-20%.
Customer Story #2 – Condition Monitoring: S3
A branch of the United States uniform services engaged Nutanix to provide a platform to inject weather data from NEXRAD S3 buckets into a weather analytics service running in their private data center.
The source S3 bucket is a collection of real time and archival data from the Next Generation Weather Radar network. NEXRAD is a network of 160 high-resolution Doppler radar sites that detects precipitation and atmospheric movement and disseminates data in approximately 5 minute intervals from each site. The data can be fed into weather visualization services for analytics and predictions.
The goal was to ingest and transform the files downloaded from the S3 bucket, then deliver to an NFS share (Nutanix Files). Given that the data was being updated in real time, we were not able to directly download the files to the NFS share. The data interface allowed us to process the bucket contents in chunks and leverage real time processing to push the data into the weather analytics software.
We were able to get the system running in a couple weeks and the customer was able to leverage the data interface to ingest the NEXRAD data of four different sites. This allowed them to monitor composite radar data in realtime to make accurate decisions.
Customer Story #3 – Service Domain Interconnection: Kafka
An existing Oil & Energy customer engaged Nutanix with a use case to provide interconnection between existing KPS Service Domains. The goal was to move recorded data from the edge into their private data center. The Service Domains were running in different networks so entities such as Data Pipelines would not work well. The solution we arrived at was to use event streaming to pass data from the edge to the cloud. We chose to go with Kafka due to its ability to provide high throughput with low latency for handling real time data streams.
Given that Kafka architecture consists of producers and consumers we needed to implement two data interfaces. The first spins up a Kafka producer to publish data to the KPS Kafka Service broker running on the Service Domain. This data interface runs at the edge level and ingests data from the customer’s recording vehicle. There’s another data interface running in their private data center with a Kafka consumer to ingest data being published to the broker. With both data interfaces running, our customer is able to pass data/messages across multiple Service Domains and have a truly interconnected system of devices.
Get Started Now
You read above how Nutanix Karbon Platform Services can enable you to start your Industry 4.0 journey from data injection to service domain interconnectivity. While we discussed many use cases in this blog, a similar framework could be leveraged for multiple other use cases with different settings, as deemed fit for your own Industry 4.0 journey.
Our commitment to you is that we will hold your hand throughout this journey – from hardware selection, right design choices with our platform, bringing in the right partner (as appropriate), kickstarting a pilot to expanded production deployments and subsequent day-N operations and management! If you are a CxO or representing OT or IT teams, don’t hesitate to contact your local Nutanix sales team or reach out directly to the Karbon Platform Services team by sending an email to kps [at] nutanix [dot] com. You can also start right away with a free trial at nutanix.com/products/karbon/platform-services. Either way, let’s start and accelerate your journey together!
Sharvil Kekre, Member of Technical Staff, Karbon Platform Services