Home
Log
Application
Auditlog
Category
Certificate
Cloud_Profile
Container_Registry
Data_Pipeline
Data_Source
Event
Function
Helm
Auth
Application_Status
LogCollector
ML_Model
MLModel_Status
Node
Node_Info
Project
Runtime_Environment
Service_Domain
Service_Domain_Info
Project_Service
SSH
User_API_Token
User_Public_Key
Powered by Stoplight
post

/models/{name}/versions/{version}:infer

The end point takes the request data and runs inference on the given trained model. The input data format depends on the model input. Input data can be simple datatypes or complex nested lists to represent n-dimensional Tensors.

Python Example to perform inference on object detection model http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz

 //Read image using Python Pillow library
image = Image.open("/test.jpg")
ai_inference_endpoint = os.environ[“AI_INFERENCE_ENDPOINT”]
modelName = “ssd_mobilenet_v2”
version = 1
url = “http://%s/v1/models/%s/versions/%d:infer” % ( ai_inference_endpoint, modelName, version)
image_np = numpy.asarray(image, dtype=“int32”)
image_np_expanded = numpy.expand_dims(image_np, axis=0)
data = json.dumps({“signature_name”: “serving_default”, “instances”: image_np_expanded.tolist()})
headers = {“content-type”: “application/json”}
response = requests.post(url, data=json_data, headers=headers)
inference_payload = json.loads(text)
predictions = inference_payload[“predictions”]

The host address is injected into Xi IoT Applications or Data Pipelines as an environmental variable named AI_INFERENCE_ENDPOINT.

Request Parameters

2 Path Parameters

Request Body

Schema
object

InferenceRequest is a JSON object that encodes the input data given to the model. The JSON object properties are as follows. signature_name is an optional field. If not specified, the default serving signature is used. InferenceRequest can have either instances or inputs but not both. If the input data has same 0th dimension, use the instances field. Otherwise use the inputs field.

"instances": value | list | nested list | list-of-objects
"inputs": value | list | nested list | object

For the instances or inputs field, value is determined by the input definition of the model. Value can be simple JSON values (boolean, number, or string), lists of simple values, or complex nested lists.

signature_name
string
instances
object

Message that represents an arbitrary body. Use it only for payload formats that cannot be represented as JSON, such as raw binary or complex data structures

inputs
object

Message that represents an arbitrary body. Use it only for payload formats that cannot be represented as JSON, such as raw binary or complex data structures

Responses

Successful Response

Schema
object
example: predictions:[ { "detection_scores":[], "detection_classes":[], "detection_boxes":[], "num_detections":2.0 }]
predictions
object

Message that represents an arbitrary body. Use it only for payload formats that cannot be represented as JSON, such as raw binary or complex data structures

Send a Test Request

Send requests directly from the browser (CORS must be enabled)
Path Params
2 path params not set
name
version
$$.env
1 variable not set
host