Zoltar

Common library for serving TensorFlow and XGBoost models in production.

Zoltar is a library that helps load predictive machine learning models in a JVM. It provides several key abstractions which you can implement to help your service load a serialized model, featurize input data, submit feature vectors to the model, and serve the model’s predictions. Below is a quick overview of these abstractions which can be found in zoltar-core.

Features

• TensorFlow model loading and prediction.
• XGBoost model loading and prediction.
• Filesystems:
  • Local
  • Google Cloud Storage
• Featran featurization library integration.
• Easy integration with Apollo services.
• Easy integration with Semantic Metrics.

Abstractions

Predictor: The core functionality of Zoltar. This object loads a model and calls functions to featurize input vectors and submit them for prediction. To these ends, a predictor is composed of a ModelLoader, FeatureExtractor, and PredictFn.

PredictFn: A function that submits a feature vector to a model for prediction.

Prediction: A wrapper around a single feature vector and its predicted output.

ModelLoader: An object that loads an XGBoost or TensorFlow model from a supported filesystem.

Model: The Java object that houses the predictive model itself.

FeatureExtractor: Takes an input vector and applies a FeatureExtractFn to it.

FeatureExtractFn: A function that takes a raw input vector and extracts a feature vector from it.

For more details, take a look at the source code and follow the documentation in the comments. If you’re using TensorFlow or XGBoost, you can find model specific implementations of these abstractions in zoltar-tensorflow and zoltar-xgboost respectively.

We’ve also provided an example service built on the Apollo framework. It demonstrates how Zoltar components are implemented and organized. We highly recommend you walk through this example before you try integrating Zoltar into your applications.