REST and gRPC
Most model-serving frameworks are based on REST. TensorFlow Serving and TensorRT offer gRPC endpoints which are fussier but more performant.
Benefits of REST
Stateless - No client context is stored on the server between requests
Self-contained - All information that is needed to service a request is packaged with the request itself
Flexible - REST is programming language agnostic, has universal browser and language support, and supports a large number of filetypes
Benefits of gRPC
Bi-directional - gRCP supports two-way communication
Simplicity - No headers, methods, or body, and better status codes
Performant - Binary data via protocol buffers for serializing structure data, performs better under high loads
Last updated