Serverless ML: FaaS and Lambda

Function-as-a-Service (FaaS) and Lambda functions are types of serverless systems.

An example of a common serverless website configuration. Source: NBS System

Going serverless for model serving or inference sounds attractive. Theoretically, there would be less infrastructure to manage and less idle GPU/CPU cost.

In practice, however, cold-start times and other unavoidable hurdles have slowed widespread adoption.