Serverless ML: FaaS and Lambda
Function-as-a-Service (FaaS) and Lambda functions are types of serverless systems.
An example of a common serverless website configuration. Source: NBS System
Going serverless for model serving or inference sounds attractive. Theoretically, there would be less infrastructure to manage and less idle GPU/CPU cost.
In practice, however, cold-start times and other unavoidable hurdles have slowed widespread adoption.
Last modified 3yr ago