Serverless ML: FaaS and Lambda

Function-as-a-Service (FaaS) and Lambda functions are types of serverless systems.

Going serverless for model serving or inference sounds attractive. Theoretically, there would be less infrastructure to manage and less idle GPU/CPU cost.

In practice, however, cold-start times and other unavoidable hurdles have slowed widespread adoption.

PreviousREST and gRPC NextSynthetic Data

Last updated 5 years ago

Was this helpful?

Function-as-a-Service (FaaS) and Lambda functions are types of serverless systems.

Going serverless for model serving or inference sounds attractive. Theoretically, there would be less infrastructure to manage and less idle GPU/CPU cost.

In practice, however, cold-start times and other unavoidable hurdles have slowed widespread adoption.