Boris Grot
Serverless-native data analytics
Traditionally, large-scale data analytics jobs have run in a dedicated on-premise or cloud-based compute cluster. While effective in cases when the query load is steady, dedicated clusters can be inefficient from a cost and/or performance perspective if analytics jobs arrive sporadically or in sudden bursts. Serverless computing, with its extreme elasticity, rapid resource provisioning and usage-based billing, can fill the efficiency gap of cluster-based compute for irregular query arrival patterns. Alas, compared to a traditional cluster, serverless exposes a radically different compute substrate in the form of a vast pool of stateless workers that cannot directly communicate with each other. These differences motivate a need for a serverless-native analytics engine.
This talk will discuss our ongoing work toward that. I will describe the challenges and opportunities in serverless data analytics, and present our approach to navigating these with the Edinburgh Data Analytics Engine for Serverless (ENDLESS).
This talk will discuss our ongoing work toward that. I will describe the challenges and opportunities in serverless data analytics, and present our approach to navigating these with the Edinburgh Data Analytics Engine for Serverless (ENDLESS).
back to overview