Todd Underwood, Google
We are now a full year into the latest AI revolution, this one in generative AI, or large language models. For many organization leaders and SREs the most relevant question that is rarely discussed is: what will it cost and will it be worth it? Important models only matter when they are integrated into some product and served to users. Large model training is incredibly expensive as is large model serving. This talk looks at the history of serving cost curves for simpler applications (web applications!) and understand what the future might bring. We will look at the possible future of the breathtaking costs of large language model training and serving.
Todd Underwood, Google
Todd Underwood is a Senior Engineering Director at Google. He leads ML capacity engineering in the office of the CFO at Alphabet. Before that, he founded and led ML Site Reliability Engineering, a set of teams that build and scale internal and external AI/ML services and are critical to almost every Product Area at Google. He was previously the Site Lead for Google’s Pittsburgh office. He recently published Reliable Machine Learning: Applying SRE Principles to ML in Production (O’Reilly Press, 2022).
author = {Todd Underwood},
title = {Artificial Intelligence: How Much Will It Cost You?},
year = {2023},
address = {Dublin},
publisher = {USENIX Association},
month = oct
}
