SREcon23 Europe/Middle East/Africa Conference Program

Traditional software monitoring best practices are not enough to detect problems with machine learning stacks. How can you detect issues and be alerted in real-time? This talk will give you a practical guide on how to do machine learning monitoring: which metrics should you implement and in which order of priority? Can you use your team's existing monitoring and dashboard tools, or do you need an MLOps platform?

This talk explores how QUIC and HTTP/3 are used for a range of diverse use cases. It provides a rapid on-ramp to the technology concepts and focuses on practical considerations that help to effectively use, operate, manage, or debug these fairly new protocols. We'll dive into the different approaches that applications can take to using QUIC, by focusing on HTTP/3 and related MASQUE tunneling techniques. We'll cover QUIC features, caveats, opportunities, and potential pitfalls. By the end, the audience will be comfortable with QUIC and familiar with a range of tools and resources that will help aid with specific needs of today or future developments of tomorrow.

Deploying Node.js at scale is an art mastered by few. The most common problem is an exhaustion of resources that allows the application to denial of service itself. The result is Node.js systems that are massively overprovisioned, wasting enormous amounts of computing and memory - keeping most of them idle. In this talk, we will do some math, discover hard truths, and implement a fix.

Until recently, you could not control outgoing traffic to given Fully Qualified Domain Names (FQDN) using Kubernetes Network Policies. You could use ordinary firewalls, but these are defined by IP, not domain. Even network firewalls that recognize domains do not work in terms of Kubernetes, for example restricting the namespaces and labels which are allowed egress.

Cilium and Istio do provide this ability, but require the complexity of an additional network layer.

Just now, Google Kubernetes came out with a preview release FQDN-aware egress control in Kubernetes Network policies.

I will describe this and show how it fits into the effort now in progress in the Kubernetes Networking Special Interest Group to define FQDN egress control as a standard part of every compliant Kubernetes cluster.

Join us as we discuss how we optimized multi-process Python applications. Decreasing CPU variance, leading better CPU utilization, while improving latency by 20%. We’ll share our journey from a Ruby-based system to creating Monoceros to tackle Python’s limitations in large scale Kubernetes deploymnents. Learn about our initial problems which lead us to improve load-balancing and observabily with Monoceros. We’ll wrap up with a comparison of SO_REUSEPORT load balancing and full load balancers. This talk is perfect for those interested in efficient system performance at scale.

Discover the secrets of hosting GPT-type models in a Kubernetes cluster with multi-GPU nodes. As the demand for custom GPT models grows, SREs are increasingly tasked with providing these capabilities in their organizations. We'll dive into the complexities of serving such models, including their large size and the need for GPU sharding and tensor parallelism. Learn about model file formats, model quantization techniques, and leveraging open-source tools like Huggingface Accelerate. Gain insights into the trade-offs between serving latency, prediction accuracy, and distributed serving, and explore best practices for optimizing resource allocation. Don't miss our live demo showcasing the performance and trade-offs of a GPT-based model. Empower yourself with practical knowledge to meet the demands of hosting language models effectively.

We seemed to be all set in terms of AWS cost optimization–we had just finished migrating our largest, most costly services to the less expensive Graviton architecture and our projected savings forecasts for the year were looking fantastic. That is until March, when drastic changes in the AWS Spot market overturned all of our underlying assumptions. Suddenly, our “cheaper” machines were costing more and most of our discounts completely disappeared by the end of April. The unthinkable had happened and we had no idea if it was temporary or the new normal.

This talk will explore the most likely underlying causes of the AWS Spot market instability, the tools and techniques we used to see the full view of its effects on discounts, and the major actions that we took to keep our infrastructure costs under control when faced with increasing uncertainty.

We all have experienced in our organisations the case where we build a quick solution to solve an immediate problem, and eventually find the software fulfilling other needs. This is the story of Chubby, Google's distributed lock service, and how it began as a mechanism to provide leader election for infrastructure and evolved rapidly to provide service discovery, config-file distribution, and other production-critical services.

During this talk, the presenter will explore the evolution and maturity of the field of Site Reliability Engineering through the lens of this specific piece of infrastructure software. The audience will hear foundational experiences with monitoring, caching, proxying, and isolation — and learn about our experiences, both good and bad. The audience will also hear suggestions for the direction that SRE practice will take in the near future.