Podevcast

Resiliency at Shopify with Camilo Lopez and Tai Dickerson

Google Cloud Platform Podcast

Episode | Podcast

Date: Wed, 26 Jan 2022 17:07:41 +0000

<a href="https://twitter.com/carterthecomic">Carter Morgan</a> and <a href="https://twitter.com/stephr_wong">Stephanie Wong</a> host Shopify guests <a href="https://twitter.com/CamiloLopez">Camilo Lopez</a> and Tai Dickerson this week. Shopify streamlines the online purchasing process so merchants and customers can transact with confidence. Camilo and Tai talk in-depth about Shopify’s tech stack and why the choices made are so important to performance. Shopify engineers use a combination of Ruby on Rails, MySQL and Google products like Kubernetes. Resiliency systems like active-active configurations, chat ops for quick solutions, and bot and overload protection are worked in. By leveraging these tools and staying flexible in their resiliency efforts, Shopify is able to adjust to new merchant requirements and teams are able to work efficiently. While tech continues to progress and change, the Shopify culture remains a driving force for advancement, Camilo tells us. The company ideals and axioms help steer the brand and dictate which technologies they’ll use to solve new and changing client demands. The 2014 outage shaped the future of these cultural ideals, emphasizing the need for quick action and resiliency components like constraints to ensure system safety. Shopify engineers also built enhanced testing tools like Toxiproxy to simulate poor network conditions and account for potential issues. The 2021 Black Friday Cyber Monday shopping season was Shopify’s biggest yet. Camilo and Tai describe how Shopify’s resiliency culture and intense prep work made the biggest shopping weekend of the year so successful. By offering educational tools and a support network that values good communication, their company culture continues to grow, and Tai tells us how it’s not just the software that should be resilient. Building a resilient, flexible company culture is just as important. Camilo talks about Shopify’s recent shift to a completely remote work place and the new challenges and opportunities it presents. <h5>Camilo Lopez</h5> <a href="https://twitter.com/CamiloLopez">Camilo</a> has worked at Shopify for more than 10 years, he has been an IC and a manager leading teams that take care of Shopify’s scalability and resiliency. <h5>Tai Dickerson</h5> Tai is a production engineer at Shopify, where she shares her passion for resilience engineering with others via paper discussions and as a leader in Shopify’s Resiliency SIG. <h5>Cool things of the week</h5> <ul> <li style="font-weight: 400;">Machine images is GA <a href="https://cloud.google.com/compute/docs/machine-images">docs</a></li> <li style="font-weight: 400;">New Cloud Logging and Monitoring capabilities</li> <li style="display: inline;"> <ul> <li style="font-weight: 400;">Monitoring third-party applications: MariaDB <a href="https://cloud.google.com/monitoring/agent/ops-agent/third-party/mariadb"> docs</a></li> <li style="font-weight: 400;">Monitoring third-party applications: MySQL <a href="https://cloud.google.com/monitoring/agent/ops-agent/third-party/mysql"> docs</a></li> <li style="font-weight: 400;">Monitoring third-party applications: Memcached <a href="https://cloud.google.com/monitoring/agent/ops-agent/third-party/memcached"> docs</a></li> <li style="font-weight: 400;">Starting with version 2.8.0, the Ops Agent supports Ubuntu 21.10. For more information, see Linux operating systems <a href="https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent#linux_operating_systems"> docs</a></li> </ul> </li> </ul> <h5>Interview</h5> <ul> <li style="font-weight: 400;">Shopify <a href="https://www.shopify.com/">site</a></li> <li style="font-weight: 400;">Kubernetes <a href="https://kubernetes.io/">site</a></li> <li style="font-weight: 400;">GKE <a href="https://cloud.google.com/kubernetes-engine">site</a></li> <li style="font-weight: 400;">Kafka <a href="https://kafka.apache.org/">site</a></li> <li style="font-weight: 400;">Redis <a href="https://redis.io/">site</a></li> <li style="font-weight: 400;">Elastic Search <a href="https://www.elastic.co/enterprise-search">site</a></li> <li style="font-weight: 400;">Memcached <a href="https://cloud.google.com/monitoring/agent/ops-agent/third-party/memcached"> docs</a></li> <li style="font-weight: 400;">Toxiproxy <a href="https://github.com/Shopify/toxiproxy">site</a></li> <li style="font-weight: 400;">Shopify Engineering <a href="https://shopify.engineering/">site</a></li> <li style="font-weight: 400;">Shopify Careers <a href="https://www.shopify.com/careers">site</a></li> <li style="font-weight: 400;">BFCM Twitter Thread <a href="https://twitter.com/ShopifyEng/status/1465806691543531525"> site</a></li> <li style="font-weight: 400;">Shopify engineers deliver on peak performance during Black Friday Cyber Monday 2021 <a href="https://cloud.google.com/blog/topics/retail/shopify-and-google-cloud-team-up-for-an-epic-bfcm-weekend"> blog</a></li> <li style="font-weight: 400;">Cloud, Load, and Modular Code: What 2022 Looks Like for Shopify <a href="https://shopify.engineering/cloud-load-modular-code-shopify-2022"> blog</a></li> <li style="font-weight: 400;">Terri Haber on Resiliency at Scale <a href="https://twitter.com/ShopifyEng/status/1455245541193891840"> site</a></li> <li style="font-weight: 400;">Terri Haber on Enforced Pacing <a href="https://twitter.com/ShopifyEng/status/1456276549099917319"> site</a></li> <li style="font-weight: 400;">Bart Jedrocha on Load Testing <a href="https://twitter.com/ShopifyEng/status/1458077906391941128"> site</a></li> <li style="font-weight: 400;">Bart Jedrocha on Tooling for Load Testing <a href="https://twitter.com/ShopifyEng/status/1459226003666976770"> site</a></li> <li style="font-weight: 400;">Bart Jedrocha on The Future of Load Testing <a href="https://twitter.com/ShopifyEng/status/1460625455275462663"> site</a></li> <li style="font-weight: 400;">Ryan McIlmoyl on Code Red <a href="https://twitter.com/ShopifyEng/status/1461710353759358979"> site</a></li> <li style="font-weight: 400;">Ryan McIlmoyl on Working with IMOC <a href="https://twitter.com/ShopifyEng/status/1463164686719852548"> site</a></li> <li style="font-weight: 400;">Camilo Lopez on The 2014 Outage <a href="https://twitter.com/ShopifyEng/status/1466120916279037953"> site</a></li> <li style="font-weight: 400;">Camilo Lopez on Holiday Season Learnings <a href="https://twitter.com/ShopifyEng/status/1466857211943133190"> site</a></li> <li style="font-weight: 400;">Tai Dickerson on Doing Things Differently <a href="https://twitter.com/ShopifyEng/status/1468665682220908544"> site</a></li> <li style="font-weight: 400;">Tai Dickerson on Learning & Community <a href="https://twitter.com/ShopifyEng/status/1469404804426477581"> site</a></li> </ul> <h5>What’s something cool you’re working on?</h5> Stephanie is working on season 2 of the <a href="https://www.google.com/about/datacenters/podcast/">Where the Internet Lives</a> podcast. Carter is working on season 2 of <a href="https://www.youtube.com/watch?v=AGa6fUTtCgE&list=PLIivdWyY5sqIteZONy-rTwyf3w3OddFQ1"> VM End to End</a>. <h5>Hosts</h5> Carter Morgan and Stephanie Wong