How Discord Stores Trillions of Messages - A deep dive

The Backend Engineering Show with Hussein Nasser

Episode | Podcast

Date: Sat, 11 Mar 2023 06:41:48 GMT

<p>Discord engineering goes into details of how they migrated from Cassandra to ScyllaDB, improved the performance of their reads and writes and rearchitected their backend to support the new load. It is an interesting episode lets get into it</p> <p>0:00 Intro</p> <p>1:50 Relational vs Distributed</p> <p>7:00 The Cassandra Troubles</p> <p>11:00 SnowFlake vs UUID</p> <p>14:30 B+Tree</p> <p>19:20 B+Tree and SSDs</p> <p>25:30 LSM Trees</p> <p>31:00 Hot partitions</p> <p>36:00 Cassandra Garbage Collector Pauses</p> <p>40:00 Changing the Architecture</p> <p>45:00 The Data Services</p> <p>55:00 The Migration</p> <p>1:02:00 Zoned Named Spaces</p> <p>1:04:00 Summary</p> <p>Article here <a href="https://discord.com/blog/how-discord-stores-trillions-of-messages">How Discord Stores Trillions of Messages</a></p> <p><a href="https://discord.com/blog/how-discord-stores-trillions-of-messages">https://discord.com/blog/how-discord-stores-trillions-of-messages</a></p>