We just finished re-architecting one of our core systems responsible for pushing massive amounts of data (over 250 TB / day) to our partners. This re-architecture aimed at improving throughput and long-term scalability and gave us the opportunity to reflect upon some of the core challenges of building scalable systems. This post is an ...