Hackweek XXXVI ushered in an exciting 2016 for LiveRamp. While there were a wide range of projects, three themes emerged. First, there were several projects geared towards internal monitoring which culminated in the Blingin’ Monitors project. Second, migrating how we access our stats, opened the door to several investigations into our product and how to make it better. Finally, investigating new technologies to make our systems more performant and maintainable as we organize our engineering system towards a more service-oriented architecture.
LiveRamp has made a Herculean push over the last 6 months to improve its monitoring of all internal systems in an effort to improve reliability and optimize our workflows. The icing on the monitoring cake was displaying all of these metrics on new screens placed around the office.
Integrating Amazon’s Redshift, made accessing and performing statistical analysis on our web traffic logs far easier. By the lowering the technical barrier to access, non-engineers were empowered to complete a wide range of investigations, including new ideas in how we can improve match rate.
LiveRamp has a history of scaling quickly. Integral to iterating rapidly is cutting down bloat along the way. Sweeper is an automated service that deletes old items (i.e. files, database entries, s3 objects) based on user-defined policies. It’s a step towards finding a safe way to remove derelict information, so it does not weigh down our development in the future.
As we move towards a more maintainable service-oriented architecture, internal and external APIs are going to become a centerpiece of our engineering services. Across the engineering team we need to establish agreed upon conventions to make consuming these APIs easy. Swagger is one technology that was investigated for handling this problem.
Interested in reading more of our past Hackweek projects? Check them out here.