Hackweek XXXII Recap

Engineering

Learn from our challenges and triumphs as our talented engineering team offers insights for discussion and sharing.

Hackweek XXXII Recap

Engineering

Every ten weeks, LiveRampers spend a week working on projects of their choice. These projects may focus on things they are particularly passionate about, things they think will be particularly valuable, or aspects of the company they would like to learn more about. During these Hackweeks, you can often find groups of people working on projects in the team room, sometimes even well after midnight on Hackday! Hackweeks are always a memorable experience and this one was no exception.

In case you missed our previous Hackweek recap, you can read it here.

presentation-pic-3

Project Highlights

Codility: Matt, Nicole, and Jack looked at using Codility to streamline our engineering recruiting process. Codility is an online interview platform where candidates solve, test, and debug coding problems in an online editor. It has lots of cool features, including allowing graders to see a candidate’s problem solving skills in action by showing the progression of the candidate’s solution over time. Of particular interest is seeing how and when they tested their code. This project earned the Audience Choice award! codility

 

hackweek-in-action-pic-2

Are We Missing Cat Lovers: As part of our onboarding process, we use anonymous identifiers purchased from match partners. What if this data failed to capture certain specific demographics, such as cat lovers? Justin, Ryan, Kevin, Evan, and Jose investigated this question. They gathered statistics about match rates across various demographics to check how well we are matching each one. To everyone’s relief, it turns out that LiveRamp does a great job matching cat lovers! This project has a lot of potential in helping to choose new match partners strategically.

 

Naming New Office Rooms: As the move to our new office fast approaches, Allyson, Andy, Evan, Kevin, and Soerian decided to take on the important challenge of deciding on a naming scheme for our new conference rooms! In our current office, conference rooms are named after continents and their relative locations roughly correspond to the locations of their namesakes on a map. But the new office will require a new naming scheme, both for the sake of novelty, as well as because of the large increase in the number of rooms. Among the options the team considered were naming rooms after mathematical axioms and naming them after San Francisco neighborhoods. In the end, the team decided to put the matter to a general vote so we’ll have to wait and see which idea prevails!

hack_2

Cluster Dashbaord: Ben had previously developed a user interface which is already widely used to monitor workflows running on our Hadoop cluster. This hackweek, Ben, Jeremy, Soerian, and I worked on adding new features to the interface. Among these were a view that provides insight into which steps within a workflow take up the most resources, a view that shows estimated costs for each workflow based on the fraction of total cluster resources it consumes, and a visualization of how these costs have evolved over time. The aim was both to provide a means to identify and investigate changes in the behavior of important workflows and also to increase awareness of cluster usage costs, since our Hadoop cluster is one of our most important assets. cluster-dashboard

 

presentation-pic-4

Find CIs: Jessie, Parker, Shreyas, and Steve worked on achieving 100% self service for customers who import files and then need to set configurations, monitor the progress of their imports, and perform other important tasks. This project has the potential to lighten the load for our CI (Customer Implementation) team, as well as to reduce the possibility of mis-communications when customers need to perform these tasks indirectly through CI.

 

There were many other interesting projects in addition to the ones featured above!

On the non-engineering side:

  • Andy revamped LiveRamp’s slide templates, aiming to improve their look and feel.
  • Justin, Nicola, Jack, Lauren, and Colleen worked on a combined Salesforce link for LiveRamp and Acxiom to improve standardization of accounts.
kafka monit

On the engineering side:

  • Yi, Liren, and Shawn investigated using Kafka for our log processing systems.
  • Yinpeng and Armaan looked at using Monit to keep track of some long running processes, and make them more resilient to transient failures.
  • Shia and Kevin worked on using bunny1 to help LiveRampers use their browsers more effectively.
  • Michel worked on using Graphite to monitor our server applications, complementing the work being done to improve monitoring for workflows.
  • Alex, Shia, Shreyas, Yining, and Zhiqi worked on developing a dashboard to coordinate internal communication related to our Data Distribution Service.
  • Yining and Parker worked on a BootRamp project, inspired by the popular Bootstrap framework, which will make it easier for engineers to use standard LiveRamp styles when building UIs.
  • Piotr and Yuwei built a monitoring tool that will help make extra sure we protect the privacy of users who have opted out.
  • Porter single-handedly tackled the problem of redesigning our core datastore infrastructure.