Hackweek XXXI Recap

Engineering

Learn from our challenges and triumphs as our talented engineering team offers insights for discussion and sharing.

Hackweek XXXI Recap

Engineering

It’s been a while since our last Hackweek recap, and I’m happy to say that Hackweek persists as our strongest tradition here at LiveRamp. Hackweek is a company-wide event that happens five times a year. It’s an opportunity for engineers and other LiveRampers to devote a full week towards whatever project they believe might provide enormous benefit to the company.

hackweek6

Project Highlights

JobWatch: Jared and Roshan used the week to put together an impressive OS X app called JobWatch, which engineers can configure to alert us about the progress of our Hadoop jobs. Their presentation quickly won a following among our engineers who are constantly interrupted by the need to manually check our job tracking webpage.

Cascading Improvements: Ben, working solo, dug deep into the internals of our open-source extension to Cascading (an abstraction layer on top of Hadoop) and emerged on the other side with a suite of new features to improve the checkpointing, history tracking, stat generation, and overall control of all of our core workflows.

hackweek1

Cascading PigPen: Piotr wrapped up a long-term project this time around. Over the past three Hackweeks, he built a Cascading-based backend platform for PigPen, Netflix’s clojure-based map-reduce framework. During his presentation, which ultimately earned him the coveted Audience Choice prize, Piotr had this to say:

“Now we can leverage the power and expressiveness of Clojure in our Hadoop workflows, with seamless interoperability with our existing Java/Cascading infrastructure.”

LiveRamp API: Armaan and Vlad  took on the challenge of creating a way to programmatically access LiveRamp distribution and reporting capabilities by third parties. With so much recent growth, our teams were noticing a swelling demand by clients to integrate LiveRamp deeply into their respective platforms. A full-fledged API seemed like a natural solution, and that’s exactly what this duo created.

hackweek2

Our other projects this time include a Gmail integration with Streak‘s API, an interview question stat interface, an experimental API to skip portions of our lengthy matching workflow, operations to speed up that lengthy workflow, and some great statistical investigations into our system strain and market growth.

As always, this past Hackweek was a huge success and a ton of fun for everybody involved. With 14 total projects this time around, our portfolio of awesome creations and talented hackers continues to expand. And, perhaps the best part about the end of Hackweek, there’s only nine weeks to go until the next one!