LiveRamp has been building a data science team to explore the massive and complex data sets that compose our Identity Graph. (For more information about what we’re doing on the team check out our data scientist job posting.) In building this team, I’ve had to clarify my philosophy for how to manage scientists working on research projects. Drawing on my experience within Academia, I’ve come up with the following guidelines.
Good organizations ensure that decisions are made at the appropriate level of the management tree, and in modern organizations, with amazing team members many important decisions will be decided at lower levels. An extreme case of this happens in research groups in which the individual researchers know the most about their current project since they spend the most time working with the relevant data and thinking about the problem space. Therefore in managing scientists, a manager should be careful not to put too much weight on their own ideas. Instead, they should think of themselves as an advisor that provides potential ideas, which the individual researcher prioritizes alongside his or her own ideas using their judgment.
In general, the advisor’s lower-level ideas will be relatively worse than those of the researcher since the advisor has less specific context on the project. Therefore a good advisor should provide high-level ideas that add in the context that the individual researcher may not be familiar with. E.g., highlight the important product context so that the researcher knows what types of questions their project could help answer. E.g., explaining how results could be better presented to more directly answer these questions.
Even then the advisor needs to be careful not to assume that those high-level questions can be definitely answered in the short-term. Managing uncertainty is core to scientific research and an advisor that demands too much certainty in too little time is a poor scientific manager. At the extreme, I’ve seen such advisors repel real scientific researchers and attract mediocre ones who don’t respect the inherent uncertainty of what they’re doing. I’ve even heard rumors about such research groups engaging in unethical scientific behaviors such as withholding conflicting data.
A reversal of these rules happens when mentoring a researcher on a new project of which the advisor had greater familiarity. At the beginning of such a mentoring process, it can be useful for the advisor to share many example ideas to provide the mentee with relevant context quickly. It is also great if the mentor can point the mentee to existing examples of related projects so that the mentee can learn from these on their own. Over time, the mentor should take a more hands-off role and start acting more as an advisor.
It is important to explain this scientific management philosophy to new researchers, so they know what to expect from their advisor. Some new researchers may expect their manager to give them a constant stream of good ideas that the researcher simply execute. It can be surprising for them when they start to recognize that their own manager is not constantly giving them great low-level ideas and when they ask for suggestions their advisors gives them relatively worse ideas than those that the researcher developed on their own. This is the nature of scientific research, and new researchers should learn to get comfortable with it.