Three Principles for Hiring Data Scientists
A variety of factors contribute to good hiring — the structure of the interview, the questions asked, review committee, comp packages, etc. — but before you focus on those particulars, I think it’s important to identify the principles that underpin a general philosophy of hiring and candidate assessment that will maximize your chances of success. In this article, I share three principles that have been derived from my own experiences with hiring and that I believe are both critically important and often overlooked. These principles are not intended to be comprehensive or the most important aspects of hiring, and they do not touch on the substance or format of an interview. However, I feel strongly that applying these principles to any organization’s hiring practices will more effectively drive quality hiring, growth, and retention of top talent.
While the primary focus of this piece is on hiring data scientists, it is my belief that these these principles apply equally well to most roles. So why did I pick these three? There are a lot of great think pieces on hiring data scientists, but these pieces tend to focus more on interviewing particulars(whether to use take home questions, the best math questions to ask, how to assess coding skills, etc.) and seem to have blindspots around guiding principles. Additionally, irrespective of the size of organization or whether rigid hiring protocols are already being utilized, these principles can supplement existing processes and/or provide the backbone for a more structured hiring process.
The three principles TLDR:
- The Principle of 51%: The hiring manager should have 51% of the vote on which candidate to hire.
- The Principle of 180 Days: Hiring is a prediction about a candidate’s contributions between month 6 and month 36, not an assessment of their current skills and experience.
- The Principle of C > S: As the saying goes, culture (C) eats strategy for breakfast, and it also eats skill sets (S) for breakfast. Never compromise on culture.
The Principle of 51%
Large organizations generally have a hire-by-committee process. Overall, requiring consensus among a broader committee vs. the opinion of a single individual is a good thing. It ensures different perspectives on each candidate are taken into account, forces hiring managers to articulate and defend their hiring rationale, and makes it much harder to hire friends or friends of friends who might be under-qualified. I firmly believe that hiring managers should absolutely get feedback from other members of their team and from cross-functional interviewers from other departments. This feedback is critical and should be taken seriously. You should (almost) never make a hire that your team or your peer managers strongly oppose. However, the hiring manager should be the one making the final call (i.e., retain 51% of the vote). Without this, you get a vote-by-committee approach where the hiring managers distance themselves from the actual decision making. This distancing is bad for a number of reasons.
First, it sends a bad signal to the new hire. Rapport with one’s boss is critical in any role. Indeed, in many studies it is the single most important predictor of job satisfaction and job success. ‘I reviewed 100 resumes, phone screened 20 people, did 5 on-sites, and you were the top person across the board. You demonstrated the strongest aptitude to learn, the best understanding of how to connect data science to business problems, excellent coding and a comprehensive of complex statistical concepts, and a great culture fit. Once the team and I interviewed you we knew you were the person’. This is what you want to tell your new hire. You do not want to tell your new hire, ‘Hi, really excited to have you, the committee really liked you and you scored high on the coding interview and emotional intelligence questions’.
Second, it sends an equally bad signal to the rest of your team. Do you not take responsibility for those with whom your current team members have to work? At some point, rest assured that someone will question the hiring decision. Someone isn’t going to like how the new hire named a function or why they chose one algorithm over another or how they colored a graph or the cut of their jib. If you have a good culture, you’ll hear about it directly. If you have a bad culture, you’ll hear about it indirectly. But either way, you’ll eventually hear about it. How are you going to respond? ‘Hmm yeah everyone on the committee said good things, let’s be patient’. Or, ‘Hmm yeah that’s really great feedback. <Candidate> came in super strong in <these areas> but we knew that <this other area> would need more attention so I’ll do a better job of setting aside extra time for <candidate> to do trainings in this area’. You want to be able to say the latter, but it’s much harder to say this if you didn’t have the final say in the hiring decision.
Third and lastly, in a medium to large size company, your boss is more or less responsible for your well-being and success. They decide what projects you get, your rating, your compensation and your promotions, they get your back when your back needs getting, and they give you constructive feedback when that’s needed. When the hiring manager isn’t the one making the final call on who gets hired, the sense of responsibility for the hire’s success is just not the same. ‘If it works out, great. If not, oh well, it’s not like my reputation is on the line, the committee chose’ — is fundamentally different than — ‘this is the person I chose, their success is my responsibility, and therefore, I will ensure they will be successful’. Everyone wants the latter from their boss.
The Principle of 180 Days
When hiring, experience matters. Skillsets matter. The ability to hit the ground running matters. But at a large (or medium) organization, they don’t matter as much as people think. Often, the main question that hiring managers and committees use for assessment is: how well suited is each candidate for the job today? This focus on the present is wrong. The main question that should be used to assess candidates is: how much is this person likely to contribute in months 6 through 36?
Why 6 months? Because few candidates are able to hit the ground running, and expecting immediate impact is unrealistic. New hires take time to learn the org and the people and the tech stack and the business complexity. Why 36 months? Because no matter how good of a fit a new hire is or how much they love the job and organization, things happen. People get poached. Life events happen and people decide to take time off or move or switch roles. Managers should be clear eyed about the fact that even the most dedicated strong performers will start to ‘answer the phone’ when recruiters call after a a few years in role.
Organizations tend to dramatically overemphasize the skills and experiences that the candidates possess at the time of the interview. ‘But we needed someone to hit the ground running YESTERDAY!’ Ok, that’s fine, but what you need is a short term contractor. You can find someone with the exact skills you need, for the exact duration you need. This will buy you time to hire the correct person for your team long term. ‘But it’s critical that this person has experience in XZY industry’. No, it’s not. People always think that their industry is somehow unique and can only be learned over years of work experience. This is generally untrue. Most verticals within a large organization are more similar than they are different, and most strong candidates can pick up even the most nuanced subject matters in less than a year (assuming that they are committed to learning and have access to mentors and experts who can help them along the way).
Only hiring people with experience in your specific vertical is suboptimal for many reasons. It dramatically reduces the likelihood of having a team with diverse viewpoints and experience and ensures that the talent pool in your specific vertical remains stagnant since you are not allowing for any new entrants. It also majorly reduces the likelihood of finding someone who is going to meaningfully outperform expectations because you already have a pretty good sense of someone’s growth rate and ceiling when they’ve been established in a specific vertical for a while. But no ceiling has been established yet for new entrants.
Of course, a strong foundation of relevant skills is important. But reframing the interview process and hiring decision-making as a prediction of growth and likely contribution over time vs. an assessment of skills possessed today changes the entire thought process. Instead of asking questions that demonstrate what the person knows, this reframing places extra emphasis on assessing how they learned what they know.
Consider the following example:
Interviewer:
‘Tell me about a time when you had to implement <insert difficult technical technique here>?’
Candidate 1: ‘So I ran into this problem at my current job, I’d never seen anything like it before. Neither had my colleagues. I talked to my friends, got some help there. I found a Coursera course, read a couple books, blocked off my calendar for a week, tried a bunch of prototypes, finally found something that worked really well’.
This is a profoundly different answer than:
Candidate 2: ‘We covered this extensively in a grad school seminar.’
Both candidates have demonstrated a strong grasp of the desired skill. One basically figured it out from scratch and sent a strong signal about an ability to learn new skills without formal training, education, or structure. Of course you’d want to ask more follow up questions to dig in further, but Candidate 1 is likely to be much stronger in 6 months — and in 36 months — than where they are today.
On the other hand, the Candidate 2 covered the topic in class, meaning they came into the role with the existing knowledge. This is still great and of course shouldn’t be held against them, but it didn’t provide us any information about their likely continued growth or trajectory. It informed us about their likely performance on day 1, but did not provide any signals to help us make a prediction about how their performance will likely grow from day 1 to day 180 and beyond.
The Principle of C>S
Culture eats skills for breakfast. If the person doesn’t get along with their team members or exhibits the cultural tenets you’d like, they will be a detriment. Someone can be the greatest data scientist in the world. A one-person show who has the best deliverables in the shortest amount of time. If this person is mean spirited, selfish, doesn’t help, disappears for weeks, is consistently critical without proposing alternative solutions, etc., the net impact on the team and organization will be negative. Culture is more important than skill. Now, culture is a loaded word. Culture should not mean what you like to watch on tv or your hobbies or whether you have kids or if you like happy hours or if you’d be friends outside of work or which college you went to. Culture means seeking out diverse experiences and viewpoints. It means work ethic. It means putting team members first. It means collaboration. It means caring: about the organization, about colleagues, about personal well being. A candidate who has an incredible skill set but is a bad culture fit simply won’t work. Plus, it’s nearly impossible to change someone’s approach to work and their teammates, but it’s relatively easy to develop someone’s technical skills.
Good luck, now go build a great team.
