Much has been written about the ways the interview process is broken in tech. In the front end group at Ameriprise Financial, we’re trying something new.
For context: Our group builds tools for Ameriprise Financial advisors using React, Redux, and Typescript. The group grew considerably in the last year and one of the tasks I’ve taken on is standardizing our interview process. In the past, Brian, my manager, has led all of the on-site interviews. Brian has a great eye for quality developers, but despite tremendous advances in science we are still unable to clone him, a solution that would both raise the bus factor and scale our existing process elegantly.
Back to the interview idea though. It’s simple. Start by whipping up some very bad code that accomplishes a relatively isolated task using your team’s language of choice. In our case, it’s a ~90 line script that uses plain ol’ JS to bootstrap a web application. It makes service calls, it hydrates some data objects, and then it dumps the results into a generic renderUI function (not included in the script). This code, at least in purpose, is something you’d see in just about any web app at Ameriprise. It leaves out the view library because we wanted to draw from a larger talent pool (JS devs instead of only those with React experience). It has in its glorious ~90 lines ~80 issues, ranging from indentation to variable naming to misunderstandings of OOJS to misunderstandings of asynchronous JS to poor error handling and other odd things you’d be ashamed to remember are on your GitHub. Put more succinctly, it has many errors that would require both breadth and depth of knowledge to fix.
Physically print the code out. Yes, just like for your prod deployments.
Now, code in hand, go forth into your interview. Tell the candidate that you’d like them to look over the code and find the issues with it. Explain to the candidate that the code is a whole cornucopia of terrible, that they won’t have enough time to find all of the issues, and that the code is not from an existing codebase that they’d be working on if hired (it isn’t, right?). The candidate should also be afforded a writing utensil and given free reign to circle, scribble upon, and annotate their findings. Finally, let them know that it’s okay to ask questions, and that it’s even more okay if the exercise is less an awkward, silent bug hunt and more of a set of code quality discussions. That last note is for you too, Esteemed Reader, you big shot.
Stay in the room with the candidate while they go through the exercise. Given a script around 100 lines long, you’ll want to allot at the very least a half hour. If the candidate is struggling to get going, suggest that they start by first figuring out what the code is supposed to do before trying to pin down any specific issues. If a candidate is off to the races but getting too focused on finding all of the low hanging fruit, mention that there are other types of problems in the code and ask what else they see.
During the interview, keep an eye out for a few things. What’s the candidate’s process? Do they even have a process? Did they try to understand the code before beginning to furiously scribble corrections? Did they ask any particularly good questions? Did they circle back on earlier issues once they had more context? Did they have a lot of strong opinions? When you dug deeper, did you find that those strong opinions were bordering on dogmatism? For any higher level issues in the code, did they suggest a range of solutions or just one? Was it just one solution because they believe that’s the only valid option or because it’s the only option they knew of? How well did they verbalize what they were seeing? Could they articulate their solution well when prompted? Did you even need to prompt them, or were they naturally communicative?
Once the interview is done, assuming you followed the steps above, you should have a whole heckuva lot of data points (both quantitative and qualitative) to determine whether or not the candidate is up to snuff. What’s more, if you wrote the code with your existing work in mind, those data points will be relevant to what the candidate will actually be doing day-to-day if hired. Most importantly, after presenting the above as your very own, wholly-original idea to your team, you’ll finally have the organizational clout to tell your coworker, the one who always wants to ask candidates to write Fizz Buzz in Haskell, that writing Fizz Buzz in Haskell is a fool’s errand and a dead letter, as far as your team is concerned.
Like I mentioned at the top, we’re just starting to try this at Ameriprise. It’s not a holistic solution to interviewing and it’s possible that the above instructions have a few issues of their own. That said, it’s our attempt to sidestep some of the most commonly griped about interview format pitfalls and so far it’s working very well.
Have you tried something similar at your company? How did it go? I’d love to find out- let me know by dropping me a message here: lnagle@gmail.com