For an industry that prides itself on it's analytical ability and abstract mental processing, we often don't do a great job applying that mental skill to the most important element of the programmer's tool chest - that is, ourselves.
In my most recent new role (at a company called Smartsheet, and which I've been doing for six months), I'm being asked to build out a team of Developer Advocates. This innocuous-sounding request comes with an interesting hitch: The company has never hired anybody in this kind of role before and so has no interview process already in place for hiring people. They know how to hire engineers, of course, as well as salespeople, marketing, even UI/UX, but they've never had any kind of Developer Relations department (which is a large part of the reason why they hired me).
As many of you already know from reading my blog or some hints I've dropped in previous editorials, I have some really strong opinions about the ways in which our industry currently conducts interviews anyway, so when my boss turned to me and said, “How do you want to interview them?” I had no problem sitting down and defining what I thought that process should look like. As a matter of fact, I was bracing for a fight with HR, because I had a strong feeling that whatever they came up with was going to be missing some key components, but, as has happened several times already in this company, they pleasantly surprised me.
What's Wrong?
Fundamentally, the “let's watch you code a binary tree at the whiteboard” interview technique is just ever-so-ridiculously wrong. We never code at the whiteboard - we use IDEs. We don't code binary trees at all most of the time - we use existing libraries until we reach a point where the performance of doing so can be proven to be insufficient. We use existing libraries and it's good enough 98% of the time. Also, we don't code in front of other people in a high-stress situation. We write code under relatively low stress, sitting at a desk with a coffee or other relaxing beverage nearby, headphones on, or perhaps as part of a coding pair actively working together to accomplish a task. The coding-at-the-whiteboard exercise is so fundamentally foreign to our day-to-day routine that asking a candidate to perform it may as well be asking the candidate to stand up and sing an aria out of the Phantom of the Opera - and then deciding whether to give them the Web front end development position based on how well they hit that high E.
This coding at the whiteboard is the product of the “enlightened” firms. These companies plopped a developer into a chair and asked them such questions as “How do you handle stress?” or “Where do you see yourself in five years?” or the perennial Microsoft favorites “How would you move Mount Fuji?” and “Why are manhole covers round?” These questions might have served a purpose once, but if they did, nobody really knows what it is anymore.
If we put this into analogous terms, this kind of screening is like looking for a band to play at your wedding by asking them their thoughts on music composition and how it influenced the folk songs of Italy during the Renaissance period - or sitting them down to ask what their ambitions are as a band or what their acceptance speech would include if they were to be voted into the Rock-and-Roll Hall of Fame.
How Do We Fix This?
To understand how we got here, we have to realize that most of the time, nobody is trained on how to interview. Your boss simply shows up in your cubical and says, “Hey, I want you to interview a candidate this afternoon. I just sent you their resume in email, and HR has some guidelines on what you can and can't ask them on the share drive. Your slot is at 2pm. Thanks!” Faced with a situation where we're asked to do something we're not comfortable with, we often fall back on what's familiar - in this case, what was used to interview us, even if that experience was two, five, ten, or even twenty years ago. We do this even if that experience really wasn't all that useful back then. That set of questions, bad as they were, are already implicitly acceptable to HR because that's what they asked you when you interviewed.
To fix this, we have to channel the philosophers of old, ask some fundamental questions that may lead to more questions than answers, and build from the ground up. So let's do that. I'm going to walk through some of the questions that I ask myself every time I think about how to construct an interview process, and how I answered them for the Developer Advocate position that we're currently interviewing. Bear in mind, your/your company's answers may be a little bit different from mine, but the important thing here isn't necessarily the specific answers we come to, but that the questions yield answers that work and stand up to examination and scrutiny.
What is an Interview?
In a nutshell, an interview is the company's effort to ascertain whether a given candidate is suitable for the position to which they are applying.
What Determines the Candidate's Suitability?
Suitability varies with the position. For a Developer Advocate, it's reasonable to expect that they'll be engaged in a variety of tasks involving their ability to code (across a variety of different programming languages and platforms at my company, because Smartsheet has an HTTP API and SDKs across four languages), such as samples and demos and such. They'll also need to be able to write technical articles for our developer portal, and of course, give presentations to customers and/or at conferences and meetups. This means that we need to test their coding skills, writing skills, and presentation skills.
There's also a more “soft” component to this; the candidate needs to demonstrate that they're going to be comfortable working within our environment. Usually this is classified as a “culture fit,” but it's also seeing whether they exhibit some basic levels of professionalism: can they meet deadlines, can they be accountable, and so on. In the case of a Developer Advocate, because they'll represent our company, we're concerned with how they will comport themselves in public, particularly when challenged (usually as part of a presentation, but certainly regarding article or code comments that come up as well).
How Can We Test for Suitability?
The coding skills are probably the easiest to test - just have them write code. It could be argued that for a Developer Advocate, it's not unreasonable to be expected to be able to code at the whiteboard, given that they will be standing at a whiteboard in front of customers or conference audiences, but far more often than that, they will be writing samples. Because most of the time those samples will involve our API, it makes sense to ask them to write some code against our API. Just something simple, and reasonably straightforward, like maybe taking the classic “TODO” app that's so commonly the “getting started” project for many platforms and languages, and adapting it to use our API as the storage back end. I don't need them to be in the office doing it - in fact, I want them to use whatever language, platform and/or tools they're comfortable with, so this is something they can do from home. But I don't want to ask them to work on it for too long - I need to be respectful of their available time if they're currently already working a full-time position - so let's put a cap of five to ten hours on it. I'll review the code sample once they submit it, and if the code looks even halfway decent, it's probably safe to assume they've got the coding skills we need.
Writing is also pretty straight-forward: the candidate must submit a writing sample. It could be something they write specifically for this situation (such as technical write-up of what they just coded up for us), or it could be a previous blog or article they've written before, so long as we get to see the raw, unedited version. Again, this can be done from home, on their own time, without needing to come in to the office. I'll have one of our Marketing people review the writing, both because it gets people from outside of the R&D division involved (thus avoiding the echo chamber that develops when you work within the same group of people for a while), and because I need to know that anybody working as a Developer Advocate is comfortable working with the people in Marketing. If the candidates can't handle Marketing telling them how their articles need to be corrected, they're not going to fit in well in this position.
We also really want to see them present, in person. This needs to be the in-office component; assuming that they pass the writing and coding sample reviews, they'll come into the office and do a 30- to 45-minute presentation on a topic of their choice. I'll bring in a dozen or so people from around the company to be the audience, and then we'll review the candidate's presentation to make sure that there aren't any egregious flaws, like complete panic or a total disregard for the audience's questions if they're “too stupid to bother answering.” I don't need a professional speaker for this job, just somebody I can work with and coach if necessary.
Summary
I'm not entirely finished with the list of questions. One of the big ones is, “How do we know the process is working?” and its immediate follow-up is, “How can we test to see if the process is working?” It's not too difficult to see where this discussion is going. As with any new process or procedure, it's entirely likely that there are aspects to this particular process that will need revision over time, and/or some human judgment calls to deal with unexpected circumstances that we can't anticipate in advance. But by asking these questions up front and building the solution based on the answers, we can go back to those questions to deal with the unanticipated situations - if, for example, a candidate can't make it into the office easily because he currently lives in South Africa but he has an extensive speaking portfolio that we can view online, we can satisfy the presentation component by looking at those online videos. We still have to think about how to get him into the office to satisfy some of the other concerns, but we'll have to address that based on the specific situation he's in.
They key here is to realize two basic facts: first, the standard interview process is fundamentally broken and needs fixing; and second, the best way to fix it is to start by asking philosophical questions and building up from each answer that comes back.
