Software Developer Interviews Are Garbage

Last week I had an interview for a "backend engineer" position at Docker. The position was for a role where the team was using Clojure actually, which came as a surprise to me when a colleague sent me a link to the job posting, as I had originally thought Docker was entirely a Golang shop. I figured since I've got 10 years of experience writing Clojure for different companies, have shipped multiple complex working solutions, contributed to open source projects and even published some of my own, that hey, I should have a decent chance here, right?

I applied, and after a friendly chat with a recruiter, I was given an overview of how their interview process works. For software developers, they do a 60-minute technical interview with the hiring manager which will usually involve a coding challenge, followed by two 90-minute technical interviews with other developers each involving a coding challenge, and then a final non-technical interview with someone in a senior leadership role. I was then invited for the first-round technical interview. At this point, the recruiter stressed to me (without me prompting him, I might add) that the coding exercises given to candidates during interviews would be all practical exercises and not algorithm or data-structure problems. That was refreshing to hear, and I thanked him for telling me this.

Several days later, I had the first-round technical interview with the hiring manager. After a few simple questions relating to culture and what my ideal team would look like, we switched to the coding exercise portion and I was given the problem description: "Given a list of integers, write a function that will find all of the pairs of numbers that sum to a given total."

So, as I re-read this today in hindsight, I know that it's not a terribly complicated problem. But I think most developers would join me in classifying this as an algorithm problem and not as a practical problem.

And of course, in the interview last week when I was given this problem, I could feel my heart rate shoot up and the combined feeling of stress, shock and dread overcome me. Over the next 40 minutes or so I struggled with this "simple" problem and finally eked out a working solution. It definitely wasn't pretty.

This morning, as expected, I received an automated rejection email.

This is hardly a unique story today in this industry. You don't have to go far before you encounter developers complaining about this problem and how very often the interview seems to be harder than the actual job. I've personally encountered this problem before, and I know that I will encounter it again.

It is troubling to me that, I, as a developer with 18 years of professional experience, and even beyond that having been writing code in some form or another since the mid 90's, cannot rely on my past experience 'nor my publicly viewable open-source projects to help me get a job.

I've delivered working solutions for multiple companies, solved complex technical problems, completed complex integrations involving multiple (often poorly documented) moving parts, maintained existing working systems for lengthy periods of time to ensure the business kept working smoothly, documented and helped improve on processes, fixed some of the most broken and/or brittle code written by other developers who seem to always be moving on to senior leadership roles, and more.

But in an interview setting? None of this is worth a damn. Nowadays, from what I've personally seen anyway, you're very lucky if the interviewer even reads your resume at all before the interview starts. No one cares what you've done in the past. The only thing that matters is whether or not you can solve this arbitrary puzzle with dubious real-world practical use live in front of one or more interviewers you just met in a setting that is very unlike any other setting you've been in doing actual real work.

Because this is the real problem I have with this style of interview. It is not a good test of anything other than a candidate's ability to perform live on a stage. This is a completely different skill-set than what a company is probably (hopefully) more interested in. Day-to-day development work at any company I've ever worked at is not done live in front of strangers whose sole purpose for watching you do this work live is to judge your ability. No, back in reality your actual coding work is usually done on your own. And when you actually are collaborating with others live, it is an actual team effort where you are all working together to arrive at a solution. If one person doesn't know the answer, your team mates will jump in to help instead of sitting back and silently withholding answers while sneering at you. These are incredibly different situations. We're talking objectively night and day differences here.

There are a bunch of software developers out there that seem to not really understand this. They seem to think that your ability to write code and think through a problem should be unaffected by the environment you are working in. That you can think clearly through an algorithm just the same when you are locked in a room alone as when you are set up in front of others you are sharing your screen with while they silently judge you and expect you to verbalize your every thought. These are probably also the same developers who will then, after they finish interviewing a bunch of candidates, head off to Hacker News to unironically rant against having to work physically in an office and how they're so much more effective working from home where there are no distractions. Hmmm. Interesting.

The first I ever remember reading about the idea of testing a software developer's coding ability live during an interview was this now well-known piece by Jeff Atwood about the "FizzBuzz" test. I do remember being a little surprised to read about how many people were applying to software developer roles when they apparently couldn't code at all. Of course, I was pretty green behind the ears back then. And the idea of coding during an interview was very new to me then. None of the software developer interviews I had done up to that point had had any coding in them whatsoever. But as I thought about it more, I decided it made sense, and yeah, I definitely wouldn't want to end up hiring someone for a software developer role who actually couldn't code something as easy as "FizzBuzz."

This problem is something I keep hearing about to this day. But I think what really bothers a lot of interviewers today, and what I think is at least partly to blame for the proliferation of live interview coding exercises, is the lengths that some people will go to in order to cheat their way through a software developer interview. My favourite story about this is one I heard from a colleague several years ago where she was doing a remote interview with a candidate. She was asking them some technical questions (not even live coding at this point) and my colleague started to notice that there was always this several second delay between when she would finish asking a question and when the candidate would start to speak. Initially, she shrugged it off as maybe some lag in the video and/or just the candidate taking some time to think before responding. But at some point she noticed that during the brief pause before the candidate would start to speak they seemed to be moving their lips, as if silently reciting something they were hearing on their end before speaking aloud. This type of problem, where someone else is hidden out of sight while listening in to the interview and telling the candidate what to say in response to each question, is a known problem today and very unfortunate. And there are plenty of other ways in which candidates try and cheat during interviews.

So, even though I personally hate it, I can absolutely understand the interviewer's desire to want to see candidates write code live during an interview. You need some assurances that you can weed out the people who are trying to cheat their way through. This is unfortunately the world we live in today. Everything that can be gamed, will be gamed, eventually. Especially when money is involved.

But why do software developers think that asking candidates to undergo arbitrary algorithm or data-structure implementation coding challenges is the best way to gauge a candidates ability? When was the last time the vast majority of developers out there actually did this outside of a computer science class in university? It is such a very poor test of any software developer's ability. I've yet to see any demonstrable correlation between a developer's ability to deliver real-world working, maintainable solutions and their ability to invert a binary tree, or find specific pairs of numbers in a list of integers that sum to a desired total, or whatever.

*Sigh*

Presumably the position that the candidate is being interviewed for is one where the candidate will be asked to build solutions (e.g. applications, services, integrations) for the team and organization they would be working at should they get offered the role. Why not test for something more directly practical toward this eventual goal while still bite-sized enough to fit into a 60-90 minute interview? As a side-effect of this, since a qualified candidate has (hopefully) been actually doing this over the course of their career, this should help to somewhat reduce the candidate's stress level and allow them to focus on doing things that they actually do day-to-day!

The single best software developer interview I had was actually at my last job. And I'm not just saying this because I happened to pass that interview. In fact, I am very critical of the last company I worked for. I think they did a lot of things wrong. But when I look at just their process for interviewing software developer candidates, I think they did a lot of things right:

• A 60-minute chat with a technical manager. No coding, but there was a systems design problem where the candidate is presented with a simple description of a solution that was built and then a diagram showing the architecture and how the different pieces fit together. The candidate is then asked to point out the problem with this design and their reasoning and what they would do differently. The bulk of the interview is describing the role, the company, and diving into your background and past projects you worked on.
• Another 60-minute chat, this time with two developers. No coding. This is really just a tech chat, where the candidate is asked questions about their background, how they approach different problems, their opinions and experiences with different tech stacks, libraries, methodologies, etc. This honestly just ended up feeling like a random friendly chat with a couple other developers and during most of the interview I'm pretty sure I totally forgot it was actually an interview. Plus this format allowed for tons of time for the candidate to ask questions.
• A 90-minute coding exercise interview, administered by another developer. The exact problem varied depending on the specific role the candidate was interviewing for, but it was always practical. In my case, I was asked to build a RESTful web service using whatever tech stack I like, from scratch that implemented endpoints that were described with examples and would be expected to, during the interview, expose it using ngrok so that a test tool they had built specifically for this interview exercise could call into the candidate's web service and validate the expected results. Basically, you build a web service that can be called successfully by their existing web client. This exercise was highly relevant to the role.
• A final 60-minute interview with someone in senior management (in my case, the VP of Engineering), which was more a discussion about company culture, values, team fit, etc. Not really technical and allowed the candidate lots of time to ask questions.

So, while I think this is still a lot of interviews (I'm firmly of the belief that many different rounds of interviewing is highly indicative of your organization's and/or team's inability to make decisions), the structure was still better than most others I've done or otherwise read about. The structure places some more emphasis on conversation and communication than on raw coding ability, while not totally ignoring their coding ability either. This is important I think because, well, even for software developers communication is incredibly important! If you're an interviewer and you end up just having your candidates heads down coding for the majority of your interview process (yes, even if you're asking them to verbalize everything they're thinking while coding), I'm sorry but you're doing it wrong. You're not spending nearly enough time testing their communication skills. As well as just, you know, getting to know them and figuring out if they'd be a good team fit.

And this is what I think also bothered me about my recent experience interviewing at Docker (as well as other places in the past), as described at the beginning of this post. The interview process as described to me was very heavily coding focused. Too heavily in my mind. Why does any company really need the candidate to undergo three different live coding exercises before you, as an interviewer, can say you've gauged their ability to write code? Frankly in my mind what this demonstrates to me is your inability to gauge their ability to write code. With this style of process, each step along the way you're basically telling me that your reaction to each exercise is going something like "Well, hmm, I dunno, they solved this one fine, but what if they couldn't solve this other problem?" What is wrong with the first problem then? Why are the subsequent problems any better? If you need more people on your team to weigh in (a reasonable concern), then perhaps you should get more than one person involved during each exercise given to a candidate. At this point, really I'd be suggesting you may need to review exactly what you're asking your candidates and how it could be tuned better to get the results you need.

What eats me up inside while thinking about this mess that is the current state of software developer interviews is thinking about a conversation I had recently with a friend of mine from back in high-school. He got into game design in the mid 2000's and has been very successful. Currently he works as a Senior Technical Designer at a big studio that most people would recognize. He was telling me about how his job interviews have always gone as a candidate. He shows up and talks about past projects he's worked on. That's pretty much it. The emphasis is placed on his past work.

Why can't we do this as software developers? While I am not trying to say that the focus should be entirely on your past work, I don't see why you'd want to ignore it either. I've never gotten the impression that anyone I've interviewed with has ever looked at my Github profile which has a number of "interesting" and original projects written using different tech stacks that could easily be picked out in a couple minutes of looking. No, it's not all just forks of existing repositories where I fixed a typo. This is honestly a pretty easy thing to filter for, it doesn't take hours of effort while reviewing a potential candidate.

To be fair though, I understand that some companies get many hundreds of applicants per job posting, so it still might not be practical. But if you're not even reviewing candidate's resumes I seriously question how fit you are to be conducting interviews. I rarely get the impression that the people who are interviewing me have even read my resume prior to the interview starting. No one seems to care what the candidate has worked on anymore. Why do we even have to submit resumes? Why don't I just feed in a list of buzzwords for the ATS system to scan and be done with it?

The whole process is overwhelmingly garbage today. I hate it.