In February of 2016, we published a post about how people’s technical interview performance, from interview to interview, seemed quite volatile.

In other words, technical interview outcomes do really seem to be kind ofarbitrary.

Performance from interview to interview really is arbitrary.

The y-axis is standard deviation of performance, so the higher up you go, the more volatile interview performance becomes.

In a previous post, we talked about how women quit interview practice times more often than men after just one bad interview.

If you’re hiring people, in the absence of a radical shift in how we vet technical ability, we’ve learned that drawing on aggregate performance is much more meaningful than a making such an important decision based on one single, arbitrary interview.

Not only can aggregative performance help correct for an uncharacteristically poor performance, but it can also weed out people who eventually do well in an interview by chance or those who, over time, simply up and memorize Cracking the Coding Interview.