Tech Interviewing is Broken (Part 1 of n), How we got here and why we're still stuck here.
This is the Internet of Bugs. My name is Carl and today we're talking about tech interviewing is broken. This is going to be part one if I don't know how many parts. My first run through this material ended up being 73 minutes long. So this will be a series. I'm not doing all that to you today. Important note, I am
not selling anything in this video. No product placement, no endorsements. Any ads you might see are added by YouTube. I have no idea what they are. So I'm not trying to push a product or a service on you. I don't have any great magic formulas for job seekers. Sorry about that. If I
knew a magic handshake, I would tell you, but I don't. I made another video about this too. We'll talk about that in a little while. This is my opinion and experience.
Honestly, aside from a handful of people that might have been an upper management at some of the tech companies that were part of the like no poaching agreement between some of the big tech companies, I don't think there's anyone in the industry that's in a position to really, really know exactly how we got here. So I think my opinion is as valid as anybody else's. I don't know that there's really any real ground truth that's available. So this is my perspective. Hopefully it's
useful. I'm only going to talk about the U.S. stuff here. I've never tried to interview in another country. I have no idea what happens in those kinds of situations.
I don't know how relevant this will be to you. If you want to bail, I don't blame you, but you're welcome to follow along if you think it might be helpful. This video is mostly about where we are and how we got here. There will be videos coming later on how I think we might be able to make things better and some of the other underlying causes. Only the employers can fix this and only one company at a time.
There's no magic bullet. I'm not going to be able to give you a solution that's going to make the whole problem go away. It's it's a mess and it's going to stay a mess for a while. For those of you
that are looking for a job right now, there are a ton of people out there that are trying to take advantage of your desperation to rip you off. There are a bunch of people that are packaging up stuff that's available on the Internet for free and charging you several hundred dollars for it. If you want to do that, if you want to have it all in place, so be it, but I have never seen any evidence that there's any course-and I've looked at a bunch of courses and a bunch of books-and I have never seen any evidence that there's any course out there, any "How to pass an interview" thing that's any better than what you can find on the Internet searching for free these days. Back
in 2008, there was a book called Cracking the Coding Interview. I have a copy under somewhere. I got it while I was interviewing at Amazon. It was a really good resource. Pretty much everything that's in it is online for free now. You can get it if you want.
I understand there's a new edition, but pretty much what's in it is generally available information now. If you are wanting to buy something, it might make you feel better. In my experience, it says if there's any effect at all, it just can be a placebo effect, but if you want to, that's fine, just if you're going to spend hundreds of dollars on something, please make sure it's a good source. Please make sure it's someone who actually has been through it all and knows what they're talking about. If I had to recommend a book
for job hunters, believe it or not, it would be whatever the most recent edition you can find of a book called "What Color Is Your Parachute?" You can probably find it at a used bookstore. It's been around since the 70s. They updated a lot. It's actually really useful. Still, not very many tech people look for jobs that way. It's all
about personal networking and finding people and that kind of thing. It's actually really useful. It's not tech-specific at all, but if I had to recommend a book to help people with job searches, it would be that one. Once upon a time, I actually got fired for reasons, and as part of my severance package, I negotiated a line item to make the company buy me a most recent copy of What Color Is Your Parachute. That's a long story. I might tell it one of these days. Anyway, when
I am between jobs and I'm in between jobs a lot, and we'll talk about why later, the only things that I spend money on are either networking related, so like LinkedIn Premium, which lets me find out who's working at a company, sending them blind messages to ask them about whether that company is good or not, that kind of thing. I spend money on education-related stuff, courses, training, and stuff to let me learn about technologies that are currently hot and people are interviewing about. Lately, it's been PyTorch. I've been
learning a lot about that. That's been fun. So anything that I haven't done before that's currently hot in the job market, I might spend money to learn about that. But aside from that and LinkedIn, which I use to find people at companies and talk to them, that's all I spend money on. LinkedIn is not paying me to say that. I'm not going to give you an affiliate link for LinkedIn. If you
want to do that, that's great. If you don't, that's great. But that's honestly what I do, and I'm not trying to sell you anything. And I don't think it's necessary, but I think it saved me some time in the past. So when I passed my
interview for Amazon back in 2013, I think, 2014, maybe, I watched all of the Intro to Algorithms class from MIT on iTunes University for free. If it's not still on iTunes University, I don't know if iTunes University actually exists, but there are lots of classes on the Internet for free from reputable universities. I did the homework. I actually had the textbook already. I think it was an edition older than the one that they used because I have a lot of books oddly enough. So having the textbook for the class that you're following along with might be useful, although, honestly, a lot of them you can find for PDFs online these days, especially if you're going to get an edition or two back. I filled up almost all of one
of those composition notebooks working out problems by longhand. I would just Google, you know, I had a list of like topics, binary search, linked list, that kind of stuff, and I would just Google, you know, "Interview problems, This topic," I would print off some of them and I would just sit with a paper, read the problem, write it all down on my book, no notes, no autocomplete. It was completely miserable. And then I would go back and grade myself later. I generally didn't do the same problem over and over again unless I screwed it up and needed to do it again. But there's no substitute in my experience for actually working it out longhand typing is just not the same thing. So listen to
this very closely. The rest of me getting hired at Amazon was just luck. You can prepare yourself, and you need to if you're going to go through one of those LeetCode interview things. But I honestly do not believe that there's a single person at any of those companies who got through one of those whiteboard coding interview processes or online HackerRank coding processes that, if given a different set of questions or on a different day, might not have failed it. It's strictly a matter of "Did the person who's interviewing you happen to ask questions that still happen to have been fresh in your mind from the last time you worked through those problems?" And if you haven't worked through one of those problems or you haven't worked with one of the problems that you get asked in a long time, chances are you're going to fail and there's nothing you can do about it. It's just a dice roll. The people
that work at those companies all believe that they're better than the people that didn't get hired at those companies because they passed the interview process. And I have never seen anything that leads me to believe that it's not just blind luck of who can who manages to by chance get questions that they know the answers to or not. No one can answer every possible question on every day without making mistakes. It just it can't
happen. Sometimes you draw a blank and it's over. Sometimes you don't remember that particular kind of question. It's over. Sometimes the solution that you have is not the one the interviewer has in mind and it's over. It's just a dice roll. There
is no substitute for practicing to get yourself in a position where you might get lucky. I wish there was a silver bullet, but in 35 years. I've never seen it. I've never seen any evidence of it. There are people out there that paid for some kind of course and they passed the interview. I'm sure that there are people who went through the exact same course and failed an interview. The people that passed want to believe that they're just better. I've
never seen any evidence to support that. There's no silver bullet. Once you prepare yourself to the extent that you can, after that, it's just a dice roll.
So, that advice plus another video that I'll link, which is called the "Programmers real problem with AI" which is all about the war between the AI bots that screen job applications and the AI bots that auto submit resumes and ChatGPT4 created cover letters. That constitutes the advice I have for job seekers. The rest of this video is all about kind of where we are. So my history. I've been doing this for 35 years. I
started my first job out of college at 1993 at a company called EDS. I have worked at Amazon. I have worked at IBM. I have done a bunch of consulting. I've done consulting
gigs for Motorola, AMD, ExxonMobil, Amaco production, Sun Microsystems, when they still existed, Sprint Telecom, when they still existed, Motorola and AMD, I'm going to stop before I start repeating myself, but, like, lots of companies. In 1995-ish, I worked for a consulting company and I wrote the guide for basically how to score a candidate after an interview so that we would all, all the different locations would hopefully be close to the same answer for the same candidate. The company I went to after that was another consulting company. There, I got a reputation for being one of the more difficult interviewers.
I tried to be fair, but I was a very intimidating interviewer. The secret to being an intimidating interviewer is to ask someone a question that you know that there's no chance they're going to know and then just look at them like, "Hmm, I expected you to answer that," and then just go on with the interview. It makes people very nervous, freaks them out. It's not necessarily the best thing. I kind of feel bad about it sometimes, but it actually makes a big difference. It makes more difference when you're interviewing someone who's going to be a consultant, who's going to be on a client site by themselves and who needs to be able to handle more of that kind of pressure.
It's still not a nice thing to do, but it's actually, when you're being interviewed, it's actually not that hard for someone to seem a lot smarter than you even if they're not just because they have home-field advantage. Once upon a time, the consulting company I worked for bid on a project, and they didn't really expect to get it, and we got it, and I spent two, two and a half weeks interviewing like 16 people a day. I had half hour slots back to back to back to back, booked with an hour for lunch for eight hours of interviewing a day for like two, two and a half weeks. By the time I got through of all that kind of stuff, I had a pretty good handle on interviewing, or at least I thought so. So hopefully what I have to say is relevant. I have worked as an employee and as a contractor at both companies that use LeetCode, whiteboard kind of interviews, hacker rank, online code sharing kind of interviews, and companies that didn't do that. I have not seen
any evidence to believe that the people that use the whiteboard Leet Code HackerRank interviews are better. I have seen no evidence that the people there are smarter. There are some differences, we'll talk about that later, but I have seen no evidence that that leads to better outcomes. None of the highest performing teams I have ever been associated with use LeetCode interviews. All of the best performing teams I ever worked with had three things in common.
One, they were all smaller companies. Two, they all had very clear goals. We were all working toward the same thing. We didn't always agree on exactly how to get there, but we were a startup or we were a small part of a bigger company and we had very clear success criteria for what we needed to get to. And then three, we had openness and willingness to let people go if they weren't working out. And we talked to people about
that during the interview process. We're going to bring you on. We're going to work through this. It's going to be hard.
It's going to take a lot of effort. If you're not working out, then we're going to have to let you go. Some people self-select after that and just decide they don't want to deal with that. That's fine, but it allowed us to do more flexible interviewing. It allowed
us not to have to really care that much about how well somebody could code because if we ended up stuck with someone who couldn't hack it, then we would be able to let them go easily. I don't know that that's the best thing. I don't know that that's the most ethical thing. And certainly those were
really stressful, really hard-working teams, but those were the teams that I was with that were by far the best performers. And there's some of the ones that I look back on as being the most-that I've been the most proud of what we managed to get done. I have not seen a difference between live whiteboard coding interviews and take home coding interviews in terms of outcomes. You would think that the take home interviews would be easier, but the expectations are a whole lot higher when you spend more time on it. And there's way more subjectivity. There's way more "You didn't use the design pattern of the week" kind of grading in my experience. So in my experience, you're just as likely to fail one of those or a person is just as likely to fail one of those. And it doesn't
actually help any in terms of getting a better quality of candidate. Either one of those you do is going to end up with the same outcomes in my experience. Okay, so how did we get here? Back in 2004, which was the year that Google went public, so we're talking about all the long, long ago. There was a book published that I don't actually have a copy of with me anymore called "How Would You Move Mount Fuji?" And it was a book about the kind of riddle stuff, the Why are manhole covers round?", and "How many manholes would you find in Seattle?" or those kinds of things that Microsoft used to do. This book, at least from what I remember, is the place that basically started the industry-wide obsession with recruiting superstar candidates. I don't believe in superstar candidates. I've got a lot to say
about this, but it ended up getting cut for time. It's going to be in another video. That video is tentatively titled "The Myth of the 10x Developer." So when I get that recorded, I will put a link to it here and in the show notes. After that, there was on the Internet in 2006-ish, I think, a thing called the "Microsoft Underground Hiring Manual" or "Guerrilla Hiring Manual." It was not a "Guerrilla Guide to Passing an Interview". It was
a "Guerrilla Guide to how to screen people out of your interview process." I think it just made things worse. The guy, Joel on software, who's one of the guys that founded Stack Overflow, I've read an interview with him if I can find it. I'll stick it on the show notes.
That admitted that he was part of the what broke interviewing in the first place. Next thing, this was just dumb. I've got a link to the WebArchive article or archive of this thing. There was an article or a blog post or whatever on a thing called FizzBuzz, which is basically a really simple, really stupid kind of program that people used at interviews. One of these interviewer people, the one that wrote the article, asserted that a very high percentage of the people that they interviewed could not pass this simple programming test. Even if
you look at the bottom of that first page back in 2007, 2008, whenever the hell it was, the comments on that page are like, "Huh, I've never seen anything like that." But a bunch of people in the industry just took it as gospel, that the vast majority of the people who were interviewing for developer positions couldn't code at all, and that you needed to screen them all out. That started this whole, "You have to have coding interviews and you have to do whiteboard things," and it just turned into a nightmare. I think that's honestly garbage.
There might have been a handful of people that I've interviewed over 35 years that I don't think could have passed that kind of easy interview or easy programming test. Those were people that I knew within the first five minutes didn't know what they were talking about. I don't know if that was just for clicks. We didn't really have clicks like that back then. I don't know. Maybe he had just a really bad source of a bunch of people coming out of a university or a bootcamp kind of thing or something that just didn't know what the heck they were talking about. I have no idea,
but that never made any sense to me at all. But man, did that become the zeitgeist. Keep in mind, all the data that we have on interviewing, all the actual data, is held by private companies or public companies, and they're all incentivized to be better than their competitors, and even to the extent that they release studies or things that are the truth that might not be the truth or might not be all the truth. So it's possible that some of the things that the industry has internalized as being very important about interviewing are things that maybe people posted just to try to throw their competitors off. It's been kind of
a cut-throat thing off and on over the last few years. It's quite possible that a lot of the things we know were just because the Googles and Microsofts of the world were trying to maintain their advantage. It <i>is</i> definitely true. I have experienced this many, many times, that other companies, not big companies, especially startups, smaller companies who wanted to be big companies who aspired to be the next Google or whatever, tried to emulate what they saw or heard that the successful companies said that they were doing. It made investors think they had a clue.
It made them think that they were following best practices, and I don't think it actually helped, but man, it spread like wildfire. So in 2013, I'll put a link to this article. Google's senior vice president of people operations said that scores from this kind of interviewing had no correlation with job performance at all. Now, like I said, he was a Google at the time. He might have been trying to
lie to throw his competitors under the bus. That said, he's left Google for quite a while and as near as I can tell, he's maintained that position the whole time. So I don't think he was lying at the time, although I can't actually prove that. There are some incentives that companies have to continue doing this kind of whiteboard puzzle interview crap that fails a bunch of people. One of them is that companies just don't know. if they weren't going to do that, what would they replace it with? and whether that replacement would be effective or not? I have a whole separate video coming about the way that I like to interview people and things that I think are more effective. At the moment, the
tentative title of that is called, "If not LEET, then what?" We'll see what it actually ends up being called. And when I post it, when I get it done, I will put a link to it here and in the video description. One of the reasons that companies continue to do that is it's kind of like a hazing ritual.
The folks that work there are like, "Hey, I had to go through this, so everybody else has to go through this as well!!!" And there's a potential for a riot, if you were to tell people, "Hey, the new people are going to come in are not going to have to go through what you went through." So there's an incentive to keep doing it because of that. The employees that do get into a company think that they're special because the interview process was such a gauntlet. And they think that they're smarter than everybody else. And that can be advantageous to the people running the company. It cultivates this us versus them mentality that encourages camaraderie and loyalty and that kind of thing. It also, and this is
actually, I guess, useful, but I don't think it's healthy. It does identify the candidates who want so much to work for that company that they're willing to spend a ton of time practicing and learning stuff that has no bearing on any job in order to be able to get even to the point that they have a chance of passing that interview. So it does cause people, it does allow you to filter for people that are very motivated to want to work at some kind of big LeetCode interviewing company. I don't know that that's healthy, and I don't know those people are any better than the people that wouldn't go through that effort, but it does do that for the companies. The other thing, I don't even want to say this. So. The high interview failure rate supports the big companies' narrative in the press and in lobbying efforts about their assertion that there's a lack of skilled programmers and skilled workers in the US. Any
time I mentioned anything about the government or lobbying or any of that kind of stuff, people come out of the woodwork to cheerlead for their favorite political party or to whatever. Anything that looks political, <i>remotely political</i> in the comments, I'm just going to delete on site. So please don't do that. That's not what this channel is about. We're here to talk about fixing bugs and fixing computing and fixing the industry. Politics don't have a place in this channel. So if you want to argue about that, argue about it somewhere else. Okay, so that's all I have
to say about that. So there's a belief amongst the big companies, at least some of them that I know of, who believe that people who have been through this kind of interview process are-at least one of the companies-the word they like to use is "fungible", which is the same "F" in "non-fungible token" or NFT. They want to believe that basically if you put everybody through this kind of interview process that you'll be able to move programmers around like puzzle pieces or not like people pieces, like bricks in a wall. And it doesn't matter where you put anybody and you can just reassign people willy-nilly. I
don't like being a brick like that. It matters to me what kind of technology I'm working on. So that doesn't endear me to the process. But companies do believe that being able to move programmers around it will is useful to them. So that's one of the reasons that they like going through that process still. It also
creates a standard and uniform process that does not rely on a lot of skill from the actual interviewers that may be less likely to result in discrimination lawsuits. If you put everybody through the same set of questions that come out of the same question bank and you have the same set of, "Well, you made a syntax error, so we're not going to hire you" kind of criteria, then it might be harder for you to... I mean, you know that the big companies are going to get sued because they have deep pockets, whether they did anything wrong or not. And having this kind of really arbitrary, really well-defined process might help them from lawsuits, or at least that's what I've been told by folks in legal departments at big companies, for whatever that's worth.
It allows the interviewing to be spread across a ton of people at the company without having to give them a whole bunch of interview training or practice. And that lets them hire a whole bunch of people because they need to hire a whole bunch of people because one, they want to grow and two, they're chasing a whole bunch of people off in their turnover rate is crazy. So if you want to hire a bunch of people and you don't want to have all of your best people who are actually writing code have to spend all of their time on interviews, then being able to spread the interviewing out across as many of your developers as you can is vitally important and this is a way to do that. It should go without saying, but this interview process has bad consequences, not just for the industry as a whole and not just for the interviewees, but for the companies themselves. It creates very insular cultures. People get very arrogant, they don't want to look outside the company for a better solution.
I've had a lot of headbudding arguments with people because there was a better solution outside the company, especially like, you know, "Look, it's an RFC, a bunch of really smart people made a standard for this. Why don't we do this the standard way?" And the answer is "Well, Because we think we're smarter than the... Well, not the stated reason, but the actual reason is "Well, because we think we're smarter than the people that actually wrote the RFC. So we're going to do it our way." And sure enough, several times I had to deal with the consequences of bad decisions that wouldn't have happened if we had just done what the RFC said, because the RFC thought of that and the people that were inside the company that set up the internal standard didn't. And of course, it causes people who would be great hires and would be great people to have on the team and to be very productive not to get in either because they don't want to go through the effort because learning all the LeetCode stuff is not actually helpful to an actual job or being a better developer. And they don't
want to go through the effort. So they don't get in or because they happen to have missed one of the questions because they drew a blank that day. And that's very unfortunate. You don't think about this, but it also causes a bunch of people with no actual experience, but a bunch of algorithm knowledge to get hired.
And those folks end up under a lot of pressure. I have talked to a lot of people I've spoken at conferences. I've run meetups. I'm talked to one person once who had a master's degree in computer science from a university I had heard of. And from the conversation, I didn't ask this directly, but from the conversation, I am pretty sure that that person had never touched source control and never written a program that wasn't intended to be printed on a dead tree and graded before they were interviewed and hired at one of the big companies. And they were freaked. They were under a ton of pressure.
And that was just-I felt really bad for them. Like I said earlier, it allows people to it allows companies to get rearranged people like bricks. And a lot of people don't like being a brick. I don't like being a brick. You
know, I don't like being told, "Oh, you really like these languages. You don't really like JavaScript. Well, we're putting you on a JavaScript project anyway, because we said so." For a lot of
people, that's a deal breaker. I hate that for me. I don't want to work at a place where they're going to tell me, you know, "Hey, we've decided you're going to work on this stack." And especially not since a lot of times, people that don't actually understand the problem or that hasn't-we haven't actually talked about what exactly the problem is yet- pick what technology and what stack they want to use before they actually decide what the requirements for the project are. And if those two things don't match, it ends up being a giant mess. You really, this sounds obvious, but a lot of people don't get it.
You really ought to take the requirements and figure out what you're trying to accomplish <i>before</> you decide what language and what framework you want to use to do that. I don't know why that's hard so hard for people to understand. Maybe it's just because a lot of people only know a handful of frameworks and languages. And so they-that's the only hammer that they know. And so everything must be a nail. But man, you really ought to do it the other way. You really ought to
decide what it is you are trying to do. And then once you know what you're trying to do, figure out what language is a good fit or languages or frameworks or good fits for that because. Oh man. So if you are job hunting, if you are trying to decide, or you're switching and looking for another job, if you have a choice between a company that does Leet Code stuff and a company that doesn't do LeetCode stuff, I recommend you pick the not Leet Code one. In my opinion, and maybe you're different, but in my opinion, the downsize created at a company by using that stupid whiteboard LeetCode interview HackerRank interview process is just it's worth avoiding if you can. It really is. Maybe if you get a whole bunch
more money or you just want the big company name on your resume, it's worth doing for a while, but it doesn't make a healthy environment. It doesn't make a fun environment. And I recommend you try to avoid that. Anyway, hope that helped. Until next time, don't forget be civil in the comments, anything political I'm going to nuke immediately. Always remember: "The Internet is full of Bugs, and anyone who says different is trying to sell you something, likely a scammy, "How to ace your big tech interview for the low, low price of several hundred dollars" kind of course. Let's be careful
out there.
2024-04-19 14:12
