My Problem With Your Interviews
This article comes right after Facebook rejected me after 3 phone interviews, but it is not going to be a hate-post. In fact, I’ve been planning to write it for a couple of months. But now onto the topic: tech companies (Google, Facebook, VMWare, at least, but certainly many more) are all trying to find the best technical talent. (So they contacted me and asked if I’m interested in “exploring opportunities” with them). But how do they do that?
The typical interview (be it a phone screen, or an onsite interview) consists of solving a problem. Some call these problems “puzzles”. They are usually non-real world problems that aim to verify your algorithmic skills and your computer science knowledge. The simple ones include recursion, binary search, basic data structures (linked list, hasthable, trees). The more complex ones require red-black trees, Dijkstra, knowledge of NP-completeness, etc. If you are on the phone, you write the code in a shared document. If onsite – you write it on a whiteboard. So, these puzzles should verify your computer science and algorithm skills. But let’s step back a little and see the picture from another angle.
- what you do on these interviews is something you never, ever do in real life: you write code without using any compiler or debugger. You do that in a limited time, with people watching you / waiting for you on the line. But let’s put that aside for now. Let’s assume that writing code without being able to run it is fine for interview purposes.
- the skills that these puzzles are testing are skills that the majority of developers have never needed. Most people are writing business software, and it does not require red-black trees. What was the last time you used recursion in your business software? So the last time you’ve done anything like that is in college. And many of these problems are really simple if you are a freshman, you did them as a homework just the other day. But then it becomes a bit more tedious to write even things as simple as a binary search. Because you just didn’t do it yesterday. Of course you will be able to do it, but for a little more time, so that you can remember, and for sure by using a compiler. (By the way, the puzzles at facebook were really simple. I didn’t do them perfectly though, which is my bad, perhaps due to interview anxiety or because I just haven’t done anything like that for the past 3 years)
- the skills tested are rarely what you will do in your daily work anyway. Even in these cool companies like Google and Facebook, there are still pretty regular projects that require coding to APIs, supporting existing code, etc. I don’t think you will be allowed to tweak the search engine in your first week, no matter how great you did on the interview
- interview preparation is suggested and actually required before these interviews. Exactly as if it is a college exam. But that’s dumb – you don’t want people to study to match your artificial interview criteria. You want them to be…good programmers.
- focusing on these computer science skills means these companies will probably miss good engineers that are simply not so interested in the low-level details.
Btw, here’s an excerpt from my feedback after my first phone interview with Facebook:
On the other hand, I don’t think having 1st year CS homework problems on interviews for senior developers is a great idea. One thing is – most people (including me) haven’t done this since university, and it looks a bit like trivia questions rather than actual programming.
The problems outlined above are what I don’t like about these types of interviews. And that’s obviously because I don’t like solving these sorts of puzzles. I just don’t like them, they are not interesting for me. You could argue that in addition to your daily job, you can participate in programming competitions (like TopCoder) in order to keep your algorithm skills trained. I’ll give a short story about my high-school years. There were two student competitions – one was about exactly these types of programming puzzles – you are given a number of them for a fixed period of time, and you must submit a solution that covers as many of the pre-defined (but unknown to you) test cases as possible. The other competition was about creating a piece of software at home, and then presenting it in front of a jury. I was a top-competitor in the latter, and sucked quite a lot in the former. Why? Because I hated solving useless, unrealistic problems for the sake of solving them. I liked building software instead. I would probably be good at solving puzzles if I liked them. I just don’t. And these are not two levels of skill – one who can solve complex algorithmic puzzles (superior), and one who can’t, therefore he builds some piece of software (inferior). These are two different types of skills. And both of them are very useful in the process of creating good software. One writes the low-level stuff, the other one designs the APIs, the architecture, the deployment scheme, manages abstraction in the code. So, to get back to the question what I do now in addition to my daily job – I build stuff. I’ve worked on a few personal projects that I enjoyed. Way more than I would’ve enjoyed a TopCoder competition.
Unfortunately these cool companies are hiring primarily the TopCoder-type of people. Which probably works for them, because they have a lot of candidates and they can afford a lot of “false-negatives”. But many smaller companies adopt these interview practices, and so they fail to get the best technical talent. The best article on software engineer interviewing I’ve read appeared just a few weeks ago. Jeff Atwood advised how to hire a programmer, and I completely support his approach.
And my problem with interviews is that they don’t actually verify if you can do real programming work. And obviously my problem is that I don’t like low-level and algorithmic stuff, so I wouldn’t be able to work for cool companies like Google and Facebook.
Important note: I’m not saying you should not know what computational complexity is, how a hashtable works, or how to write recursion. You should, because that is basic stuff that you need in order to be able to write good code. But focusing too much on these things is what I consider irrelevant to day-to-day programming. (And for the trolls: I wouldn’t have passed the 2 phone screens if I was a complete dumbass who can only write websites in PHP and thinks a hashtable is some sort of furniture)
This article comes right after Facebook rejected me after 3 phone interviews, but it is not going to be a hate-post. In fact, I’ve been planning to write it for a couple of months. But now onto the topic: tech companies (Google, Facebook, VMWare, at least, but certainly many more) are all trying to find the best technical talent. (So they contacted me and asked if I’m interested in “exploring opportunities” with them). But how do they do that?
The typical interview (be it a phone screen, or an onsite interview) consists of solving a problem. Some call these problems “puzzles”. They are usually non-real world problems that aim to verify your algorithmic skills and your computer science knowledge. The simple ones include recursion, binary search, basic data structures (linked list, hasthable, trees). The more complex ones require red-black trees, Dijkstra, knowledge of NP-completeness, etc. If you are on the phone, you write the code in a shared document. If onsite – you write it on a whiteboard. So, these puzzles should verify your computer science and algorithm skills. But let’s step back a little and see the picture from another angle.
- what you do on these interviews is something you never, ever do in real life: you write code without using any compiler or debugger. You do that in a limited time, with people watching you / waiting for you on the line. But let’s put that aside for now. Let’s assume that writing code without being able to run it is fine for interview purposes.
- the skills that these puzzles are testing are skills that the majority of developers have never needed. Most people are writing business software, and it does not require red-black trees. What was the last time you used recursion in your business software? So the last time you’ve done anything like that is in college. And many of these problems are really simple if you are a freshman, you did them as a homework just the other day. But then it becomes a bit more tedious to write even things as simple as a binary search. Because you just didn’t do it yesterday. Of course you will be able to do it, but for a little more time, so that you can remember, and for sure by using a compiler. (By the way, the puzzles at facebook were really simple. I didn’t do them perfectly though, which is my bad, perhaps due to interview anxiety or because I just haven’t done anything like that for the past 3 years)
- the skills tested are rarely what you will do in your daily work anyway. Even in these cool companies like Google and Facebook, there are still pretty regular projects that require coding to APIs, supporting existing code, etc. I don’t think you will be allowed to tweak the search engine in your first week, no matter how great you did on the interview
- interview preparation is suggested and actually required before these interviews. Exactly as if it is a college exam. But that’s dumb – you don’t want people to study to match your artificial interview criteria. You want them to be…good programmers.
- focusing on these computer science skills means these companies will probably miss good engineers that are simply not so interested in the low-level details.
Btw, here’s an excerpt from my feedback after my first phone interview with Facebook:
On the other hand, I don’t think having 1st year CS homework problems on interviews for senior developers is a great idea. One thing is – most people (including me) haven’t done this since university, and it looks a bit like trivia questions rather than actual programming.
The problems outlined above are what I don’t like about these types of interviews. And that’s obviously because I don’t like solving these sorts of puzzles. I just don’t like them, they are not interesting for me. You could argue that in addition to your daily job, you can participate in programming competitions (like TopCoder) in order to keep your algorithm skills trained. I’ll give a short story about my high-school years. There were two student competitions – one was about exactly these types of programming puzzles – you are given a number of them for a fixed period of time, and you must submit a solution that covers as many of the pre-defined (but unknown to you) test cases as possible. The other competition was about creating a piece of software at home, and then presenting it in front of a jury. I was a top-competitor in the latter, and sucked quite a lot in the former. Why? Because I hated solving useless, unrealistic problems for the sake of solving them. I liked building software instead. I would probably be good at solving puzzles if I liked them. I just don’t. And these are not two levels of skill – one who can solve complex algorithmic puzzles (superior), and one who can’t, therefore he builds some piece of software (inferior). These are two different types of skills. And both of them are very useful in the process of creating good software. One writes the low-level stuff, the other one designs the APIs, the architecture, the deployment scheme, manages abstraction in the code. So, to get back to the question what I do now in addition to my daily job – I build stuff. I’ve worked on a few personal projects that I enjoyed. Way more than I would’ve enjoyed a TopCoder competition.
Unfortunately these cool companies are hiring primarily the TopCoder-type of people. Which probably works for them, because they have a lot of candidates and they can afford a lot of “false-negatives”. But many smaller companies adopt these interview practices, and so they fail to get the best technical talent. The best article on software engineer interviewing I’ve read appeared just a few weeks ago. Jeff Atwood advised how to hire a programmer, and I completely support his approach.
And my problem with interviews is that they don’t actually verify if you can do real programming work. And obviously my problem is that I don’t like low-level and algorithmic stuff, so I wouldn’t be able to work for cool companies like Google and Facebook.
Important note: I’m not saying you should not know what computational complexity is, how a hashtable works, or how to write recursion. You should, because that is basic stuff that you need in order to be able to write good code. But focusing too much on these things is what I consider irrelevant to day-to-day programming. (And for the trolls: I wouldn’t have passed the 2 phone screens if I was a complete dumbass who can only write websites in PHP and thinks a hashtable is some sort of furniture)
Hi.
If we’re here, reading your blog article after article, we know you’re not a complete dumbass who can only write websites in PHP… having said this, I am forced to say that I can’t completely agree with you, even if I see your point and I partially share your thoughts on this.
First of all: I’ve been doing telephonic technical interviews for my current company for the last year, and I must say that it is quite hard to spot a good developer. Sometimes it is even difficult to define what a good developer is. It depends on the needs of your company at a certain time. Some positions are more interesting to be filled with a junior without preconceived ideas about certain topics but with a lot of will to learn and eager to learn new technologies. Some others can require more specific profiles, ie Spring experts, etc… so I guess every single interview focuses on specific position.
This kind of companies, though, don’t seem to interview for specific positions, but rather than that, they look for intellectual excellence and academic success. This kind of puzzles don’t prove you’re a good developer but they can prove that your brain works fast and adapts to the underlying mathematical problematic to where many computer-related problems come down to at some point. I suppose that is enough for them. This comes from the assumption that if you have that, everything else is secondary and will come alone with good practices and the proper environment. And they think that all that is guaranteed for anyone working for them. Motivation is also a factor that will turn a brilliant (again, from a mathematician-algorithmic point of view) guy into an amazing developer. And you know… they have a point too.
So, understanding that coding under pressure, with no debugger, with no access to documentation… is not the best way to measure how good a developer is already, I think the point there is to find out how good a developer can become. Which is not only more difficult and unfair but a total jump into the unknown. Well, this is what life is after all. Difficult, unfair and a continuous jump into the unknown.
PS: I am not a native English speaker, my apologies if something is not well expressed.
PS2: f*** big corporations, software craftsmanship FTW!
I happen to agree with almost all of your observations. Instead of some puzzler, which usually has an all-or-nothing answer, I show the candidate some crappy code and ask them to critique it, which is *very* illuminating.
Very good reading. I agree, that in 95% CS puzzlers are fairly useless. What should they be interested in, is your code style, code quality, habits, api design, abstraction knowledge.
I think much MUCH more better approach is, when interviewer sits with you to a computer (for example your own laptop or OS image – something you feel comfortable with) and you write together small API, unit tests, think of usages … Interviewer can then observe your coding habits.
As a programmer you will very often be READING other’s people code, so more effort should be put how to write it properly (easy to understand) at first place.
I myself am in last grade of CS studies and I have lot of schoolmates who still write stuff like “x = temp2 = !(a[i++][–j^k] >> 1)” …
Oh and last thing about puzzlers. Did you read Java Puzzlers? Those are puzzlers that I really like, because they are about Java (something I use DAILY), not some artificial problems that you are probably never going to write again.
I haven’t read it, but I’ve watched two presentations by Bloch based on the book.
It is easy to shares your views, but not entirely. First the most you already know, the easier it is to find new solution that combine other existing one.
Second, it really depends where you work. You can work for the “average” IT where you just glue up things in Java or C#… Or you can work for a company that really do the heavy stuff… Think Oracle and DB optimization. Think google and their search engine. Think Facebook and their scalability problem.
Theses things are really advenced and well you want the best people available to work on them. Not great or good peoples. Best ones. And you don’t want just somebody that is able to glue things together with JEE or .NET stack. You want them to know everything, to invent, to inovate.
We both agree that you or me would be able to make a new search engine or a social network. It will take us time. But will it be as great as Google search engine or facebook website (including scalability concerns)?
Remember they want the best… And if you don’t like puzzle and algorythmics, maybe you are not the kind of guys they are looking for.
You are not missing anything. Getting hired by Google or Facebook would shuttle you to Silicon Valley were EVERYTHING costs too much and nothing is worth it. You are not missing out by not getting a position in the area.
You would move here and become an indentured servant unless you are a sales and marketing guy. Technicians are replaceable cogs here – and they do not care that you will NEVER be able to buy a house here.
Look, each of us developers interviewing for programming positions for want or need, will succumb to the horrible and awkward and illogical process too many big and small name companies use when hiring employees. I have been in this industry for over a decade and felt like leaving the corporate programming life solely on the basis that being “tested” and having my skills run through some all-knowing test phase feel more like a gang initiation with a big sense of rejection at the end of the ordeal, than anything else. The process rejects a nervous perfect candidate and allows a seemingly brilliant person a VIP instant-hire because he knows HOW to answer the questions perfectly but technically doesn’t think on his feet. A perfect yes-man is what a manager wants, so he feels at ease at work, but that doesn’t help the product or nurture a creative and mutually beneficial environment. Most of the teammates I have met in my career that I would recommend above me for a swank position at a great company are usually those that might have failed a screening call and or second interview because they flubbed the test or like me don’t have the corporate-cultured “perfect answer (memorized)” in their back pocket yet. When I lead the team, I dread the mechanical way some of attack problems and we end up with solutions that aren’t innovative and the human element defers to least-innovate-copy-paste-from-industry-standard which isn’t how we arrived at NoSQL or any other modern era technology we now fundamentally depend on.
Yes we need this stable, and cohesive sense of industry standards and knowledge requirements but technically two weeks of training 3 new hires of “how we do things around here” would bridge any gap far better than hoping the “right” candidate will be found in the current way we outsource and dehumanize our hiring and recruitment processes. Not to mention that once you get through the gates and get hired, the team culture can often reward the pet developer who behaves and thinks like the manager, turning good ideas that are needed and suggested by others, strongly discouraged.
All in all, this is the harshest of realities to face when working in this industry and I don’t want anybody of any skill to have to sink or swim based on the incorrect belief of hiring managers that this is how things just are and nobody gets harmed by it. We developers employed or not, should not base their career path on how horrible major and minor companies poorly interview and inhumanly reject candidates. We can’t give up what we want to do with our lives because we were not hired by a place we thought we wanted to work at. If you think there aren’t wonderful teams doing amazing work at nameless startups and organizations around the nation and around the world where you can develop and create and be a success, then you are only letting the robots I don’t want on my team win. Don’t let them win! Also, I have worked for groups I thought I wanted to work with supposedly the best corporate and team culture and turned out it was all a ruse, so don’t think you ever pass up a good opportunity. Yes you may take a job you might end up hating months down the road but such is your right to leave and look for a new position elsewhere. You cannot get blacklisted from the tech market, moving is always an option and everything can and will work out. You just might learn something you wouldn’t have if you already “knew” the answer.
I think the 10% of the time the recruitment phase works out for either the new-hire or the hiring manager doesn’t mean there aren’t enough good candidates and that we should keep the process as a way to only find the elite ones, it really means the process is elitist, more about profit and corporate mechanisms than it is about programming.
Good luck to us all, know that hiring managers out there are working to not put candidates through such a inhuman experience and know that their are good teams in places you haven’t heard of and places you end up finding exactly what you thought you would find at Facebook, Google, or wherever you thought was gonna be the best experience ever.
I disagree with @Macluq. There’s something definitely not right with the conventional recruitment processes in hiring developers or technical people. If a programmer performs better under pressure with watchers in his face and with no access to documentation, and a debugger, he’s more likely to fail at some other crucial thing related with the job and not covered at the interview.
Most current recruitment processes of IT people, do provide the employer with a false sense of making the right decision and do not earn them the best of candidates and these processes need to be refined.
Tell me if you witnessed any project, say like those typical 6 month-projects, whose deadline was ever met. And what ratio to those which are extended.
Thanks for the extensive and balanced comments. They fill what I’ve missed in the post, or was too one-sided.
Read my thoughts on this:
http://keyvan.io/why-big-software-companies-fail-at-hiring-good-talents
@Nicolas have you seen the first version of facebook? It is quite easy to build such ugly and useless site nowadays. For the record at this point there were no groups, no lists and complex sharing issues no ajax and pretty effects, no videos and pictures – only one profile picture, no tagging and so on. You just had list of friends and sharing goes to all of them with no further complications. The scalability thing at facebook started with the big impact and the big money from venture capital. At this point they were able to hire any kind of additional devs, sw engineers and so on. Because of the perfect engineering talents they ended rewriting part of their “innovative” php code in c instead of concentrating on better and scalable technology from the beginning.
For me this algorithm part is maybe not so bad approach but it’s too general. A perfect example for that is google. They have one of the best “talents” today. The result is a company full with people that really believe that they work at the most creative and innovative company in the world and that they are the top people. For some it may be the case but from my point of view they are bunch of asocial guys who shouldn’t be allowed to make any kind of social project. Same thing goes for the product managers at least in the social area. The just don’t have fresh and cool ideas anymore.
I believe that a perfect candidate may not be in tact with algorithm and data structure kind of puzzles. A candidate, who is a global thinker, pays attention on the details and is obsessed with the interception between the technology, real world problems and social skills (not facebook like) may not make it to google and facebook. It is possible that a company full with Wozniaks and not one Steve Jobs won’t be successful at all.
100% true. That’s why I stopped aiming large companies several years back. They FAIL to find good engineers while looking for math geeks instead which would have been appropriate 20 years back. In addition to that math gurus often stay away from the business world and practices which is a cons in real life projects.
I have been checking out some of your stories and it’s clever stuff. I will surely bookmark your website.
Skip the interviews. Go to networking events where higher-ups at Facebook go then mingle. Give them a call when you want a job, skipping HR entirely.
Networking is the key to any job in this industry right now.
Awesome article.
I completely agree with your sentiments. Having just done the interview process with a variety of companies I found that most companies were interested in my computer science theory, not whether I could help build their next business application. I have the experience and track record to build sophisticated web apps but many companies wouldn’t take a second look at me because I didn’t know the answer to their trivia questions. Granted, some of them I should’ve known the answers, but many of the questions I hadn’t been asked since my freshman year of college.
Hopefully companies will realize that there is a vast amount of fantastic talent out there that may not be familiar with advanced binary search trees or the minutia details about some API, but who could help build their next application.
Very nice post. I simply stumbled upon your weblog and wished to mention that
I have really enjoyed surfing around your blog posts.
In any case I’ll be subscribing to your rss feed and I hope you write again very soon!
Seeing that Real Racing 3 is out, we are going to provide some tips to get
the best situations and possess the most enjoyable
with Real Racing 3. All that by having an interest to easy methods to limit any real cash
commitment inside adventure.
Serious Auto racing is really the most wonderful racer on any portable
console, and it is free, so there is absolutely no good reason to not
ever try it. I have used Real Racing 3 for about 30 time all round now, i want
to give some the best way to discover the farthest
attainable from the shortest stretch of time and without having to pay anything.
Real Racing 3 from Electric powered Artistry can be acquired for iOS and Android instruments.
It can be a Pretty fairly race match along with some fantastic going science
every van appears distinct. It can be a tough game for
individuals who are utilized to the twitchy-cruising online games, as this is closer to Gran Turismo
or Forza line with regard to driving a car fidelity.
I’ll give you the download links then strategies and tips, as this is not a tutorial of
the game.
Hey there this is kinda of off topic but I was wanting to know if
blogs use WYSIWYG editors or if you have to manually code with HTML.
I’m starting a blog soon but have no coding knowledge so
I wanted to get guidance from someone with experience.
Any help would be greatly appreciated!
Look at my webpage :: garage door repair
It’s enormous that you are getting ideas from this paragraph
as well as from our dialogue made at this place.
Hi, I do believe this is an excellent web site. I stumbledupon it 😉 I will cokme back once again since i have
bookmarked it. Money and ffreedom is tthe greatest way to change,may you be rich and continue to guide other people.
Now that you know the number one secret, taking action, you are one giant step ahead of so
many other online business owners. No matter what you buy or what
you try, you never seem to make any more than a couple of sales every month.
Research any job you find before giving out your
personal information or doing any work, and never pay someone to hire you.
I believe that is one of the such a lot vital info for me.
And i’m satisfied studying your article. However wanna remark on some normal
issues, The web site style is ideal, the articles is actually nice :
D. Excellent job, cheers
my web-site agen Judi Poker
Brazilian virgin extension and wavy Indian Remy wigs are widely
used among ladies and have been appreciated.
I just like the helpful information you supply for your articles.
I will bookmark yoiur weblog and check again here frequently.
I am quite sure I will learn plenty off new stuff proper right here!
Good luck for the following!
my weboog :: judi kartu texas poker
Gosh..i have amazon assessment next week and i hate academic questions. They seem so academic after 15 years of experience.
Great article Bozho.
Totally agree that most programming tests in businesses are completely broken/not fit for purpose.
We need a new angle, a different angle when hiring.
I put together an article that compliments yours. It touches on ideas that are important when hiring a programmer for non-techies.
You can find the article here: https://tests4geeks.com/hire-a-programmer/
Thanks again for the great article and keep up the great work.
If you are very good solving real life problems, you must create your own bussiness, not being looking for a job.
Solving real life problems means that you can create something that gives value to the people, something that you can sell.
Good article! We will be linking to this great
post on our website. Keep up the good writing.
I have similar experience like what you have. But the test makes even less sense. I needed to finish the code, not only the pseucode or the rough-code, passing all of those test cases. They gave me less than 2 hours to finish 3 tasks perfectly and they must pass the test cases. The tests are actually not too academic although they still involve pretty complex algorithm and data structure. They are solvable and pretty interesting. If only they gave me 1 – 2 days to finish it. Not 1 – 2 hours. 🙁
@Nicholas, your logic is like saying that all low-level system developers who build libraries and can optimize databases are the best and the higher level developers who are used to build software are just inferior developers who can only glue things ??? Lol… most developers don’t do these sh!t in their daily jobs, they are used with something else.