As someone that’s given well over 600 technical interviews across multiple companies:

A well designed panel of interviews shouldn’t need to change based on the level of the person being interviewed (on a technical IC track at least). Everyone should get one algorithms focused interview, one “general programming” interview, one system design focused interview, a manager interview that focuses on fit-for-team, and a final non-technical interview that focuses on “fit for company” and ideally is given by someone not in the same directorship (or boss+2 tree) as the team hiring.

What should change depending on the candidate’s target level (not their on paper level, levels don’t transfer across companies at all. I’ve interviewed people with principal engineer titles (L8) from their previous company that wouldn’t pass the Software Engineer 2 (L4) bar at mine) is the signal you expect to get out of a candidate from those interviews.

For example, on the most high performing team I worked on, our “general skills” technical interview consisted of two parts.

The first was a 20 minute code review. We had an ~300 line “PR” that we would give the candidate, and ask them to give feedback on it. A junior candidate might find the basic, obvious syntactical or logical errors (the straight defects in the code). A mid to low senior might find that some of the code could be replaced by standard library facilities (IE “this for loop is just std::find”). A high senior might find flaws in the actual design of the functionality and suggests potential improvements to the API to better serve the original requirements.

The second was implementation of some basic data structures with a few wrinkles put on top (for example, a hash table with O(1) deletion of a random key while maintaining the rest of the O(1) properties). We actually let candidates use any existing facilities for these, since the point was to test how you might extend functionality with a wrinkle. The more senior the candidate, the more we would expect them to finish, and the more we would expect them to understand where to use something that already exists vs writing a purpose-specific optimization.

The same “scaling difficulty” applied for the algorithms question. We had answer keys for the stock couple of questions we asked that broke down what things we would expect candidates of different levels to know, and what follow up questions to probe them to test that knowledge. We actually didn’t care if they wrote functioning code at all for this one, pseudo code was fine, since the point was to test algorithm understanding and critical thinking skills. I once had a particularly bright junior candidate actually derive breadth first search during this interview from first principles because it didn’t click that that was what he was doing for him till he finished. He didn’t actually finish the problem, but that was more than enough signal to pass a junior candidate. Whereas a senior candidate not knowing BFS would have been harder to pass (keep in mind this team worked in a problem space directly concerned with graph searching, so BFS is domain relevant knowledge).

I guess what I’m saying is: the problem is not asking the same questions to candidates of all levels. You can get very good signal from doing that. The problem is asking bad questions with no clear understanding of the signal you’re trying to get out of them. And this happens because many companies severely undervalue interviewing and the general hiring process at their companies. For example it took one of the senior engineers on my team over a week to come up with that sample PR and its answer key, as well as several rounds of trialling it against other people on the team to get feedback. And then we spent time training everyone on the team on how to give each of the interviews. We took it really seriously. And the company itself had extensive interview training and guidelines (that some teams took more seriously than others). The debrief process during the hiring panel was also focused not only on making a hiring decision, but giving feedback to the interviewers on how they could improve their interview skills to get better signal in the future. That company had an incredibly good track record of making quality hires even when it was in hyper scaling mode.

On the other hand, other companies I have worked for, despite having similar interview panel compositions, don’t take the process seriously at all. There is no standardized processes, interviewers are not calibrated against each other to have similar ideas of what constitutes junior/senior/staff level performance, there are no standardized question banks with answer keys, forget being tuned to domain knowledge relevant to different teams. Hell they don’t even require debriefs. Suffice to say, these companies do not have good track records hiring talent, and it shows. The candidate experience also sucks, because it winds up seeming like a total crap shoot if they get hired.

TL;DR is that hiring for software engineers is really, really hard. Companies that invest in interview process and care about both hiring outcomes and candidate experience do better. But no one really knows for sure how to objectively evaluate skills as complex as those needed for software engineering in a series of 6-7 hour long interviews. It’s easy to fall back then to blindly doing leetcode interviews without thought at assume that’s going to be “useful” signal, but really all that tests is how well the candidate studied leetcode problems (and thus, generally, how recently they were in university). Those companies generally have bad track records of hiring good candidates.