How to deal with drop in quality of candidates?
Posted by dExcellentb@reddit | ExperiencedDevs | View on Reddit | 244 comments
I've interviewed many people over my career. About a year after chatgpt's release I've noticed a sharp drop in candidate quality, especially new grads. It's like they can't understand code anymore. I always ask candidates to implement a tictactoe game, extending to an n-by-n board, then supporting multiple game instances. AI use is not allowed. This isn't some brain teaser leetcode trivia. It's actually testing one's ability to decompose problems and write maintainable code.
Before AI, people were at least able to make progress. In fact, new grads sometimes performed better than experienced people. Nowadays, it seems that everyone has a hard time building even the initial phases, especially new grads.
I'm not sure if I'm an outlier so would love to hear if you folks are experiencing the same thing. I still want to test people's abilities to reason through code because in an enterprise setting, codebases are complex so AI is prone to hallucination. Plus, problems are multi-faceted, complex systems need to coordinate in non-trivial ways, and teams need to collaborate. An engineer needs to think well and seeing how they reason manually through problems is a great way to test that. Unfortunately most of the people I've been interviewing turn out incapable. So I'd appreciate some tips on how to deal with this situation.
Apprehensive_Ad_7451@reddit
It seems unlikely that all the people in the world suddenly became worse candidates.
It's more likely to be either something having changed with your hiring procedures, or your perceptions.
Foreign_Addition2844@reddit
Are you a game dev studio? If not, you could try asking something more closely resembling the work they would do. I have 20 years of experience and make 250k. If someone asks me a puzzle I am just quitting the interview.
WearMediocre6830@reddit
Amen.
I don't want to be rude with OP, and working in EU, where I know our interview styles are more real-life oriented, but this assignment looks like asking your cook to use a wood and stick fire...
GroundbreakingImage7@reddit
I'm going to assume that you don't know how the game of tic-tac-toe works. Because implementing an n by n tic-tac-toe game is pretty trivial. It's the bare minimum I would require for somebody who has programmed for more than a half a year.
WearMediocre6830@reddit
I'll close my eyes on the free insulting and answer as a 30+ years of adulthood (not experience, that seems to be the only self jerkoff metric around here).
A smart interview could also assume (just as you did) that passed couple of years of experience, the candidate does know how to implement a fantasy theorical assignment. Mostly, there are so many ways during the process that would allow to expose the minimum level of a candidate.
Ill double down on the initial comment, I too would leave the interview in this case as I am not applying to a 3 month coding holiday bootcamp.
There are so many things that your comment echoes from this sub 😂. The more I see answers like yours (pedantic asf on a tic tac toe), the more I feel confident about my job safety. This whole "look at my amazing line of code bro, my tic tac toe just rules man, i have 10337 YoE trust me bro" and "IA iS SlOp It cANt BuILd A CoMplEx ApP" is the exact reason why most of the people here are actually going to be replaced. This whole dev mentality has exhausted the industry and you are exhausted me. You sound like wrong professional (insisting and professionals and not line of code technicians) and colleagues (as human).
Pfiou. I guess I needed to take it out
SummeR-@reddit
Im sorry, this is something freshman undergrads implement in cs 102.
I wanted to try and see how hard it would be, I spent 3 minutes to write a basic version in Java.
Its literally the basics of using an array.
ProbablyNotPoisonous@reddit
Actually implement, or, like, pseudocode? Because I could write enough pseudocode to explain my thought process on a whiteboard in an interview, but actually making a functional tic-tac-toe game in any particular language is a several-hours-long process at least, including testing and debugging.
misogrumpy@reddit
OP only said that candidates used to “make progress,” not that they did the whole thing.
rando-on-street-said@reddit
I want the teammate that takes hours, not the bug jockey who takes minutes.
VisiblePlatform6704@reddit
It's been a while since I hired people, but I also did a tic-tac-toe game as an interview. I gave 1 hour, and it was pretty decent time:
I gave the candidate 6 steps to implement:
write a class/function/variable that will contain the board and print the empty board in the terminal as -|-|- , 3 lines
implement a setposition(..., 'X'|'O'), they choose the logic of how, 1-9 or coordinates. (here, they have to manage used positions throwing exceptions.
ask the user for a position and place an X in that position using the previous function. Deal with the exceptions.
write a function that places an O in an empty space
write winning detection logic
- write a game loop to place everything together.
I told them not to worry about finishing it, but just get as far as they could. They shared screen, and I gave them tips if they got stuck.
It was all console based, in their language of choice.
It was interesting to see how people didn't know Exceptions. When using python/ruby/JS they used for loops instead of map/join/reduce . People that struggled printing the board. Things that I would consider simple.
I got pretty good at reading candidates skills from that hour.
XenonBG@reddit
Indeed, how is this comment not higher up? How long does the OP's interview take?
kerrizor@reddit
I mean, I’m not jumping through your leetcode hoops. 🤷♀️
rando-on-street-said@reddit
What’s the phrase? TC or gtfo?
ablaut@reddit
This is an "I hate X types of interviews" post from the other side of the table.
not_you_again53@reddit
I screen a lot of candidates and yes the floor dropped once take homes became “paste from AI” and live coding became I can’t think without it”. What’s worked best for me is making the first 10 minutes purely reading and explaining a small existing code snippet, then one tiny change request (like “add support for multiple boards”) before any greenfield build, you instantly see who can reason vs who memorized patterns. Also if your pipeline is ATS heavy you might be selecting for people who AI polished their resume, I’ve seen that a ton and it’s partly why we built our services around pre screening for real code comprehension; are you seeing the failures more in design decomposition or in basic syntax and control flow?
drearymoment@reddit
Does your company use an ATS to filter out or rank candidates?
One problem is that the candidates who are most likely to rely heavily on AI to generate code without a comprehensive understanding of that code in the first place are also more likely to rely heavily on AI to generate resumes and cover letters that are perfectly tailored to the job description and ATS.
Could be worth looking further down the list of candidates to see if anyone legit slipped through the cracks.
DizzyAmphibian309@reddit
I recently looked at some Google job ads lately, not sure if it was just the ones I looked at, but they were basically all the same and didn't actually list any required skills. I think they're catching on.
drearymoment@reddit
How does anyone apply to something like that then though? If all the roles are more or less the same and there aren't any required skills? You stunt the LLMs, sure. But at what cost?
Maktube@reddit
/u/DizzyAmphibian309 is probably on to something, and also keep in mind that Google doesn't hire for specific jobs. They give you and extremely generic series of very leetcode-y interviews plus a behavioral, and if you get an offer THEN the next step is "ok, we've decided you're going to work for us, now let's figure out where and on what", which you have a reasonable amount of say in. The offer is good for up to a year, too, if they don't have anything you both think it's a good fit right now. So I think they don't put skills on their job posts because the only skill they know ahead of time is "must be competent". Plus, even if they did want to hire for specific jobs they can't really put a lot of the traditional "most have experience with because they've built almost all their own tools from scratch so... no one they hire is going to have experience with them.
DizzyAmphibian309@reddit
I work in big tech, not Google, and we do this sometimes. We'll find that the person is worth hiring, but not for the role that they applied for, and will recommend a shorter loop for a different role. That's actually how I got my job there. I'm guessing that Google just skips that first step like you said. It probably works well for generic engineers, but it would make it more difficult for them to hire a specialist.
Maktube@reddit
Yeah, they do (or they did at the time, anyway) have different pipelines for SWE, devops engineer, and test engineer (I forget what that called them, but those were the categories). I think if they needed something more specialized than that, they'd typically be looking at someone who's enough of a domain expert than the normal hiring pipeline doesn't apply to them, and I suspect they mostly specifically recruit individuals instead of having a posting anyone can apply to.
DizzyAmphibian309@reddit
It makes sense if you've got AI analyzing thousands of applicants. They don't want a tailored resume, they just want your resume with your job history and skills. They'll figure out from your resume whether you're a good fit or not.
It's actually rather clever, but wouldn't work for anyone other than the top tech companies.
ikeif@reddit
I read a post on LinkedIn about a recruiter who tried to talk about how hard it was to find candidates - and it amounts to what aligns with your comment.
They took the top ten percent (of whatever their scoring was) to interview, and trashed the rest.
Rinse and repeat.
Without any verification that maybe the other 90% - possibly in that first round - would’ve been a good hire.
It’s so dumb.
oupablo@reddit
I also say the same thing about giving priority to FAANG hires. Seeing these names on the list seems to immediately bump the candidate up in the recruiters/managements eyes. Then you interview these people and 90% of them couldn't even tell you how the services they worked on got deployed.
ImportantSquirrel@reddit
Some dev teams have devops people who handle the entire deployment. So devs who've only worked on such teams might have little to no knowledge of how things are deployed but could still be really good devs.
Maktube@reddit
Well... sure, in isolation. But if you're NOT Google very often your devs need to know how devops things work because you can't afford a dedicated devops team/engineers.
This is actually one of the main reasons I haven't tried real hard to get a job at Google in particular. I have a friend who works there, and their entire tech stack is not only nearly perfectly abstracted away from everyday devs, it's also SO incredibly different than what anyone else is doing that I worry after a few years working there that working at another job would be like trying to switch from qwerty to dvorak.
max123246@reddit
If it looks too good to be true, it almost certainly is
new2bay@reddit
They may have better luck taking the middle 30% these days.
ThatShitAintPat@reddit
Was thinking that. Take the top 10% and throw that away.
drearymoment@reddit
Play stupid games, win stupid prizes!
brat1@reddit
I have this recruter in my linkedin who was like that and praised AI, then got laidoff. Hes been out of a job for 1 year now am I am laughing my ass of on how much his tune changed lmao.
Wide-Pop6050@reddit
One flag is actually if the resume it TOO catered to the job description
mysterious_whisperer@reddit
Are you saying that’s a flag the screeners look for or that’s a flag we should look for?
Wide-Pop6050@reddit
Screeners really
You shouldn’t put the JD and your resume into AI and ask it to customize the bullets because that looks really obvious
thy_bucket_for_thee@reddit
In this industry that is hardly a fair concern. The vast majority of jobs all use very similar technology, doing very similar things. It's not hard to find candidates that match everything you want.
VictoryMotel@reddit
That's exactly the problem though, finding people who have used the same programs other than good people. Learning new languages or specific cloud nonsense isn't hard for an experienced programmer.
thy_bucket_for_thee@reddit
No one disagrees with your comment, but it doesn't solve the original problem of hiring managers being so incompetent they don't know how to hire people effectively.
Maybe I'm jaded with middle age, but if you're hiring local candidates and hiring through local channels, it becomes quite easy to tap into a competent labor pool. But this requires actually being part of a community and wanting to help others, something most business owners don't care about so of course the incentives become misaligned when you have managers that hire for control instead.
Only now the managers can't control the pipeline of labor as they thought and now we're left with a situation where wildly unqualified people get screen over normal software devs more than capable of doing the job.
VictoryMotel@reddit
I don't know what in the world you're talking about.
drearymoment@reddit
Is the ATS smart enough to flag that?
I've noticed that myself when using Claude to generate a first-pass attempt at a resume or cover letter. It often copies phrases verbatim from the job posting and fits them in as best it can.
It looks very low effort to me, but maybe the ATS considers that a stellar match for the job.
Wide-Pop6050@reddit
I’m not sure about ATS but if you upload the resumes into Claude it will catch this
new2bay@reddit
They’re saying Claude is producing these resumes. Are you talking about companies uploading resumes to Claude?
Wide-Pop6050@reddit
I’m saying when we uploaded a chunk of resumes to Claude and gave it our JD for context it identified a bunch that it thought were AI generated or at least AI helped - and we would agree
northrupthebandgeek@reddit
Of course the advice recruiters peddle ends up being red flags.
Wide-Pop6050@reddit
If you do to a normal human amount it’s fine. Highlight the parts of your experience most relevant to the job. But it’s unlikely that your bullets will match the job description bullets line for line
Think_Inspector_4031@reddit
I'm trying to cheat the system by injecting all the ATS keywords into my resume as transparent tiny text. ATS should see it, recruiter does not.
Best part is that I use AI to create all the words and phrases.
Unfortunately still no interviews 😔
mirageofstars@reddit
Yep. We had to invert our hiring to find actual quality candidates.
SecretWorth5693@reddit
I had GPT regenerate my entire resume. It looked better, but sounded nothing like me. I went back to my original resume and had it provide me with ideas and wrote them in my own words.
new2bay@reddit
Can you give some examples? I don’t think my resume has ever sounded like me. It’s an artificial marketing piece, and I’m an engineer. I don’t do marketing.
VictoryMotel@reddit
Why wouldn't you write it yourself?
Dziadzios@reddit
Same. ChatGPT version overloaded my cringe-o-meter, it was obvious that r/LinkedInLunatics type posts were in the training data because it sounded like that.
caboosetp@reddit
Yeah, I do the same. I've been slowly updating the same resume for a decade, but in the last year I've done some bigger edits. I used gemini and copilot to check what was missing for AST. I added stuff in and trimmed some of the older sections. I never let it write the actual content.
In the past I'd do the same thing with recruiters. I think this is one of the big things people aren't taught now. Recruiters benefit from you having a good resume because you getting a job is their job.
Find a good recruiter, and they'll go over your resume with you. Some won't because they're focused on the churn or don't have time (they're also inundated with thousands of resumes). But it only takes 5 seconds to ask. They're the ones who will know what to look for because they're the ones looking.
hw999@reddit
I no longer deal with ATS systems. The only waybto search for jobs now are via your network or via recruiter. The company pays 25% more when im hoted via recruiter, but its the only way to talk to a human. Using a recruiter is essentially like spammying the 0 key until they patch you to an operator.
dExcellentb@reddit (OP)
I'm not involved with the "frontend" of the hiring process. I only test technical skills. Though, I don't think it's changed much.
Mizarman@reddit
Favor enthusiasm and inherent talent, and coach them up.
morswinb@reddit
If you interviewed me some 15 years ago I would do great, but probably because in my free time I would write some games, simple tetris, snake, asteroid etc.
The University assignments were no different, sudoku solver, labirynt 2D shooter etc.
Heck office workers were familiar with minesweeper.
Its gone now. People don't work with 2D arrays. Your n by n grid problem is dark magic.
On the other hand AWS, load balancers, auto scaling, kafka, cloud, vapeing, etc. Thats all thought by sponsored university courses. Students get Amazon infomercials instead of education. You don't want next gen competitors, you want your cloud addicted developers.
TheRealChizz@reddit
It’s one thing to not be able to implement this by hand, but do the majority of CS grads seriously not understand this concept?
Cahnis@reddit
i dont know what you guys are talking about, most job opennings do leetcode style interviews.
symbiatch@reddit
Depends highly of location. I’ve never seen a single one of those and neither has anyone I know.
So not true.
saphyrre@reddit
Not looking to argue with you but go to the leetcode website and you can literally see *thousands* of people posting their interview questions at *hundreds* of different companies, and they're.. letcode problems. I'm honestly surprised you haven't seen any if you've been doing any interviews the past few years.
morswinb@reddit
Most of the leetcode problems dont require 2D arrays. And those that operate on a grid are usually classified as hard, while companies pick easy/mediums.
I have seen discussion on Reddit that reffered to game of life as "harsh.
I am not debating that leetcode is popular, but there that problems solved there are actually very narrow in scope.
FCBStar-of-the-South@reddit
Tic-tac-toe on n by n gird is at most CS2-level assignment at most schools… If interviewees are failing that it would be because they used LLMs to pass their CS2
Ok_Chemist_3576@reddit
And that's exactly the problem. That's a thing fresh for new grads. Not experienced developers. In fact, that's not a real-world problem for 99% of the businesses in the world.
Now, think about why he didn't hired the "new grads performed better than experienced people" in the interview, even tho [allegedly] their performance are the best.
midwestcsstudent@reddit
Whether or not it’s a real world problem is almost irrelevant to the signal you’re looking for in a technical interview.
Anyone can handle the day-to-day “place this button there” or “add a PDF export button”, real-world request.
But can they decompose a large problem into smaller solvable chunks? Can they reason about many moving parts and consider tradeoffs? Can choose the right tools for the job? If they’re green, do they know the fundamental building blocks of CS, which everything else is built upon?
new2bay@reddit
Sure, but “can they do that under the pressure of a technical interview” is a completely different question. There is recent research that shows people do twice as well on interview problems when they’re not being watched.
thekwoka@reddit
Bruh, if you they fall apart that much under the slightest pressure, why would I want to work with them?
new2bay@reddit
Bruh, it’s proven by research that people do twice as well on the same, exact questions, when the only thing that changes is the interviewer’s physical presence watching them. I’m happy to link you to it, if you don’t believe me.
thekwoka@reddit
Show me the report that says that that is the case with every single person.
Not some on average thing.
FCBStar-of-the-South@reddit
What you say doesn’t explain OP’s reported data
Ok_Chemist_3576@reddit
Or u/OP is contradicting himself
This or that. It is obvious there's something wrong in his hiring process and mindset.
MistahFinch@reddit
OP isn't contradicting himself. There's a full paragraph of context for the like you're quoting
VictoryMotel@reddit
It's table stakes like fizz buzz, it's something anyone should at least be able to make progress on to show how they can think through a problem. It seems like a pretty good technique to me. If someone can't figure out a single aspect, they will probably sit on their hands during the job too.
vilkazz@reddit
Depends a lot on the candidate.
I’d, unironically, fail at initializing a fixed length array required for the game because I haven’t had a need to use one for years now …
TitanTowel@reddit
That's hyperbole right? You've forgotten int[x][y]??
vilkazz@reddit
Half half. I’d get the correct bracket order right in my language of choice in a couple of tries, but in an interview setting that would already be bad.
And also, not that I’d ever go to a coding test without brushing up on stuff like this :)
TitanTowel@reddit
yeah i mean the brackets are y then x but still
darthsata@reddit
It doesn't actually matter here, as long as the candidate is consistent. There is no preferred orientation that benefits from memory adjacency.
thekwoka@reddit
....how?
Like...how would you not just have that knowledge even if you don't use it regularly?
vilkazz@reddit
You … forget what you don’t use? Especially with all the info creep.
2 to 3 backend languages, shell, aws, kubernetes, terraform, local infra tools, bits of frontend stuff.
I don’t hav enough cache to keep unused stuff hot
Free-Huckleberry-965@reddit
LLMs didn't exist when I took CS2, and I haven't had to think about that style of problem in more than a decade
throwaway1847384728@reddit
I’m honestly not following. This would be easy to reason about from first principles within like 30 seconds. I work on enterprise crud web apps and feel that way.
Free-Huckleberry-965@reddit
Sure, but that wasn't the ask. The ask was to implement it, and immediately start with feature requests. Under the pressure of an interview setting.
I've mostly been devops/sre for a long while now, but even when I was product-side it was never a "here's an empty file in an IDE.. go.. and make no mistakes because your mortgage depends on it". Even when I was writing code implementation (vs deployment, ci, ifra, monitoring, etc config), it was either lift-and-shift, find-and-fix, or minor enhancements. I can't recall a single time I started from absolute zero.
thekwoka@reddit
Yeah, so you start with reasoning it from first principles...
thekwoka@reddit
Anyone with logical reasoning skills can reason out tik tac toe without being taught it.
Distinct_Bad_6276@reddit
Schools should teach students theory 99% of them will only ever use once in a blue moon instead of tools they’ll use daily, got it
sweetno@reddit
AWS doesn't require higher education though, it's vocational training.
new2bay@reddit
I think that was the point they were making.
Void-kun@reddit
You can do this in just a terminal it's not difficult. Anybody that can do hello world and a loop could figure this out.
f1VisaMan@reddit
I would excel at the vapeing portion
Deathspiral222@reddit
I'm a principal engineer with 20YOE. I haven't manually written code in a year and I'd struggle with this, simply because I don't memorize the syntax and libraries of the 12+ languages I've written in and unless I'd been manually writing code in a language this past year, I probably wouldn't be able to do this.
That said, I *would* be able to explain how I'd lay out the data structures and the algorithms I'd use and be able to talk about their complexity and edge cases and all the other stuff. I could write pseudocode and could decompose the work into appropriate methods (or objects if we want OOP) and talk about recursion and all the rest if it, but manual writing of code? No one does that any more outside of specialized areas like flight control systems and embedded systems and a few others.
apartment-seeker@reddit
huh
RandomPantsAppear@reddit
This is such a thing. I’m of a similar experience level. Recalling specific syntax while surveilled but a team of senior engineers was never my forte, and AI autocomplete has weakened that.
I know what to do, I just don’t immediately recall how the sort argument to handle sorting list of deeply nested dicts by value from memory 🤷♂️ Because that’s not required.
My structure is solid, my design principles solid, my code easy to read and understand. But at this point super specific recall I don’t need to do in the real world feels a little “gotcha”.
power78@reddit
I've noticed the same thing. Without AI the candidates have no idea what's going on. It's scary.
jarjoura@reddit
With all of these layoffs, I’ve seen the opposite. There is a flood of high quality candidates hungry and willing to put in the effort to interview well.
You might want to check your sourcing and check where your candidates are coming from. Might want to also update your screening process to avoid wasting other engineers time.
Icanteven______@reddit
Yeah I’m torn. What you’re experiencing is totally real…but I also look at how I code today, and I exclusively use AI. I literally don’t hand craft code at all.
My knowledge of how to write code architecturally though is extremely useful, as I guide the AI with a plan to develop the correct thing in the correct way according to architectural principles that I know will make it easier to modify in the future. How do you obtain such skills without having spent years writing code?? I don’t know.
However I think having someone write code in an interview for a job where they aren’t going to write code feels weird to me. I don’t know what the right answer is anymore…but I I think it’s probably something more like “pretending” we have an AI writing the code for us, and then you throwing curveballs about what it built or saying “poof it does that”, and then seeing how they evolve the system according to business demands and changes.
Do they guide the system to be modular, does it obey open closed principle, do they just treat it like a big black box?
We use different higher level skills now to write code, and we should test for those somehow and stop expecting them to be able to reverse a linked list by hand (so to speak), which they will never ever have to do.
Beli_Mawrr@reddit
You'd better relearn how to code normally because I wouldnt want to hire someone who doesnt know how to code and I definitely wouldnt want to code review them or merge their code in.
My personal experience with these bulk code Gen tools is that they really do a bad job. The code may run and sometimes adds whatever you asked but its unmaintainable slop that needs to be rewritten almost immediately. It doesn't fit in with the rest of the codebase. Etc. Fine for small projects but useless for the codebase.
Icanteven______@reddit
I didn’t say I didn’t know how to code. I know how to code extremely well. I just don’t do it by hand. And I review and cleanup all ai generated code
dExcellentb@reddit (OP)
I'm considering just doing an AI-assisted interview where I give candidates a business problem, plus a ton of features to implement, expecting them to only implement a small fraction in an interview.
The goal is for the candidate to pick a limited basket that best solves the problem given the time constraint, and group them into a system in a way that's performant and understandable. They can use AI to build these out so the exercise is still in decomposing problems and producing maintainable code.
If the question ever gets leaked, just swap out the business problem.
Master_Addendum3759@reddit
Sounds better than tictactoe
matthkamis@reddit
I mean come on, writing a tic tac toe game should not be hard
gjionergqwebrlkbjg@reddit
Yeah if you can't code that the problem became way before AI became a thing.
JandersOf86@reddit
I appreciate your honesty regarding your uncertainty. Im learning programming right now in hopes of getting a job as an SWE in the near future and, if Im being honest, I don't enjoy the idea of allowing AI to code for me. Maybe it'd just the old man in me.
Im 40, spent the majority of my career in software QA and been doing electrician work for the last 2+ years, due to getting laid off from Blizzard in 2024 after the Micro$oft acquisition, and due to how difficult its been for me to find stable work in the tech industry w/ medical benefits for my family and I since then.
During all that time in QA, I was heavily involved with different engineering departments that had no AI in use, but a solid team of knowledgeable engineers that built amazing things from the ground up. I think I just still view SWE as that classic style, and part of me thinks I may not ever get to do SWE work because of being intimidated by utilizing AI so much, and honestly feeling weird about using it. Maybe I need to work on not being so set in my ways perhaps.
Would you still consider a junior candidate who hadnt utilized AI much but still coded by hand, still wanted to be able to show they could reverse a linked list? Is that even desirable anymore?
dbxp@reddit
I have heard other people doing code review tests rather than programming tests to some success. The only downside is they don't scale as well as a simple leetcode
opinionsOnPears@reddit
I haven't seen anything that would indicate AI is causing issues other than people defiantly against it.
What I have seen is a lot of developers have a singular focus, with a specific stack and/or framework and that's all they can do and want to do.
hello2u3@reddit
Not my precious arbitrary question with arbitrary constraints
javatextbook@reddit
The solution to this is 100 percent in-person hiring
studmoobs@reddit
you're 3 years behind on the state of ai
also it's your automated filtering system
zaemis@reddit
If you want insight into a candidate, you have to ask insightful questions. It doesn't sound like you're doing that. Rather than implement tic-tac-toe, just have a discussion. Ask "if you were to implement online chess, what would architecture be?" See what they think about back end vs front end. Ensure they understand the tradeoffs. Ask about where validation would be? What type of format would represent the game? If they're doing everything front end, that's a problem. If they're doing everything backend, that's a problem. Probe them to make sure the understand tradeoffs and consequences. Not a single line of code needs to be written - people use AI for code professionally these days that's where the industry is headed/went. But can they think about systems and consequences?
Better yet, give them a project and have them code the solution with AI, and ask they provide you the transcript. You're looking if they can use the tool effectively, guide the system. How clear are they making their requests? how do they iterate? How much control do they maintain and what do they relinquish? They can't finish everything in the task in the timeline, even with AI, but what did they focus on and what did they set aside? How did they prioritize? How did they address bug fixing and code quality?
We've always had mediocre candidates and we've had stellar candidates. Your job is to filter them. And your current approach is not effective.
bluetrust@reddit
I'd like to challenge why you think implementing tic-tac-toe this would be be simple for a senior developer. Is it because it's a child's game so it sounds simple? The surface area to cover includes board game modeling, the game loop, ai play or two-player play, win/lose/draw detection, scene management, multi-turn input parsing/validation, and more.
This would be a lot for a game developer. If it's not for a game development job, I don't know what you're doing.
hippydipster@reddit
Could be without a GUI, just a simple command line program. I would not start coding up a gui in an interview.
bluetrust@reddit
I didn't mention a GUI.
Let me try a different tact. Implementing a playable tic-tac-toe game with a cli would still be a 100-200 lines of code task. Is that reasonable to ask for in an interview? I don't think it is.
ElvishParsley123@reddit
As an experiment, I wrote this up. It took about 20 minutes to implement in 110 lines of code in a plain text editor with no auto-complete. Half an hour seems perfectly reasonable if the programmer knows how to program.
No_Shine1476@reddit
Post the code
ElvishParsley123@reddit
akc250@reddit
You're probably not a new grad, you did this in a low-stakes environment without the pressure of people judging you to decide if you can afford to eat, and seeing how far down you are in this reddit thread, you had some time for the question to sit in your mind, read other's posts, rather than being sprung at you and told to immediately implement it. So, not even a remotely close scenario.
Strict-Protection434@reddit
playable 3x3 TTT board with simple logic in 45 minutes to an hour would seem somewhat reasonable for me. I don't see the need to crush interviews down to 30 minutes and rush developers.
but definitely that wouldn't be good enough for these style of interviewers, and I'm assuming if you don't have an N of N board with all condition checking running by the end of the interview, you're not going forward.
I sit with most people where I just don't do puzzles because most interviewers (not all to be fair) just use them as a pass/fail condition.
Icy_Cartographer5466@reddit
Isn’t just a 2d array, a while loop that checks for win/draw conditions, and reading from stdin? Doesn’t seem that hard
RandomPantsAppear@reddit
I had a similar issue in an interview - minesweeper.
After decades of developing API, scrapers, data pipelines, etc I found myself trying to remember a design pattern I used when I was 14 years old. I am fucking 38.
lphomiej@reddit
I've been hiring several people over the last year and the candidate pool has been absolutely outstanding, especially for fully remote jobs where you just have 100x more options (we switched to hybrid halfway through this hiring spree). I had a really hard time going through so many resumes and collaborating with recruiters about their screening calls -- it was just analysis paralysis. But everyone I interviewed was super capable, had their own skills and specialties, were capable of having human conversations, and everyone was absolutely capable of doing the job. This was for senior-level .NET full-stack devs.
Adventurous_Wolf6535@reddit
Didn’t know people still make money with 2D games. Btw u sound a bit like god complex.
Ok-River6335@reddit
If you're conducting interviews as some performance test you're doing a terrible job.
Software development isn't some perform on command task. There's a massive difference between working on a team/project and hey do this random task right now and tell me what you're thinking.
So many companies do number 2 and then wonder why it doesn't correlate with number 1.
funny_lyfe@reddit
Is this supposed to be hard?
For an n sized board you need n in a row. Now the shapes are 3 ways. Across, up/down, side to side.
Okay now we need a win function. There are 3 states for each element blank, x, o.
The UI is a click handler that changes between o and x. Each click alternates.
Okay now win function implementation- Each move does a win function if the row or column or across is full. So now we need a full function. The issue is now an algorithm o(n) problem instead of o(n)^n which would happen if you kept scanning every time.
This is just in 2 minutes without thinking much on my phone. What kind of engineers are we producing?
ProbablyNotPoisonous@reddit
You forgot to check for diagonal wins :P
SnugglyCoderGuy@reddit
MOAR AI OBVIOUSLY!
jimmytoan@reddit
AI tools have lowered the cost of producing code that passes screening, which means the signal that "wrote a working solution" used to provide is gone. The candidates who stand out now are the ones who can explain the tradeoffs in their solution, catch edge cases the problem setter didn't anticipate, or articulate why they chose one approach over another.
The practical fix for interviews is shifting the signal from "did you get to a working solution" to "can you reason about a solution you're shown." Give them working code with a deliberate design flaw and ask what they'd change and why - that's a lot harder to AI-assist than coding the problem from scratch.
dbxp@reddit
Have a look at your hiring adverts. HR and recruiters have a habit of adding things which scare off applicants.
caboosetp@reddit
HR asks for a wish list. Engineers describe a unicorn with the caveat that they only need certain things and the rest "would be nice".
But the "would be nice" gets lost in translation and suddenly they're actually looking for the fullest of stack developers. The recruiters then only find people that learned how to pad their resume for AST or architects who actually do know it all but aren't going to take your pay rate or a mid level role.
A bunch of good people that fit what the engineers actually wanted don't apply or get filtered out by AST because they don't have 6 years in Java, Python, AND C#.
dbxp@reddit
I've seen recruiters just add things themselves like AWS and Azure experience even though all our kit is hosted internally
Void-kun@reddit
They treat adverts like a website that needs SEO so they add what they think are buzz words that are highly searched for to appear to more users.
Can see how that would backfire and get the wrong type of applicants
ExitingTheDonut@reddit
Bingo
They want higher visibility at the cost of accuracy
new2bay@reddit
But then they turn around and reject people who don’t match everything, and make us guess what the real requirements are.
Void-kun@reddit
Most recruiters aren't good at their job. They're just glorified salesman that got into the industry during the tech boom.
I ignore most recruiters simply because they're inefficient and waste time.
If you aren't getting the candidates you want, get rid of the recruiter. When you find a good one stick with them.
lurco_purgo@reddit
I was recently approached by a recruiter asking about a fullstack role. I've send her my resule with a pretty detailed list of tech, which includes a lot of frontend stuff (for example: React, TS, Vite, Vitest).
At which she asks me about Babel and Webpack experience... I say I did used them of course, but it's not exactly something you spend every day in works mastering. And she says great, can you add it to your resume please?.
I mean I did it, but it's fucking stupid. No developer is going to ask about my Babel proficiency, unless I'd be joining React's core team or something (even then I doubt it would be a concern).
Ok_Chemist_3576@reddit
HR have turned this into a buzzword-matching game.
Ok_Chemist_3576@reddit
100% agree HR is the problem here.
qtechno@reddit
Never met a recruiter with something passing for a brain.
new2bay@reddit
Always has been.
Bright-Appearance316@reddit
i noticed this too. candidates seem less prepared for problem-solving, not just coding.
Professional_Log728@reddit
are you noticing any common gaps in their reasoning or problem-solving approaches?
CalmLake999@reddit
Sorry but leetcode questons are really dumb way to interview people. You give them real world challenges and ask them to design and create it.
VisiblePlatform6704@reddit
Remember how around 2020-2023 there was a "draught" of decent developers?
There still is. The number of people who know how to code is still the same.
The problem is that we got inundated in noise, by people using AI.
It's kind of like the bootcamp effect, but stealthy. At least with bootcamps you knew what you were buying immediately after reading the CV.
BothWaysItGoes@reddit
Your funnel sucks. With recent layoffs there are more qualified candidates on the market than ever.
erythro@reddit
This isn't x. It's actually y. 🤔
wind_dude@reddit
I mean if new grads previously outperformed experienced devs on your test… was your test that good to begin with?
Ok_Chemist_3576@reddit
Ah this. u/OP This tells you all you need to know about the hiring method. Is HR/recruiting in charge instead of a technical knowleadgeble person? Here you have your answer.
The-Fox-Says@reddit
Like tic tac toe? I would laugh at an interviewer who asked me to do something like this for anything more than an entry level role
belgian-newspaper@reddit
why?
The-Fox-Says@reddit
I mean we’re on /r/experiencedevs so I would be aiming for senior/lead/staff positions that would hopefully focus more on job duty related questions.
Ask me about how I tackles certain problems that resonate with your team’s own problems. Ask me system design questions on how I would lead a team to build a solution. Ask me how we would harden our pipelines and create new highly scalable pipelines with high fault tolerance
Izacus@reddit
Before I ask you the questions you'll bullshit me about, I need to know whether you understand what a for loop does. A lot of people with your credentials and ego tend to fail at actually coding when they come for a job or an interview.
Your questions come after we know you actually ran an IDE in last 20 years.
No_Scallion174@reddit
I actually had a programming exercise I quite liked for a role, we created a bad script (in a language relevant to the role) with a bunch of problems ranging from trivial to catastrophic and just asked applicants to read it and talk about what they found. The number of “oh this looks fine” after 5 minutes of reading was staggering.
Izacus@reddit
Yeah, at previous companies we even put a basic task on the application form to make filtering easier. Doesn't work in the AI era anymore, but man the type of people you get...
new2bay@reddit
Ok, cool. Here’s my emacs configuration. We good?
Izacus@reddit
Nope, but you're welcome to work somewhere else. We got plenty of good candidates :)
No_Scallion174@reddit
The problem is I’ve seen bad engineers manage to get themselves promoted to senior because they were good at bullshitting. If I’m interviewing a senior I’m still gonna want a simple “can they write code” check. The balance is finding a real problem that doesn’t rely on having memorized some trick like half of the leetcode bullshit.
Ok_Chemist_3576@reddit
That's a problem of office politics. No matter what you do to find the perfect technical employee. It's the Mr Buzzword bullshitter who will climb the corporate ladder. From Development, from accounting or whatever.
You might as well bet that there's a non-Tech MBA as a leader of the technical team in that place.
zdubbzzz@reddit
I genuinely don't understand why people are asking staff level engineers leetcode questions. They should be giving them answers to bullshit leetcode questions and asking them to explain why it's important as if they were explaining to stakeholders. THAT will tell you a lot about how someone will perform in that role
The-Fox-Says@reddit
Yeah idk people are so hungup on leetcode I’ve seen people just memorize leetcode and be dogshit engineers after. Answering some coding question in an interview also doesn’t mean you’ll be a great engineer or teammate either
Ninjez07@reddit
"Hello Mr Senior Team Lead Man, please invert a binary tree. Wait, where are you going?! Why is it so hard to find competent engineers, the only people capable of passing our tests are grad school kids."
That said, a reason why even seniors might face a basic technical challenge is that there are a lot of "senior engineers" out there who are good at office politics but actually awful at anything technical, so I can fully understand companies wanting to screen for people able to are least put a program together themselves in a language and tooling of their own choice. If you can't "hello world", why should they trust your opinion on hardening a scalable pipeline?
thekwoka@reddit
I mean it makes sense for a "is this dude a total imposter?" filtering.
Like I think any advanced dev would still reason through it decently. A New grad that paid attention would probably be faster at it, but an experienced dev could get through it.
Only imposters would have real issues with it.
thekwoka@reddit
If it was an interview specifically for entry level stuff...
You can't really ask them the same kinds of things.
arstarsta@reddit
Experianced don't mean good new grad can absolutely be better.
thutch@reddit
For a good test this result could be explained by collider bias on salary range. (The salary is appealing to good juniors and OK mid career engineers)
dExcellentb@reddit (OP)
If an alleged senior dev struggled with tictactoe, then the test just caught a serious red flag.
bstaruk@reddit
Looks like you got your answer then, huh? You're not a skilled interviewer or manager, so you think everyone sucks.
ncmentis@reddit
If I got paid six figures to write fizz buzz and tic tac toe that would be awesome, but alas.
Windyvale@reddit
Underrated comment.
bdanmo@reddit
Ah yes my daily tic tac toe implementations.
Some-btc-name@reddit
LLD tic tac toe?
Tiktoktoker@reddit
💯
KAID3N@reddit
I don't understand the people claiming this is a bad interview question. I'm a juniorish dev for reference and I'd much prefer this to actual leet code. If you can't even attempt to wrap your head around this problem, I can't help but feel like you're lacking fundamentals. I get what people are saying that the industry has changed and there are more relevant questions to ask, but I feel like a question like this demonstrates your ability to think through problems rather than remembering cloud concepts. Whether cloud concepts are the big thing right now or not, programming fundamentally requires you to have this reasoning ability, so how exactly is it not a good interview question?
necheffa@reddit
The base version of the question is so trivial that it is maybe a notch or two above fizzbuzz in complexity - basically any idiot can do it.
To really flesh this problem out and make it an interesting demonstration of your software engineering chops, you are going to need more than 45 minutes to develop your solution, especially if OP doesn't lead with "oh by the way, I'm going to want support for an arbitrary square board size and an arbitrary number of concurrent games".
These types of interviews tend to crucify the interviewee for not anticipating some scope creep mid interview. Which, in a round about way, really does reflect reality I guess.
KAID3N@reddit
Yeah I see where you're coming from. The way I seeing it, taking that into perspective, is it really comes down to how the interviewee is evaluated based on this challenge. If it's just to confirm that you can demonstrate your ability to think through an issue, I think that's valid, but if it's graded the way you're describing it might be, I think I agree that it may not be a good question. I wasn't evaluating it as, can you build this full-fledged game in an hour, rather, planning through the architecture of it.
adav123123@reddit
Hmm, maybe not the best use of your interview time! For new grads the main thing I look for is curiosity and learning mentality and ability to take leadership/ownership. Hell, they don’t even need to have a CS degree. I know that because I don’t have a degree in CS. I would have failed the fuck out of this question as a new grads. So the thing you should ask yourself is, is this interview question relevant?
zubinajmera@reddit
the drop in manual coding ability is real but I'd push back on the diagnosis. Banning AI to test reasoning is like banning calculators to test financial modeling ability , you're measuring the constraint, not the skill.
The candidates who can't build tic-tac-toe without AI aren't necessarily bad engineers. Some of them have genuinely restructured how they think around AI tooling and are exceptional at it. Your current filter can't distinguish them from the ones who are just dependent and hollow.
what's actually worked for us: give candidates a broken system in a live environment, AI fully allowed, and just watch what and how they do.
Problem decomposition, maintainability, reasoning through complexity — all of it shows up, just through different behavior. The ones who understand what they're building catch AI mistakes and fix them. The ones who don't paste and pray and the system breaks.
You get strong signals you're looking for without filtering out the engineers who've legitimately moved on from hand-coding.
what do you think u/dExcellentb ?
No_Imagination_4907@reddit
It has been bad for the past few years, at least from my perspective. Last week I asked a candidate to implement some business logic that resembled what we're running in prod, but simplified, and the candidate struggled to the point that he didn't even know how to start. He kept asking questions like "what is the end-point? What are the parameters of the end-point?". This tells me the candidate is so wired to REST that taking it out of context renders him completely confused. And he's not the first one I found having this syndromme.
Dijerati@reddit
I’d be willing to bet that your interview standards and difficulty are much higher now than 5 years ago
dExcellentb@reddit (OP)
Not really. I'm still asking the same questions.
HowTheStoryEnds@reddit
It's not you. I'm getting people that fail fizzbuzz and more than before.
It's quite obvious that they have way more trouble than before with reading and analyzing code.
Caramellatteistasty@reddit
Jesus really? I am feeling less bad about my qualifications now.
Phantom1165@reddit
Same lol
Legitimate-Trip8422@reddit
Unfortunately you aren’t the only firm interviewing candidates. Candidates have to prepare for n types of interviews while having a full time job. It’s just throwing a dart on the board at this point with the amount of permutations in skills the firms want.
400Volts@reddit
If your company uses ATS then the applicants using AI are crowding out the qualified ones
sleepyguy007@reddit
funny we used to ask this exact same question to candidates at a startup i worked for 5-6 years ago. and even then probably 70% failed our phone screen. so people have always been not the best.
hallcyon11@reddit
If it rains on your birthday, it’s ChatGPT‘s fault
Beneficial_Map6129@reddit
I have 10 YOE, senior eng at big tech
I spent the last year churning out an entire new project every single quarter for the last year. Had to vibe code with AI for basically the entire time, just skimming the change summaries that Claude outputs
I don't think i can code anymore to be honest, i feel like I'm more of a Product Manager/Designer than a real engineer now
akc250@reddit
Same. It’s silly how hiring managers can’t evolve past leetcode hazing rituals and hands-on “build me a game” type of questions. Yet I can do the job in big tech perfectly fine, get exceeds expectations during annual reviews, and can debug complex bugs that occur in distributed systems. But because I rely on AI to invert a binary tree for me in a theoretical problem that I would never encounter in real life, I can’t pass the interview.
tcpWalker@reddit
yeah the great jobs you know you can deliver on (because you have) where you don't get them because of flawed interviews are super annoying.
psyyduck@reddit
How many LOC per project?
It’s not a very good measure of complexity, but I do notice my max project size (with LLMs) is about 15k LOC.
nates1984@reddit
I hope you never need to crawl into the machine yourself.
MatthewMob@reddit
There is no drop in quality. Your hiring process is just ineffective.
shill_420@reddit
hire me, im great
jonmitz@reddit
stop asking them questions not related to the job anymore? the field has changed
bandersnatchh@reddit
Isn’t it sort of a design discussion though?
How are you going to implement the board? How are you going to implement it for flexibility?
How do you implement the game rules?
I feel like it’s a design question more than a coding puzzle.
But maybe I misunderstood it.
SarmackaOpowiesc@reddit
What an awful terrible take.
If you can't code by hand, I don't want you. How else are people going to figure out what the dumb ass chat bots spit out is right?
jonmitz@reddit
do you need to be able to leetcode to use ai and review it? you dont.
instead you should be giving interviewers prompts with issues and see how they use the llms and correct the answers.
sorry the industry is changing. i hate it too. but im not gunna pretend like things are changing
good luck
SarmackaOpowiesc@reddit
It's not.
The only thing that's changing is younger "engineers" don't actually understand what is going on. There still are great candidates out there, they just take time to find.
No one I ever interview will use an LLM during my time. I insist on face to face interviews now.
dontquestionmyaction@reddit
Did you read a different comment?
Leetcode has always been nonsense brain-teasers. Give people practical problems and nobody is gonna complain. You'll get a far better idea of how they code too.
TechySpecky@reddit
this isn't leetcode?
new2bay@reddit
It’s a Leetcode-style problem. It’s completely divorced from the real world, except for the constraint that it’s supposed to fit entirely within a 45 minute interview, while under pressure.
jonmitz@reddit
i mean it basically is. this is kinda semantics. you never do leetcode at work, and you certainly never code tic tac toe on a cli. so why interview for it…….
MistahFinch@reddit
This isn't leetcode.
You don't use arrays or matrixes at work?
jonmitz@reddit
i think what im getting at is using actual code in an interview in a variety of fashions is always a better guide than asking some toy problem regardless of what it is
the best interviews are when they have sample code and ask you to fix a bug or make a feature
devfuckedup@reddit
haha good problem this is one of the first things I did when leanring to write code. If I am honest I am not sure I could do it now especially after the last 4 years. time are chainging and its getting fucking wild out there.
iegdev@reddit
The best interview test I ever had was being asked to implement CRUD endpoints for GET, POST, and DELETE using in-memory storage, then show that it worked and explain my code.
Why was that a good test? Because it was directly related to the work on my resume and the kind of work I’d actually be doing day to day. It wasn’t some arbitrary puzzle or random concept the hiring team decided every developer should know on command.
We all have different strengths and weaknesses, and we all fall into the trap of thinking, “This is easy for me, so it should be easy for everyone.” That mindset shows up a lot in technical interviews.
A lot of hiring practices also ignore the fact that many people simply aren’t good at interviewing. That doesn’t mean they aren’t good at the job. Some people are great interviewers. Some people enjoy that environment. Others don’t perform well under that kind of pressure, even if they’re perfectly capable once they’re doing the actual work.
I was friends with someone who interviewed me for my first job out of college. He was young and thought he knew what a strong candidate looked like. He later told me he had actually voted against hiring me but was overruled. After working with me for a couple of years, though, he came to understand that just because someone doesn’t perform the way you expect in an interview doesn’t mean they aren’t a good fit for the role.
I firmly believe our interview process is broken. Too often, it’s shaped by people who mistake interview performance for job performance and don’t really understand what makes someone a strong candidate.
psyyduck@reddit
As a general rule, the interview should resemble the actual day-to-day work as closely as possible. Like anywhere from "here's a tricky problem we ran into last week", to a 3-month probation period. If you want someone who can build chairs, test them on building a chair. Don’t test them on “who invented the pentalobe screw” or “can you juggle with dowel rods”.
Beli_Mawrr@reddit
I have a code review problem i like to give. You instantly find out who is a coder and who is an engineer (hiring the coder and training them into an engineer is possible!)
new2bay@reddit
I had an interview like that. I got to a certain point that, after pointing out so many obvious problems, I told the interviewer I would have to just block this merge and have a conversation with the author about it.
Beli_Mawrr@reddit
lmao. I mean, fair enough.
Ok_Chemist_3576@reddit
These problems are created by companies and recruiters. They put in place ridiculous practices for the candidates, which in turn drop the quality and effort made by candidates.
Even if the candidate is good, the effort will be low, because recruiters turned a quality meritocratic game into a numbers low-effort game. Last thing I remember is ATS and the disrespect of AI interviews.
akc250@reddit
Exactly. You start getting candidates who game the system. They're really good at interviewing, solving leetcode questions or writing a function without AI, but when it comes to debugging complex issues in the real world, collaborating in a team setting, or working on independent assignments when the cameras are off, they struggle.
throwaway_0x90@reddit
It sounds like your coming to this conclusion all by yourself. Does management and your teammates agree with your interview style and the standards you're setting for the candidates?
Having a candidate write tictactoe within what I assume is no more than an hour does not sound reasonable to me unless the job interview is for videogame developer. What it takes to write a game is different from writing the next social media app.
throwaway_0x90@reddit
Asking to write tictactoe in an interview is unreasonable
unconceivables@reddit
I've seen a steady decline over the 20+ years I have been interviewing and hiring people. It's not even just AI, this happened way before AI. However, it's much worse now because of AI, and it affects every level.
Just look at the responses in this thread with all the people saying they haven't used a 2D array in all their years of working and so they shouldn't be expected to ever have to even deal with such silly questions, when any decent developer should be able to do simple generic problems like that even if they haven't done that specific thing in 20 years. Or the people saying they don't even write code anymore, so why should they even be expected to know code instead of just shooting the shit with the interviewer about their architecture adventures.
There are still good candidates out there, but you have to go through so many shit candidates like the ones mentioned above before you get to the good ones that it's so much more exhausting to interview than it's ever been.
AdMediocre524@reddit
The problem in short: some online problems and puzzles can be gamed, but it is necessary to filter before interviewing in person.
I think tools like Kattis can still work when the problems are hard.
honestduane@reddit
It's not the candidates, it's the recruiters.
Recruiters are not allowing the best candidate to apply to even get interviews.
People get told they're overqualified and then they send the one that they can make the most sales Commission off of.
Your not getting sent the best canidates.
humanguise@reddit
The interview for my first role was similar, I had to code the game battleship in around 50 minutes. Just setting up the game, no AI.
Shep_Alderson@reddit
Personally, I'd stop with the puzzle nonsense. If you actually want to understand if someone can read and logic through code, there's a very simple way to test this. On your application page, include a simple function, checking for if a number is greater than another number, then summing the total of the numbers that get through the function, for example.
But, and this is the key, include an = in your operator, but use CSS to make it practically invisible. (smooshed to be 1 pixel, white text override on a white background). Ask the candidate to tell you what value comes out of the function. If they paste it into an LLM, it will expand to include the "hidden" =, and most folks won't catch that and it will give the wrong answer. If they give the right answer, then it means they took the time to read the small amount of code and process it in their head. Simple but very effective.
If you asked me to come interview and handed me a puzzle question, like "make tictactoe, but with extra bits", I wouldn't take you seriously, especially because if I was asked to do something so trivial in my real job, I would absolutely throw it at an LLM and it would be done in 30 seconds. That's not where the value in an engineer lies.
If you actually want to test someone's ability to process and understand code, test for that, not logic puzzles.
Instead, you should probably expand your programming challenge. Derive a real example from work you've actually had to solve, replicate it in another codebase, and then ask them to talk through how they would solve it. Don't just let them use an LLM, encourage them to do so and then dig into _how_ they use the LLM. There's a decent chance you might even learn something that you could bring into your company that would actually make LLMs functional for your massive codebase. If the codebases you're interacting with are so complex and you can't break down the problem you need solved in a way that could be fed into an LLM and not have it hallucinate to hell and back, that's honestly probably not the codebase's fault. And frankly, if you think an newly onboarded engineer is going to be quickly effective in a codebase that's complex (and I'm guessing quite lacking in documentation) and the engineers can't use AI to work with it, then I further doubt the issue is with the LLM. Humans can only really load a limited amount of context into their heads anyways, and if you can't break down problems well enough for an LLM to manage it, how do you expect someone new, especially someone junior, to get it?
Being blunt here, but there's a massive gap between "I pointed claude code at the codebase and it shit the bed" and "I've got a custom opencode install setup with customized skills, a comprehensive plan process, and verification that the code is correct". I suspect that, with the right setup, an LLM in a decently setup agentic harness would be able to do just fine, even with your legacy code.
elusiveoso@reddit
You would be better served making an authentic problem that more closely resembles what your engineers do day to day.
I would go looking through your past issues and see if you can find something more closely tied to a problem they might face. Also, people use AI, and IDEs, and search engines. Can we quit with the nonsense of putting them in an environment where they can't use that stuff for part of the hiring process?
BoomBeachBruiser@reddit
I wouldn't be surprised if new grads had difficulty coding during in an interview. That's kind of a skill in and of itself, and I can see a 21-year-old going full deer in the headlights trying to do a generic NxN tic-tac-toe game server under pressure.
It's been a long time since I've interviewed new grads, but I liked to have them walk me through a school or personal project that they're proud of. Then, ask questions about it. How did they approach it? How did they come up with the design? What was tricky? What took the most time? How did they test and debug it? What would they do differently with 20/20 hindsight? Etc.
nodonaldplease@reddit
Q. My company is pushing to AI first. Not just engineering. Everyone. QA/ Product/ DevOPS You name it.
They provide access to Copilot/ cursor.
Non engineering folks (tech PM) presented how they used AI to write PRD, generate code and get a working version prototype based on a simple idea.
Sure it was looked at by management like wow, AI is paying off. No one looked at the code. No QA. No testing. Nothing but the fact that this was developed by using AI was highlighted multiple times.
That said do we need folks that understand coding? Not in the short term cause now companies can develop new features and ship them in no time.
DM_ME_UR_PUBES@reddit
coming at this from the opposite point of view, how would you suggest new grads should prepare to be ready for both the interview and actual day to day in the job
anoncology@reddit
Interview me!!! ;)
engineered_academic@reddit
Lots of issues here with the exam. You have to account for cultural biases. I don't like using games as an example. Some people may have never run into tic tac toe before. Some people may not know the quirks of the game for example, that it is essentially a solved game (playing optimally the first player can ALWAYS win); or have been playing with house rules.
The ruleset must be clearly explained and edge cases must be explained. For example, it is possible for a tie to occur in tic tac toe if there is suboptimal play in certain conditions. since some people know it is a solved game, they may not be expecting to have a tie as an edge case.
Furthermore, rarely these days am I implementing something purely from scratch. 90% of development isn't greenfield so you are testing muscles that have rarely been flexed. It took me a good 5-10 minutes to think of a solution and that isn't under time pressure or panipticon pressure, just me sitting on the toilet taking a shit.
IMO there are better examples to walk through, and I usually have preformatted code ready to go with key sections cut out. This tests their ability to work within frameworks and exisiting code/specs, and does decently well at foiling AI because they can't just copy and paste things into an AI chatbot because I have the rest of the application on my computer and provide them as needed.
MangoTamer@reddit
God help you if you're trying to hire me because my interviewing skills are ass right now. Do you I think people still study algorithms for fun in their free time? No, people are busy building things by using AI and they are too busy integrating existing LLMs into their systems.
I can prepare myself for the AI interview or I can prepare myself for the coding but I don't have time to do both.
midwestcsstudent@reddit
Sounds like you need to adapt your hiring process to how people actually work today. I haven’t actually found a sweet spot, but just seeing how someone codes (AI allowed) gives you a lot of intuition into whether they’re smart enough to hire.
ButWhatIfPotato@reddit
Do you have the salary on the job spec?
downshiftdata@reddit
Ask your team. Do they have any recommendations? Chances are good they'll know people who they think will fit the role and with whom they'd like to work again.
Contractor outfits can also help. I work for one. Let them do the vetting and provide you with a curated list. Yes, it'll cost you more, but that's how these things work. And if you're still seeing crap candidates, then go to their competitors.
Beli_Mawrr@reddit
HR kicks most of these people to the curb in favor of the ones that hack through the ATS.
Ok_Chemist_3576@reddit
u/OP Engineering is not a buzzword-matching game. Engineering is not a step-by-step guessing game that you expect a person to match like a choreography.
That tictactoe I implemented a couple of times college with excehllent grades? Not fresh anymore in mind.
Are your mind fresh on basic school maths? Binomials, Pascal Triangle. Basic school stuff I don't think you could answer fresh out of mind if suddenly asked. Same with tictactoe.
Exactly. That's fresh in their minds. Just as school teenager is more fresh than you on binomials and Algebra. However, you didn't hired the new grad. Think about it.
Now, You haven't asked, but I am a Senior Developer. I could tell you about the feature I deployed yesterday. Clients happy, business happy, me happy. They are today celebrating this new feature, not missing tictactoe at all.
I know you'll never ask, but I am working today on a security fix that will make everything more reliable for all of us. CORS is a simple --yet powerful-- concept that get's underrated most of time. I am solving CORS for them. They are worried about CORS, not tictactoe.
gHx4@reddit
It sounds like your screening isn't pushing quality candidates to the top, and it's getting drowned in vibecoders. Quality candidates are still out there so you may need to talk with HR and determine what part of the screening pipeline is selecting so many vibecoders and failing to find the candidates that can problem solve.
Master_Addendum3759@reddit
Wow thanks for such a sharp insight!
Michaeli_Starky@reddit
What positions are you interviewing them for?
dExcellentb@reddit (OP)
Software engineer, entry to senior
Michaeli_Starky@reddit
That doesn't tell me much. What are they going to be working on? Gamedev?
Wide-Pop6050@reddit
It made a big difference when we started working with a recruiter and also with job boards that you have to put some effort to view, like the built in ones
dbudyak@reddit
Also, how to deal with drop in quality of hiring processes, HR interviews, job positions and expectations, and rejection feedbacks?
stoooooooooob@reddit
What is your field?
jonnycoder4005@reddit
I've been writing software for 20 years and I have never had to deal with a 2D Array
sweetno@reddit
It depends on the domain.
_iggz_@reddit
If you had me n create some game I wouldn't want to work for you, what an absolute waste of my time.
Foreign_Clue9403@reddit
This is a neat problem. Wouldn’t get past phase 2 if your interview is 45 minutes and on standard coderpad because things take much longer if I have to talk out loud and demonstrate process, then debug etc.
I’ve understood for a long time that coderpad interviews emphasize the wrong thing as compared to whiteboards, so I’m glad there’s some forcing function on the “write the code right now”
_SnackOverflow_@reddit
Honestly that sounds like a really fun interview question!
What types of roles are you hiring for? How do you advertise?
tsereg@reddit
Have you really experienced drop in candidate quality, or an influx of candidates that previously wouldn't even apply? Because, if it is latter: a classroom with multiple candidates at once, leaving their devices at the door, paper and pencil on desk, and they all have to write down a pseudocode for a very simple problem that should take 30 minutes at most.
You could even employ machine learning to monitor them for no faul play. 😊
jtonl@reddit
It would be the perfect time to brush up teaching skills from the lessons of the gruelling experience we're currently having right now.
Noway721@reddit
That's the neat part, you don't.