We do a simple offline idiot test to see if they know how to rub two variables together (10 minutes of effort for a junior, say), then we go into a face-to-face interview where we give them a laptop with a blank project (no frameworks, no extra packages), and we all work through a moderately difficult logic problem (30-40 mins) together on a big screen. This gives us far more insight into their problem-solving skills, their ability to work with others, their knowledge of comp-sci and language fundamentals (the ones that count, no implementing a B-tree from scratch or some such nonsense). They still have access to online resources such as stack overflow, because that's part of problem solving and most programmers' workflow. we have had good results, and have employed some bad-on-paper candidates who have turned out to be great programmers, and turned away some candidates who had great CVs, but turned out to be not great problem solvers or team players.
We've found that large take-home tasks disadvantage certain candidates who don't have time or patience (tech interview fatigue, for example). They also don't really weed out candidates who will hack around on a 4 hour task for 20 hours until they cobble together something passable, or - as in your case - feed the whole assignment to an LLM. There's no substitute for hands-on experience with the candidate, but you need to be willing to put senior resources with them for that hour or two.
Comments
We do a simple offline idiot test to see if they know how to rub two variables together (10 minutes of effort for a junior, say), then we go into a face-to-face interview where we give them a laptop with a blank project (no frameworks, no extra packages), and we all work through a moderately difficult logic problem (30-40 mins) together on a big screen. This gives us far more insight into their problem-solving skills, their ability to work with others, their knowledge of comp-sci and language fundamentals (the ones that count, no implementing a B-tree from scratch or some such nonsense). They still have access to online resources such as stack overflow, because that's part of problem solving and most programmers' workflow. we have had good results, and have employed some bad-on-paper candidates who have turned out to be great programmers, and turned away some candidates who had great CVs, but turned out to be not great problem solvers or team players.
We've found that large take-home tasks disadvantage certain candidates who don't have time or patience (tech interview fatigue, for example). They also don't really weed out candidates who will hack around on a 4 hour task for 20 hours until they cobble together something passable, or - as in your case - feed the whole assignment to an LLM. There's no substitute for hands-on experience with the candidate, but you need to be willing to put senior resources with them for that hour or two.