I’ve spent a significant part of my career as an Amazon Bar Raiser — conducting well over two hundred interviews across my own organisation and teams throughout Amazon. Somewhere along the way I stopped paying much attention to the things that most interview scorecards are designed to evaluate. Not because they’re irrelevant — behavioural anchors and competency frameworks serve a real purpose — but because I’ve found they’re much better at identifying people who are good at interviewing than people who are good at leading.
Those are different skills. The overlap is real, but it’s partial, and conflating them is one of the more expensive mistakes in engineering hiring.
What polish hides
The candidates who worry me most aren’t the nervous ones or the ones who can’t quite articulate a framework on demand. They’re the ones who are flawlessly polished. Every answer is crisp, well-structured, hits the right notes. They’ve clearly thought about these questions before — probably coached, probably prepared extensively, and probably quite good at reflecting post-hoc on what they did and framing it well.
That’s a useful skill in some leadership contexts. It’s also a skill that’s fairly decoupled from whether someone can actually develop people, make hard calls, or hold a team together during a difficult period. The narrative coherence of a past story tells you a lot about how someone thinks about that story today. It tells you much less about how they thought and acted in the moment.
What I’m looking for is moments where the polish breaks down a little — where someone has to think, where the answer doesn’t come immediately, where they change course in the middle of a sentence because they’re working something out. That’s where I learn something real.
The question I keep coming back to
Over time, I’ve found one question that consistently reveals how someone actually thinks about people leadership: “Tell me about someone on your team whose growth you felt responsible for and whom you feel like you let down.”
The good version of that answer is messy. It involves genuine regret, specific details, and a real accounting of what they did or didn’t do. It probably also involves some honest uncertainty about whether things went as badly as they feared, and some reflection on what they’d do differently. It doesn’t resolve cleanly.
The coached version of that answer identifies a “growth area” that’s really an asset, resolves happily, and leaves the interviewer feeling reassured. It’s the version of a story that someone who’s practiced telling stories about leadership failures will tell. You can usually tell which one you’re getting within the first two minutes.
I ask some version of this question in almost every senior leadership interview because it’s genuinely hard to answer well without having actually cared about the development of your people — and if you haven’t genuinely cared about that, the rest of the job is probably not being done well either.
What I actually want to know
Underneath most of the questions I ask in an engineering leadership interview, I’m trying to get at a small number of things.
How do they think about the relationship between their success and their team’s success? Candidates who talk mostly about what they built and shipped — where the agency is clearly theirs — give me different information than candidates who talk mostly about what their teams accomplished and what conditions they created for that to happen. Neither is automatically better, but the frame reveals something.
How do they handle disagreement with peers or with their own leadership? I don’t care much about the specific outcome of the disagreement they describe. I care about whether they stayed engaged with the hard parts or found ways to smooth over the conflict without resolving it.
What do they do when they don’t know something? Not in a technical sense — in a judgment sense. How do they make decisions under genuine uncertainty, and how do they talk about the decisions that turned out to be wrong?
The thing that most separates good candidates
I’ve interviewed people who could answer every competency question perfectly and were mediocre leaders in practice. I’ve interviewed people who stumbled through the interview and turned out to be excellent. The strongest pattern I’ve found for predicting the latter: candidates who talk about people on their past teams as specific humans with names and particular characteristics, not as abstractions in a story about their own development.
The candidates I most want to hire are the ones who still seem to carry their past teams with them a little. Their people aren’t props in their career story — they’re people they were responsible for and still think about.