How to Interview AI Engineers

Interview structure should follow role shape. An applied AI engineer shipping product features, an agentic engineer owning actions, an inference or infrastructure engineer, and a research engineer are four different evaluations. Running one generic AI loop across all four produces false negatives on the strongest specialists.

Write down, before the loop is designed, the two capabilities that would make this hire successful in the first six months. Every interview stage should map to one of them.

Stage 1 — hiring manager conversation (45 minutes). Discuss one system the candidate built end to end. Push on what broke, what they measured, and what they would build differently. This single conversation is the highest-signal stage in most loops.

Stage 2 — engineering signal (60–90 minutes). Real code in a realistic setting: extend an existing small codebase, debug a failing pipeline, or work with imperfect data. Avoid competitive-programming puzzles, which correlate poorly with applied AI performance.

Stage 3 — AI system design grounded in your product (60 minutes). Give a real constraint set: latency, cost, data sensitivity, quality target. Grade the tradeoff reasoning, not the architecture diagram.

Stage 4 — evaluation and judgement (45 minutes). How would they know it works? What is offline, what is online, what regression would they watch for after a model change? Weak candidates describe vibes; strong candidates describe harnesses.

Add a values or collaboration conversation only if it carries a real decision. A fifth stage that changes nothing costs you candidates.

Applied AI / LLM engineers: retrieval design against messy corpora, prompt and context strategy under token constraints, output quality measurement, and cost control.

Agentic engineers: tool interface design, failure containment on risky actions, trajectory evaluation, and idempotency.

Inference and AI infrastructure engineers: throughput and utilization reasoning, batching and quantization tradeoffs, GPU memory behaviour, and reliability under load.

Research engineers: distributed training debugging, data pipeline design, experiment design, and honest evaluation of ambiguous results.

For every specialization, one question generalizes well: 'tell me about a time your system looked good on your metric and bad in production'.

If you use a take-home, cap it at three hours, pay for it where you can, and guarantee a review conversation with a real engineer. Unpaid multi-day exercises drive senior candidates out of the process.

A strong alternative: a paid or unpaid pairing session on a slice of your real backlog. It gives better signal than a synthetic exercise and shows the candidate what the work is.

Never ask for work that resembles unpaid consulting on a live problem you intend to ship.

Score each stage against the two capabilities you defined, on a fixed scale, written before the debrief. Free-form impressions favour candidates who present well and disadvantage strong engineers who interview quietly.

Require written evidence for each score. 'Strong' without an example is not a signal.

Debrief within 24 hours and decide within a week of the final stage.

Calibrate the bar on the same day you open the role, not after four rejections.

The most frequent cause of a lost AI hire in our searches is not compensation. It is elapsed time and inconsistent signals between interviewers.

Candidates increasingly evaluate the company through the interview loop itself: a vague AI design stage reads as a team without a clear technical direction.

Loops that include one conversation with a senior engineer the candidate would learn from convert noticeably better than loops that do not.

Framework-specific screening questions are the most common source of false negatives we see, because tooling in this space turns over faster than the loop is updated.