I. The Importance of the Interview System
Hiring top talent is crucial to the mission of an organization, and effective interviews are key. The interview process should be reliable, effective, and minimally disruptive to assess whether the candidate is suitable for the position and team.
II. Technical Skills Assessment
Software Engineering Basics
Most ML/AI positions require basic software engineering capabilities. There may be a 30-60 minute coding exercise, such as checking arrays, implementing inference endpoints, building data processing pipelines, etc. Completing the coding exercise is only part of it; more importantly is the way the problem is solved, including logical decomposition, code quality, consideration of edge cases, and response to feedback.
Data Literacy
This is a key but often overlooked skill for ML/AI positions. It includes understanding and respecting data, being proficient in data analysis, and having an intuition for questioning data or analysis. It is important to understand how data is collected and stored, check for missing values, outliers, and inconsistencies, perform data cleaning and preprocessing, and have the intuition to question analyses by asking relevant questions to assess the candidate's data literacy.
Adaptability to Opaque Model Outputs
ML models are different from regular databases or APIs in that they have uncertainty and opacity. Adaptable candidates understand that they cannot fully control or explain the model and know that the model will reflect the biases of the training data, and will establish validators and strategies to meet user and business needs. Understand the candidate's capabilities in this area by asking relevant questions.
Basic Understanding of Evaluation
Evaluation is key for those building ML products. Ask candidates how they measure model performance, deal with situations where performance exceeds thresholds, and methods for collecting initial evaluation data and building an evaluation framework. For AI engineer positions that apply pre-trained models and APIs to build products, the above skills are usually sufficient. Positions related to research or applied science also need to assess the breadth, depth, and application of science.
Breadth of Science: Assess the candidate's familiarity with different ML fields, starting from the basics of supervised and unsupervised learning, and gradually delving into specific areas to understand their knowledge breadth and gaps.
Depth of Science: The candidate demonstrates expertise in chosen projects, understanding the rigor with which they think about and execute problems, including design considerations, dealing with constraints and trade-offs, and the impact on customers and business.
Application of Science: Requires candidates to apply skills and knowledge to practical problems, focusing on the problem-solving thought process and different perspectives considered.
III. Non-Technical Abilities Consideration
Ambiguity
Refers to the degree of ambiguity of the problem when the candidate starts work, including whether the scope of the problem is clear, whether there is a known solution, the degree of guidance, and the scope of the field involved in the problem.
Influence
Includes the scope of the candidate's cooperation and influence through others, as well as the mechanisms of influence, such as involving teams, business lines, and even industries and communities, which may also extend to product and business decisions.
Complexity
Refers to the complexity of the problem space, understanding the level of complexity the candidate has dealt with and the effectiveness of the solutions.
Execution
The candidate's ability to deliver within limited resources and time, including considering the speed of learning and iteration after innovative failures, as well as the strategic significance and scope of the solution.
IV. Phone Screening, Interview Rounds, and Reporting
Phone Screening Calibration
Phone screenings should be selective enough to reduce the number of marginal candidates who struggle in the interview rounds. Hiring managers should decide whether to invite candidates for internal interviews based on the results of the phone screening, and should not invite them if they are uncertain.
Interview Round Operation and Reporting
Before the interview, conduct a pre-report to clarify the job requirements and considerations. The interview uses the STAR format to collect data, including the situation/task, action, and result. By asking technical and non-technical questions, you can gain a deeper understanding of the candidate, and also clarify the candidate's direct contributions. After the interview, the interviewer writes feedback and votes independently, then reports, and decides whether to hire the candidate based on the comprehensive situation.
V. Suggestions for Interviewers and Hiring Managers
Interviewers
Make candidates feel comfortable to uncover their strengths and potential, treat candidates as potential customers or users, and make the interview a valuable experience even if they are not hired.
Hiring Managers
Attract top talent through the organization's mission, team talent density, and personal development opportunities, while setting the right expectations, not exaggerating the position, and being prepared for the time required for recruitment, especially for senior positions.
VI. Traits of Top Talent
Top talent should have a sense of hunger, judgment, and empathy. Hunger is manifested in a tendency to act, a quick learning ability, and perseverance in overcoming challenges; judgment is the intuition to distinguish what is feasible from what is not; empathy is a sincere concern for customers, organizations, and teams. These traits are mostly considered during recruitment, rather than being developed later.