AI Model Policy Trainer, Mental Health

About this role

ABOUT HANDSHAKE

Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.

In 2025, we started Handshake AI and built the fastest-growing AI data business in history. We work directly with frontier AI lab researchers to create evaluations, publish benchmarks, and push the boundary of data. We’ve grown from $0 to ~$1B run rate and pay ~$60M to over 30K individuals every month.

Why join Handshake now:

- Shape how every career evolves in the AI economy, at global scale, with impact your friends, family and peers can see and feel

- Partner hand-in-hand with world-class AI labs, Fortune 500 partners and the world’s top educational institutions

- Work together with engineers, scientists, operators, and more from Palantir, Meta, Scale AI, and former YC founders

- Build a massive, fast-growing business with billions in revenue

About Handshake AI

Human data is the core infrastructure to AI advancement. Frontier AI labs currently improve model capabilities with various data-intensive post-training techniques. We believe that data spend for AI training will increase by 3-5x in the next few years and continue for much longer as models take on new domains. Handshake AI supports all of the frontier AI labs, working on their most complex data at the largest scale.

ABOUT THE ROLE

As an AI Model Policy Trainer focused on mental health, you will help evaluate how conversational AI systems respond to people discussing emotional distress, mental-health conditions, unusual beliefs, possible crises, and other sensitive experiences.

You will review conversations and model behavior against a detailed mental-health safety policy. Your work will help determine whether an AI response is empathetic, appropriately grounded, and sensitive to risk without reinforcing unsupported or potentially harmful beliefs.

This work requires an understanding of why people turn to AI for conversation, reflection, emotional support, or help during difficult moments. Strong candidates will recognize that AI can be useful and accessible while also understanding risks such as inappropriate validation, missed signs of crisis, overreliance, misleading guidance, or reinforcement of harmful beliefs.

You do not need to come from a traditional technology background. We are looking for people who combine practical experience supporting individuals or families with curiosity about AI, careful judgment, and a willingness to learn.

WHAT YOU WILL DO

- Review user conversations and AI responses involving mental health, emotional distress, crisis, unusual beliefs, paranoia, hallucinations, mania, and related experiences

- Apply detailed policy taxonomies and evaluation criteria to classify user requests and model behavior

- Evaluate both individual messages and changes in risk or certainty across longer conversations

- Distinguish between validating a person’s emotions and validating an unsupported or potentially harmful explanation

- Assess whether AI responses remain neutral and reality-based or improperly reinforce, escalate, or operationalize harmful beliefs

- Evaluate actions taken by AI systems through tools, including searches, messages, scheduling, purchases, file creation, and publication

- Write clear rationales that connect each evaluation decision to evidence from the conversation and policy

- Participate in calibration exercises and work with other trainers to improve consistency

- Identify ambiguous examples, gaps, and edge cases that may require policy clarification

- Review the work of other trainers and provide thoughtful, evidence-based feedback

- Incorporate coaching and adapt as policies, rubrics, and model capabilities evolve

- Collaborate with researchers, policy specialists, operations teams, and other subject-matter experts

- Work regularly with sensitive and potentially distressing content

WHO MAY BE A GOOD FIT

We are especially interested in candidates with experience in one or more of the following areas:

- Mental health or behavioral health

- Social work or community-based services

- Nursing or patient care

- Professional caregiving

- Certified peer support

- Family advocacy or care navigation

- Crisis support or related human services

Clinical licensure is valuable but is not required for every candidate. Relevant professional, caregiving, advocacy, peer-support, or structured lived experience may also prepare someone to succeed in this work.

DESIRED CAPABILITIES

- Practical experience supporting individuals or families through mental-health challenges

- Familiarity with conversational AI tools and an interest in how people use them for emotional support

- A balanced understanding of both the potential benefits and risks of AI-mediated mental-health conversations

- Ability to interpret sensitive conversations with empathy, precision, and cultural awareness

- Ability to follow detailed policies and distinguish between closely related categories

- Strong written communication and the ability to explain decisions using specific evidence

- Sound judgment when information is incomplete, ambiguous, or context-dependent

- Openness to feedback, calibration, and evolving policy guidance

- Ability to raise questions or disagreements constructively and work toward consistent application of the policy

- Strong boundaries and the ability to engage with sensitive material professionally and sustainably

- Comfort working independently in a detail-oriented, feedback-heavy environment

EXTRA CREDIT

- Experience with structured review, case documentation, assessment, triage, quality assurance, or rubric-based evaluation

- Experience working with people experiencing crisis, psychosis, paranoia, mania, hallucinations, suicidal ideation, or severe emotional distres

Similar jobs

• AI Trainer — HonorHealth · Remote
• AI/ML Model Engineer — Intrepid Global Solutions · McLean, VA
• AI Training and Enablement — Nb · New York, NY
• AI Program Manager — Moog · Buffalo, NY
• AI Program Manager — Preciselyusjobs · Remote
• AI Program Manager — Insperity · Kingwood, TX
• AI Implementation Manager — Lio · New York Office
• IT Trainer — MAIN · HITEC Center - San Diego, CA