Staff AI Engineer

About this role

Join the Team Modernizing Medicine

At  ModMed , we’re not just building software—we’re reimagining the healthcare experience. Founded in 2010 by a practicing physician and a successful tech entrepreneur, we took a radically different approach:  we hired doctors and taught them how to code.  This "for doctors, by doctors" philosophy has allowed us to create an AI-enabled, specialty-specific cloud platform that places patients at the center of care.

A Culture of Excellence

When you join ModMed, you’re joining an award-winning team recognized for innovation and employee satisfaction.   From our global headquarters in Boca Raton Florida, and extensive employee base in Hyderabad India, we are a team of 4,500+ passionate problem-solvers on a mission to increase medical practice success and improve patient outcomes:

• Consistently ranked as a Top Place to Work

• 2025 Globee Business Awards:  Gold Globee for “Technology Team of the Year”

• 2025 Black Book Awards:  Ranked #1 EHR in 11 Specialties

• Florida Venture Forum:  Venture-Backed Company of the Year

We are growing fast, thinking big, and we are just getting started.

Ready to modernize medicine with us?

Job Description Summary: As a Staff AI Engineer, you define and drive the architecture of AI and agentic systems across multiple teams and product domains. This is a senior individual-contributor leadership role: you influence high-impact architectural decisions, evolve the practices and standards for building agentic AI, and turn experimental AI capabilities into reliable production systems. You set direction for multi-agent orchestration, production RAG (hybrid search, re-ranking, and query routing), tool and MCP integration, and the evaluation and observability stack that keeps them dependable. You mentor senior engineers and represent AI engineering in cross-functional and strategic initiatives. A background in classical ML is an asset; the primary requirement is a proven track record of shipping production agentic AI.

KEY RESPONSIBILITIES • Define and drive technical direction for AI and agentic systems, and contribute to the AI platform roadmap across teams

• Influence architecture decisions for compute, cloud, and AI infrastructure across teams

• Lead the design of large-scale AI/LLM systems: inference platforms, APIs, and distributed architectures

• Architect production multi-agent systems end-to-end: orchestration, state management, tool integration, and failure handling

• Define and drive best practices and standards for AI/LLM systems across teams (agent design, evaluation, observability, reliability)

• Lead complex production debugging and incident response across teams, and harden the resulting fixes into platform guardrails

• Mentor senior engineers and emerging technical leaders, raising the engineering bar

• Lead technical design reviews and architecture decision records (ADRs) for critical AI infrastructure

• Contribute to capacity planning and cost optimization strategies for AI/LLM infrastructure

GENAI / AGENTIC AI CAPABILITIES • Define and drive vector database and RAG architecture decisions across systems and teams: structured RAG, hybrid search (dense + sparse + keyword), re-ranking, and query routing

• Lead multi-agent platform architecture decisions: runtime selection, orchestration patterns, and enterprise integration strategy

• Set the technical direction for MCP (Model Context Protocol) adoption and agent runtime infrastructure

• Shape agent infrastructure adoption: evaluate and standardize frameworks, tooling, and deployment patterns for agentic AI

• Architect evaluation infrastructure for non-deterministic LLM systems: synthetic golden-set generation, hierarchical weighted scoring (component, composite, and system-level F1), bootstrap confidence intervals, and paired A/B comparison, treating a change as real only when it is both statistically significant and clears a minimum effect size

• Gate deployments on eval results: tiered regression thresholds (hard-gate vs monitor components) wired into CI so a measurable quality regression blocks the release, with observability via tracing across multi-step chains and tool calls and drift detection on LLM inputs and outputs

• Drive LLM cost optimization at scale: model routing, caching, batching, token budget management, and provider cost analysis

REQUIRED SKILLS & QUALIFICATIONS • Master’s or Ph.D. degree in Computer Science, Software Engineering, or a related field.

• 10+ years of professional experience in ML/AI or software engineering, including 4+ years in senior or staff-level roles with production system ownership

• Demonstrated engineering leadership, including driving technical strategy and influencing cross-team decisions

• Expertise in platform and distributed-systems architecture at scale: model serving, APIs, data platforms, and AI/LLM infrastructure

• Hands-on experience architecting and operating production agentic AI or LLM systems (multi-agent workflows, production RAG, tool and MCP integration)

• Deep understanding of embedding models, retrieval algorithms, and vector database internals

• Strong production debugging, reliability, and incident-response skills

• Experience building rigorous evaluation for non-deterministic AI systems, including statistical methods (such as bootstrap confidence intervals and minimum effect-size thresholds) to separate genuine quality changes from run-to-run model variance

• Cost-awareness for cloud AI/LLM workloads: capacity planning and cost optimization

• Proven mentorship of mid-level and senior engineers

• Strong communication skills for executive and cross-functional audiences

PREFERRED QUALIFICATIONS (NICE TO HAVE) • Experience in Healthcare, FinTech, or other regulated industries

• Experience building AI/LLM systems or platform components from the ground up

• Defined best practices for AI-assisted development (Claude Code): code quality standards, review, and responsible usage

Similar jobs

• Staff AI Engineer — Pragmatike · Remote
• Staff AI Engineer — WGU · Raleigh, NC
• Staff AI Engineer — Oscar · New York, New York, United States
• Staff AI Engineer — Greenlitecareers · New York City
• Staff AI Engineer — Fieldguide · San Francisco, CA
• Staff AI Engineer — Drata · Remote
• Staff AI Engineer — Workato · Palo Alto, California
• Staff AI Engineer — Crunchyroll · Dallas, Texas, United States