ForgeApply · Job listing
Rust Engineer
Litellm
See all 10 open roles at Litellm →
Tailor your resume for this Litellm job in about a minute.
ForgeApply rewrites your resume for this exact posting, then autofills the application on Litellm's site with it. You review everything before it's sent. Free trial, no card required.
About this role
Rust Engineer
LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services. We're looking for a Rust Engineer to own the migration of LiteLLM from Python to Rust and make us the fastest AI gateway ever built.
About The Role
You'll own LiteLLM's migration to Rust end to end. This is a high-stakes, high-visibility role: you'll plan the migration directly with the CTO, structure the codebase so it's ready for Rust, and lead a gradual, stability-first rollout. The work is fundamentally about performance. Every decision you make will move us toward being the lowest-latency, highest-throughput gateway in the category, without ever compromising the reliability our enterprise customers depend on.
This is a single-owner mandate. You will be the person leading the entire Python-to-Rust migration.
Responsibilities
- Own the end-to-end migration of LiteLLM from Python to Rust, from architecture through rollout.
- Plan the migration alongside the CTO, sequencing work to minimize risk to live, high-volume traffic.
- Design and execute a gradual rollout that holds stability as the top constraint while the migration progresses.
- Restructure the existing codebase so it's positioned for a clean, incremental move to Rust.
- Drive performance gains across latency, throughput, and resource efficiency at 1B+ requests/mo.
- Collaborate cross-functionally to ensure the migration aligns with backend priorities across performance, reliability, and security.
What We're Looking For
- Deep, long-standing expertise in Rust. You've been writing production Rust for years and know it cold.
- Obsessive about performance. You care about latency and throughput at a level most engineers don't.
- Strong instinct for stability, with a track record of balancing ambitious change against reliability.
- Major bonus: prior experience leading codebase migrations from another language to Rust.
- Major bonus: a demonstrated history of shipping performance-critical systems.
- Familiarity with backend infrastructure, databases (PostgreSQL, Redis), and high-volume production systems.
Why Join LiteLLM?
- Own one of the most important technical bets in the company's roadmap.
- Define the architectural foundation that the entire platform will run on.
- Fast-paced growth environment with deep technical ownership.
- Competitive salary, health, dental, and vision benefits.
About LiteLLM
LiteLLM (https://github.com/BerriAI/litellm) is a Python SDK and Proxy Server enabling seamless calls to 100+ LLM APIs in the OpenAI format, trusted by industry leaders worldwide.
Ready to make LiteLLM the fastest AI gateway ever? Apply now!
Salary insight
The midpoint of this range ($260k) is about 30% above the median disclosed salary for San Francisco roles listed on ForgeApply ($200k across 8,773 jobs).
Based on live postings with disclosed pay on ForgeApply; refreshed daily. Not an estimate of this employer's offer.
Tailor your resume for this Litellm role before you apply.
Tailor my resume for this jobSimilar jobs
- Systems Engineer (Rust) — Maticrobots · Menlo Park, CA
- Senior Rust Software Engineer — Ionq · Pleasanton, California, United States
- Principal Rust Developer — Zscaler · San Jose, California, USA
- Software Engineer, Rust — Helsing · Washington, DC
- Rust, AI Engineer, Director — Blackrock · New York, NY
- Senior Rust Blockchain Engineer — Startale · Remote
- Senior Software Engineer - Rust — Defenseunicorns · Remote
- Staff Software Engineer, Rust — Apollo Graphql · Remote
Free ATS checker · How to Tailor Your Resume to a Job Description (Step by Step)