Staff Software Engineer - AI II

About this role

Principal Engineer

CoCounsel is our most advanced AI offering to date — combining generative and agentic AI capabilities to help legal professionals move beyond prompting and start delegating. As a Principal Software Engineer focused on Cloud Cost Optimization, you will be the embedded, go-to expert within [Product] engineering for how cloud and AI/model spend is tracked, forecast, and reduced without sacrificing reliability or user experience. You will partner closely with the platform/infrastructure team (who own provisioning and LLM-serving architecture), AI/ML engineers, product, and FinOps to make cost a first-class engineering concern inside the product team.

About the Role In this opportunity as a Principal Software Engineer, you will: • Be the team's cost optimization authority:  Serve as the go-to resource for cloud and AI-inference cost questions within the product engineering team — identifying, quantifying, and driving high-impact optimization opportunities across compute, storage, and model usage. • Analyze and tune LLM usage for cost efficiency:  Partner with AI/ML and platform teams to right-size model selection, routing, and usage patterns (batching, caching, prompt/response size, provider mix) for cost-efficiency without regressing quality or latency SLAs. • Own cost tracking and forecasting tooling:  Build and maintain dashboards, showback/chargeback reporting, and forecasting models that give the product team visibility into spend per feature, per workflow, and per customer segment. • Advise on elasticity and caching from a cost lens:  Work with platform engineering on auto-scaling, caching, and queuing designs, weighing in specifically on cost/performance tradeoffs rather than owning the underlying architecture. • Cross-functional influence:  Lead cost-reduction initiatives that cut across product, AI/ML, and platform teams. Mentor senior/staff engineers on cost-aware engineering practices and FinOps habits. • Cost observability and governance:  Establish spend-related SLOs, budget alerts, and anomaly detection; lead the response when spend deviates unexpectedly.

​ About You You are a fit for the position of Principal Software Engineer if your background includes:  • Bachelor's Degree in CS/CE or equivalent experience. • Solid hands-on cloud infrastructure experience (AWS preferred) — compute, networking, storage, IaC (Terraform/CloudFormation) — with a demonstrated focus on  cost , not just architecture. • Experience analyzing and optimizing cost/performance of production LLM-serving systems (multi-provider routing, caching, batching); you don't need to have built the serving platform, but you need to understand it deeply enough to find savings in it. • Track record driving cross-team cost optimization initiatives with measurable, quantified dollar impact at scale. • Working proficiency in Python (or similar), and enough familiarity with FastAPI/Kubernetes/distributed systems to read, instrument, and reason about production code and infra. • FinOps practices and tooling for cloud and inference cost at scale (forecasting, chargeback/showback, budget governance) — this is now core, not a nice-to-have. • Observability/reliability practices as applied to spend — cost SLOs, dashboards, anomaly alerting, incident response for cost spikes. • Strong communication across product, AI/ML, platform, and FinOps stakeholders.

​ Preferred Skills & Experience:   • GPU/accelerator infrastructure and inference optimization (batching, quantization, custom serving) — still highly relevant since inference is usually the largest cost driver. • Track record building or customizing FinOps tooling in-house. • Familiarity with vector databases/embeddings/retrieval pipelines, specifically to reason about their cost footprint. • Multi-region/multi-cloud cost tradeoffs (not required to own DR architecture).

#LI-ES1     New Position: This position is open due to an existing vacancy to support our evolving business needs. What’s in it For You? • Hybrid Work Model: We’ve adopted a flexible hybrid working environment for our office-based roles while delivering a seamless experience that is digitally and physically connected. • Flexibility & Work-Life Balance: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance. • Career Development and Growth: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrow’s challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future. • Industry Competitive Benefits: We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing. • Culture: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our values: Obsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together. • Social Impact: Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives. • Making a Real-World Impact:  We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphol

Similar jobs

• Staff Software Engineer, AI — Paveakatroveinformationtechnologies · San Francisco, CA & New York, NY
• Staff Software Engineer (AI) — Pendo · New York, NY
• Staff Software Engineer, AI Engineering — Tebra · Remote
• Staff Software Engineer, AI Engineer — Midihealth · Hybrid - SF Bay Area
• Staff Software Engineer, Engineering AI Team — Second-nature · Remote
• Staff Software Engineer, AI/ML — Digitalocean98 · Seattle
• Staff Software Engineer, AI/ML — Mavenclinic · New York, NY; Remote, US (Hub cities)
• Staff Software Engineer - AI Products — Snowflake · Remote