Data Engineering Architect, Senior

About this role

You are a data engineer who thrives in a highly collaborative environment, partnering with product, analytics, and engineering teams to deliver high-quality, trusted data. You’re motivated by building scalable data systems and shaping how data is modeled, governed, and consumed across a modern cloud platform. You bring deep, hands-on data engineering experience and are equally comfortable designing future-state architecture, building production solutions, and establishing the patterns and standards that allow others to build effectively. You bring recent, hands-on production experience with Databricks and will play a leading role in evolving our Databricks-based Lakehouse architecture, including the modernization and migration of existing data workloads. You enjoy translating complex product and user behavior into well-structured, reliable datasets that power analytics, experimentation, and decision-making. You can move comfortably between technical implementation and strategic architecture, communicating complex decisions clearly and influencing technical direction across teams. You bring recent, hands-on production experience with Databricks and will play a leading role in evolving our Databricks-based Lakehouse architecture, including the modernization and migration of existing data workloads. You enjoy translating complex product and user behavior into well-structured, reliable datasets that power analytics, experimentation, and decision-making. You can move comfortably between technical implementation and strategic architecture, communicating complex decisions clearly and influencing technical direction across teams.

Primary Responsibilities : • Partner with product analytics stakeholders to translate business-defined KPIs and data requirements into scalable, production-grade datasets.

• Own the design, build, and operation of scalable data pipelines end-to-end (ingestion → transformation → serving).

• Define and evolve the architecture of the Product Analytics Lakehouse, making technical decisions that improve scalability, performance, reliability, governance, and consistency across datasets and workloads.

• Build and maintain production-grade, well-modeled datasets (Gold layer) that power analytics and AI use cases.

• Define, implement, and drive adoption of reusable data engineering patterns, frameworks, standards, and guardrails that reduce duplication, improve engineering leverage, and make the right development patterns easier to adopt.

• Own data quality and reliability for production datasets, including validation, monitoring, SLAs, and incident resolution.

• Productionize and scale prototype datasets and logic developed by analytics partners into reliable, maintainable data pipelines.

• Build governed, purpose-built datasets to support AI/ML use cases while enforcing controlled and secure data access patterns.

• Lead the technical evolution of workloads into Databricks, evaluating existing architecture and determining appropriate migration, modernization, and coexistence strategies.

• Make and communicate architectural tradeoffs across performance, cost, reliability, governance, maintainability, and developer experience.

• Provide technical leadership and architectural guidance across Product Analytics, helping engineers and analytics partners make sound data architecture, modeling, and platform decisions.

• Mentor and provide technical guidance to engineers and other technical contributors, raising engineering standards through hands-on leadership rather than formal authority.

Job Requirements : • Strong experience building and operating data pipelines using SQL and Python in a modern cloud environment.

• Deep expertise in SQL, including complex transformations, data modeling, query optimization, and performance tuning at scale.

• 2+ years of recent, hands-on production experience with Databricks, including designing, building, optimizing, and operating production data workloads.

• Strong hands-on experience with Spark/PySpark and distributed data processing in a production environment.

• Strong understanding of modern data architecture patterns, including Lakehouse architecture, ELT, and layered data models (bronze/silver/gold).

• Proven experience designing data models for analytics, including dimensional or domain-oriented approaches.

• Experience driving database and data engineering best practices, including schema design, migrations, and performance optimization.

• Demonstrated ability to own consequential architecture and engineering decisions and drive them from design through production in environments with limited structure or support.

• Demonstrated experience establishing reusable data frameworks, standards, and guardrails that have been successfully adopted beyond an individual project or pipeline.

• Experience owning production data systems, including monitoring, debugging, and resolving data pipeline failures.

• Experience working closely with business stakeholders or analysts to translate ambiguous requirements into scalable data solutions.

• Strong technical judgment in evaluating technologies and architecture patterns based on scalability, reliability, cost, maintainability, governance, and developer experience—not technology novelty alone.

• Experience evolving or migrating legacy data platforms and workloads while maintaining continuity for downstream consumers.

• Strong understanding of data governance, including metadata, lineage, access controls, data quality, and the role of governance in creating trusted, reusable data products.

• Strong record of project execution and completion with experience with agile development practices.

• Excellent written and verbal communication skills, with demonstrated ability to translate complex technical concepts for both technical and non-technical audiences, influence technical direction across teams, and build alignment around architecture and engineering standards.

• Experience with

Similar jobs

• Data Engineer, Senior — Booz Allen Hamilton · Remote
• Data Engineer, Senior — Booz Allen Hamilton · Remote
• Data Engineering Senior Manager — Imprint · New York City
• Data Engineer Senior — The University of Kansas Health System · Shawnee Mission, KS
• Senior Technical Architect, Data Engineering — Slingshotaerospace · Remote
• Data Engineer - Senior Manager — PwC · IL-Chicago | NC-Raleigh | NC-Charlotte
• Data Architect, Lead — Booz Allen Hamilton · Charlottesville, VA | Alexandria, VA
• Data Engineer Senior Analyst — Accenturefederalservices · Arlington, VA