Lead Data Engineer
Salary: Up to $160,000 Location:
On-Site | South Los Angeles Area
Employment Type: Full-Time
Build the Foundation for Enterprise Analytics
We are seeking a Lead Data Engineer to drive the architecture, modernization, and expansion of a growing enterprise data platform built within the Microsoft Azure ecosystem. This is a highly visible technical leadership role where you'll help shape the future of data across multiple business functions by building scalable, secure, and high-performing data solutions.
If you're passionate about Databricks, cloud data platforms, and mentoring engineering teams while remaining hands-on with development, this is an opportunity to make a significant impact.
What You'll Do
- Lead the design, implementation, and optimization of an enterprise Azure Databricks Lakehouse environment utilizing Bronze, Silver, and Gold data layers.
- Architect and execute migrations from SQL Server and legacy platforms into modern cloud-native data solutions.
- Develop and optimize large-scale ETL/ELT pipelines supporting operational reporting, business intelligence, advanced analytics, and data science initiatives.
- Integrate data from multiple enterprise systems, APIs, operational platforms, and third-party sources.
- Establish best practices for data engineering, data modeling, testing, CI/CD, monitoring, governance, and production support.
- Implement and maintain data governance frameworks, security controls, lineage tracking, and access management.
- Lead code reviews, technical design discussions, and engineering standards across the data team.
- Mentor and support data engineers while fostering a culture of technical excellence.
- Partner closely with business stakeholders, analysts, and data scientists to deliver trusted, scalable data products.
For immediate consideration, apply now and direct message Reid Gormly on LINKEDIN
Required Qualifications
- 5+ years of Data Engineering experience.
- 3+ years of hands-on production experience with Databricks and Apache Spark.
- Strong technical leadership experience, including architecture ownership and mentoring engineers.
- Advanced proficiency in Python, PySpark, and SQL.
- Experience with:
- Azure Databricks
- Azure Data Lake Storage (ADLS Gen2)
- Delta Lake
- Databricks Workflows or Lakeflow Jobs
- Strong understanding of medallion architecture and modern data lakehouse design.
- Experience migrating enterprise workloads from SQL Server or similar database platforms to the cloud.
- Expertise with data quality frameworks, monitoring, alerting, reconciliation processes, and disaster recovery strategies.
- Experience with Git-based development workflows, automated testing, and CI/CD deployment pipelines.
- Excellent communication skills with the ability to explain technical decisions to both technical and non-technical audiences.
