Job Description
We are looking for an experienced Data Engineer to help advance a modern cloud-based data environment in St. Paul, Minnesota. This Long-term Contract position focuses on creating reliable, scalable data solutions that support analytics, business intelligence, and AI-driven initiatives across the organization. The ideal candidate brings strong Databricks expertise and a practical approach to building high-quality data products that enable confident decision-making.
Responsibilities:
• Build and enhance enterprise data pipelines in Databricks to support dependable, large-scale data processing and delivery.
• Create and refine ETL workflows using Spark, PySpark, and cloud technologies to improve efficiency, stability, and performance.
• Design Lakehouse-based data architectures that serve reporting, advanced analytics, and artificial intelligence use cases.
• Produce clean, validated, and business-ready datasets that strengthen self-service reporting and downstream analytical applications.
• Monitor and improve platform performance, data quality, scalability, and operational reliability across data engineering solutions.
• Collaborate with engineering, analytics, BI, and business stakeholders to translate requirements into effective data products.
• Provide technical guidance to other engineers and promote sound practices for modern data engineering development.
• Support the evolution of the cloud data platform through continuous optimization, automation, and maintainable solution design.• At least 7 years of hands-on experience in data engineering, including substantial work with Databricks.
• Advanced knowledge of Apache Spark, Delta Lake, and Lakehouse concepts for enterprise data solution development.
• Strong programming ability in Python and PySpark, along with solid experience in data modeling and transformation design.
• Proven background in building and tuning high-volume data pipelines in cloud-based environments.
• Good understanding of data warehousing principles, governance practices, and performance optimization techniques.
• Experience with platforms such as Azure, Microsoft Fabric, Data Factory, or similar cloud data services.
• Familiarity with technologies such as Apache Hadoop and Apache Kafka to support distributed data processing needs.
• Strong analytical thinking, communication skills, and the ability to work effectively with technical and business teams.