Please Accept our Privacy Policy
Location: USA
\nExperience: 8 Years (Preferred)
\n
We are looking for an experienced Databricks Data Engineer with strong expertise in Apache Spark, PySpark, Azure data engineering, and large-scale data pipeline development.
\n\nThe ideal candidate will have hands-on experience building and supporting batch and real-time data pipelines, working with distributed computing frameworks, and optimizing data processing across large-scale datasets.
\n\nTop Required Skills\n\nApache Spark / PySpark – Strong hands-on expertise using Jupyter Notebooks, Google Colab, or Databricks (preferred).
\nAzure Data Engineering – Hands-on experience in data pipeline development and data ingestion patterns.
\nData Orchestration & SQL – Experience with Azure Data Factory (ADF) or Apache Airflow, along with strong SQL skills.
\nData Modeling – Experience with denormalized data modeling for large-scale big data systems.
\nDesign, develop, and support batch-based, distributed, and real-time data pipelines.
\nBuild highly available, fault-tolerant, self-healing, and adaptive data processing solutions.
\nDevelop scalable data engineering solutions using Apache Spark and PySpark.
\nImplement data ingestion patterns and pipelines within Azure environments.
\nDesign and manage workflow orchestration using Azure Data Factory (ADF) or Apache Airflow.
\nOptimize queries processing billions of rows across distributed query engines.
\nPerform root cause analysis and implement permanent solutions for technical and business process issues.
\nProvide technical consultation and lead the implementation of complex data engineering projects.
\nDevelop and maintain technical documentation for assigned systems and projects.
\nEnsure data accuracy, reliability, scalability, and performance across data platforms.
\nStrong hands-on experience with Apache Spark and PySpark.
\nExperience working with Jupyter Notebooks, Google Colab, or Databricks (Databricks preferred).
\nHands-on experience developing data pipelines and implementing data ingestion patterns in Azure.
\nExperience with Azure Data Factory (ADF) or Apache Airflow.
\nStrong SQL development and query optimization skills.
\nExperience with denormalized data modeling for big data systems.
\nUnderstanding of distributed computing, batch processing, and real-time data processing.
\nExperience handling large-scale datasets and distributed query engines.
\nStrong problem-solving, analytical, and communication skills.
\n