Job Description
Job DescriptionResponsibilities:
- Build and maintain data pipelines using Python and SQL.
- Create and manage workflows in Databricks and migrate existing Azure Data Factory pipelines.
- Load structured and semi-structured data into the Data Lakehouse (Delta Lake).
- Write reusable and well-documented data transformations in Databricks.
- Monitor pipeline performance and fix issues to meet service goals.
- Work with analytics, engineering, and business teams to deliver clean, ready-to-use datasets.
- Optimize Databricks views so BI tools like PowerBI run efficiently.
- Help manage metadata and maintain documentation.
Take part in discussions about data platform architecture and improvements.
Qualifications:
- Bachelor’s degree in computer science, Engineering, IT, or related field, plus 5+ years’ experience (or
- equivalent).
- Strong knowledge of databases, data modeling (star/snowflake), and data engineering best practices.
- Hands-on experience with Azure Data Lake Storage, Databricks, Unity Catalogs, GitHub, and Azure tools.
- Skilled in Python, SQL, R, or Scala.
- Good project management skills; able to prioritize tasks and deliver improvements on time.
- Experienced in building ELT pipelines and working with modern data stacks.
- Understanding of Data Lakehouse, Delta Lake, and columnar storage.
- Experience with data governance, dbt, or CI/CD environments is a plus.
- Industry knowledge in construction is a bonus.
- Strong analytical, problem-solving, and teamwork skills.
- Clear communication, able to explain technical concepts to non-technical audiences.
- Focused on improving processes and learning new technologies.
- Proficient in Microsoft Office and collaboration tools.
- Limited travel required.