Job Description
Job Description
We are seeking a Senior Data Engineer to maintain legacy ETL processes and lead the development of modern, cloud-native data pipelines on Microsoft Azure. This role involves working closely with stakeholders to design, implement, and optimize data solutions for analytics, reporting, and AI/ML initiatives.
Key Responsibilities:
- Maintain and operate legacy ETL processes using Microsoft SSIS, PowerShell, SQL procedures, SSAS, .NET code
- Design and implement Azure cloud-native data pipelines using Azure Data Factory, Synapse Pipelines, Apache Spark Notebooks, Python, and SQL
- Redevelop existing SSIS ETL scripts into Azure Data Factory and Synapse Pipelines
- Support data modeling for relational, dimensional, data lakehouse (medallion architecture), data warehouse, and NoSQL environments
- Implement data migration, integrity, quality, metadata management, and security measures
- Monitor and troubleshoot data pipelines to ensure high availability and performance
- Automate platform operations with governance, build, deployment, and monitoring processes
- Actively participate in Agile DevOps practices, including PI planning
- Maintain strict versioning and configuration control for data integrity
Required Qualifications:
- BS in Computer Science or related field with 8+ years of experience OR Master’s with 6+ years
- 4+ years of experience developing and maintaining ETL processes using SSIS, SQL, SQL Agent, PowerShell
- 4+ years of experience with scripting languages: SQL, T-SQL, Python, PySpark
- Experience with Microsoft BI tools: SQL Server, SSIS, SSRS, SSAS (cubes)
- Proven experience designing and building ETL/data engineering solutions using Azure Data Lake Services, Azure Synapse
•Preferred Skills:
- Experience with data lakehouse architectures and medallion design
- Familiarity with AI/ML data preparation and advanced analytics workflows
- Strong understanding of data governance and security best practices