Job Description
Job DescriptionPosition Summary:
The ETL Engineer is responsible for designing, developing, and maintaining data pipelines that extract, transform, and load data into analytics platforms, including SQL Server and ClickHouse for real-time and high-performance querying. This role ensures data integrity, scalability, and performance for BI, reporting, and advanced analytics.
Job Responsibilities:
- ETL Pipeline Development
- Design and implement ETL workflows using SQL Server Integration Services, Azure Data Factory, Databricks, and ClickHouse.
- Develop transformations for structured and semi-structured data; optimize for speed and reliability.
- ClickHouse Integration
- Build and maintain ingestion pipelines for ClickHouse using batch and streaming methods.
- Optimize ClickHouse schemas, partitions, and materialized views for query performance.
- Data Integration
- Connect multiple sources (SQL Server, APIs, cloud storage, Kafka/Event Hubs) into centralized data platforms.
- Performance & Optimization
- Monitor SSIS ETL jobs and SQL Server and ClickHouse queries; tune for low latency and high throughput.
- Implement compression, sharding, and replication strategies in ClickHouse.
- Data Quality & Governance
- Apply validation checks, error handling, and lineage tracking.
- Ensure compliance with security and governance standards.
- Collaboration
- Work with data engineers, analysts, and BI teams to deliver reliable datasets.
- Support real-time analytics and dashboards powered by PowerBI and ClickHouse.
- Documentation
- Maintain clear documentation for ETL processes, SQL Server and ClickHouse schemas, and operational playbooks.
Required Skills:
- Strong experience with ETL tools (Azure Data Factory, SSIS, Databricks).
- Proficiency in SQL and relational database concepts.
- Familiarity with ClickHouse (schema design, ingestion, optimization).
- Familiarity with cloud platforms (Azure preferred).
- Programming skills in Python or Scala for data processing.
Preferred Qualifications
- Experience with streaming data (Kafka, Azure Event Hubs).
- Familiarity with big data frameworks.
- Understanding of DevOps practices for data pipelines.
- Exposure to BI tools (Power BI, Tableau) integrated with ClickHouse.
Education & Experience
- Bachelor’s degree in Computer Science, Data Engineering, or related field.
- 3+ years of experience in ETL development and data integration.
This is an On-Premises position. Monday through Thursday (9-5) and remote on Fridays only (No exceptions).
Powered by JazzHR
YEImEwwyhE