Data Engineer / Databricks Specialist
Job Description
Job Description
Company Overview:
Over the past 15 years, eTel has delivered essential solutions for the federal government by securing and managing data, providing scalable identity access, modernizing legacy systems, and building high-performance platforms. By integrating new technologies and ensuring reliable operations we help agencies stay prepared for future challenges As a premier technology solutions and services company to the US federal government, eTel possesses longstanding relationships across the federal civilian marketplace. Other customers include the broader Treasury Department, Commerce Department, and State Department.
eTel offers integrated CMMI Level 3 processes, tools, and techniques with innovative, cost-efficient, and secure solutions to address complex challenges. eTel also holds ISO 9001:2015, ISO/IEC 27001:2013, and ISO/IEC 20000-1:2018 certifications, and offers dedicated subject matter experts (SMEs) and thought leaders that possess a deep understanding of customers' environments and challenges.
Security Clearance: IRS MBI clearance (MUST already be in place)
Position Summary:
The IRS is modernizing its data platform to support analytics, reporting, and predictive modelling. We are seeking a skilled Databricks Specialist / Data Engineer to design, implement, and optimize big data pipelines and analytics solutions. The ideal candidate will have hands-on experience with Databricks, Apache Spark, and cloud-based data platforms, and a strong understanding of data governance, security, and compliance standards relevant to federal environments.
Key Responsibilities:
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Apache Spark.
- Work with large datasets from multiple sources, including structured and unstructured data, to support analytics and reporting.
- Collaborate with IRS teams to implement data lakehouse solutions and optimize data storage and retrieval.
- Develop and maintain Delta Lake tables and pipelines to ensure data reliability and ACID compliance.
- Implement data transformations, cleansing, and enrichment processes for downstream analytics and reporting.
- Work with Python, SQL, and other scripting languages to build and automate workflows.
- Ensure compliance with IRS security standards, NIST 800-53, and IRS 1075 for handling sensitive taxpayer data.
- Collaborate with data scientists, analysts, and other stakeholders to deliver actionable insights.
- Participate in performance tuning, troubleshooting, and optimization of Databricks clusters and workloads.
Required Qualifications:
- Hands-on experience with Databricks and Apache Spark in production environments.
- Strong data engineering skills: ETL/ELT pipelines, data modeling, and analytics.
- Proficient in Python, SQL, or Scala for data processing.
- Experience with cloud platforms such as AWS, Azure, or GCP, including cloud storage, compute, and security controls.
- Knowledge of Delta Lake, data lake architecture, and big data best practices.
- Experience working in federal or highly regulated environments preferred.
Desired Skills / Nice to Have:
- Familiarity with data governance, metadata management, and lineage tracking.
- Experience integrating Databricks with BI tools like Power BI or Tableau.
- Knowledge of machine learning workflows in Databricks.
- Understanding of IRS modernization initiatives and taxpayer data sensitivity.
Education / Certifications:
- Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
- Relevant certifications are a plus: Databricks Certified Data Engineer | AWS / Azure / GCP Cloud Certifications | Certified Analytics Professional (CAP)
Location: Remote
Commitment to Diversity -
eTelligent Group provides equal employment opportunities (EEO) to all applicants without regard to race, color, religion, gender, sexual orientation, gender identity, nations origin, age, disability, genetic information, marital status, amnesty, status as a covered veteran, and any other characteristic provided in accordance with applicable, federal, state and local laws.