Senior Data Engineer - Active Secret Clearance

Job Description

Job Description

Location: Washington, DC (Hybrid)
Clearance Required: Active Secret
Position Type: Full-Time

We are seeking a highly skilled Senior Data Engineer to support the U.S. Coast Guard Office of Data & Analytics (ODA). The ideal candidate will bring expertise in Databricks, SQL, ETL pipelines, and cloud-based data frameworks to enable enterprise-scale analytics and decision-making. This role will involve designing, building, and orchestrating data pipelines, collaborating with analysts and developers, and implementing CI/CD practices to ensure scalable, reliable, and secure data solutions.

Primary Responsibilities:

• Build, test, and orchestrate data pipelines in Databricks.
• Design and optimize data structures, schemas, and ETL processes.
• Translate business use cases into SQL queries, reports, and dashboards.
• Manage database objects in development environments without impacting production.
• Automate workflows using Databricks Workflows or the Jobs API.
• Integrate code into GitHub/GitLab and support CI/CD practices.
• Diagnose data gaps and quality issues, and design test cases for validation.
• Collaborate with developers and analysts to synchronize code and ensure reliability.

Requirements

Minimum Qualifications:

• Active Secret clearance (required at time of application).
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
• 7+ years of professional experience in data engineering.
• Expert-level proficiency in Databricks, Apache Spark, and SQL.
• Experience with ETL tools such as Kafka, Airflow, or AWS Glue.
• Proficiency with CI/CD pipelines and GitHub/GitLab version control.
• Knowledge of cloud platforms (AWS, Azure, GCP).
• Excellent problem-solving, debugging, and collaboration skills.

Preferred Qualifications:

• Experience supporting federal or mission-critical analytics programs.
• Knowledge of data governance and metadata management practices.
• Familiarity with containerization or orchestration tools (Docker, Kubernetes).

Eligibility:

• Must be legally authorized to work in the United States without employer sponsorship, now or in the future.
• Active Secret clearance required for this role.

Benefits

Salary: Competitive, commensurate with experience.