Job Description
Job DescriptionAre you passionate about data engineering, cloud technologies, and building high-quality scalable solutions?
We are assembling a team of talented engineers to develop the next generation of the Federal Student Aid system. This high-profile initiative will leverage advanced analytics, big data, and machine learning to serve millions of students and families. Our platform is cloud-native, built on modern microservices, and powered by Databricks to enable highly performant, scalable data processing and analytics.
Ideal candidates will have strong data engineering skills with experience in Databricks, Apache Spark, Delta Lake, and cloud platforms (AWS, Azure, or GCP).
Note: U.S. citizenship is required for this role.
Basic Qualifications:
- 4+ years of experience in IT or software engineering
- 3+ years of hands-on experience with Databricks or Apache Spark (PySpark, Scala, or SQL)
- 2+ years of experience building scalable ETL/ELT pipelines with Delta Lake or similar technologies
- Strong knowledge of cloud environments (Azure Databricks, AWS EMR/Databricks, or GCP Dataproc/Databricks)
- Experience with distributed computing and large-scale data processing
- BA or BS degree in Computer Science, Data Engineering, or related field
- Ability to obtain a security clearance required
Additional Qualifications:
- Experience with MLflow, feature stores, or machine learning pipelines in Databricks
- Knowledge of streaming data frameworks (Structured Streaming, Kafka, Kinesis)
- Familiarity with CI/CD and DevOps practices for data engineering (Git, Azure DevOps, Jenkins)
- Ability to optimize Spark jobs for performance and cost efficiency
- Strong problem-solving skills and ability to communicate effectively with technical and non-technical stakeholders
- Ability to work independently and in a collaborative team environment
- Excellent written and verbal communication skills
- U.S. Citizenship required