Data Engineer (Azure Databricks)- Greenfield Project
Job Description
Job Description
Senior Data Engineer
\n
We are seeking a handsgineer-on Senior Data Engineer to build a greenfield, AI-ready enterprise data platform on Azure. Working alongside Data Architects, you will spend 90%+ of your time developing scalable pipelines in Azure Databricks and Microsoft Fabric to power enterprise reporting, analytics, and GenAI/RAG initiatives.
\n
What You'll Do
\n
• Build and optimize batch and real-time pipelines in Azure Databricks (90%+ focus) using SQL, Python, PySpark, and Delta Lake, utilizing Microsoft Fabric for ingestion.
\n
• Implement Lakehouse and medallion architectures, migrating legacy Data Warehouse workloads and integrating ERP/CRM/HR sources (Dynamics 365 F&O/CE, SAP, legacy ERPs, Workday, Entra).
\n
• Develop dimensional and Lakehouse data models with built-in data quality, lineage, and observability.
\n
• Support data governance and MDM via Databricks Unity Catalog, Microsoft Purview, and enterprise catalog/MDM platforms.
\n
• Lead engineering standards, code reviews, CI/CD practices, and mentor onsite/offshore engineers (individual contributor role).
\n
What We're Looking For
\n
• Experience: Minimum 5 years in cloud data engineering (7+ years optimal) and a Bachelor's degree or equivalent experience.
\n
• Core Stack: Deep expertise in Azure Databricks (Lakeflow, Spark, Delta) plus hands-on Microsoft Fabric, ADF, Azure Synapse, ADLS, Azure SQL, and Azure DevOps.
\n
• Code & Modeling: Advanced SQL, expert Python, Spark/Pandas, ETL/ELT, and dimensional/Lakehouse modeling.
\n
• AI & Modern Data: Experience building AI/ML, GenAI, and RAG-ready datasets, including familiarity with MCP servers, data guidance, and AI-assisted data engineering.
\n
• Delivery: Strong debugging skills and ability to drive priorities in Agile/DevOps teams.
\n
Nice to Have
\n
• Manufacturing industry experience in complex data environments.
\n
• Direct integration experience with Dynamics 365 (F&O/CE) or similar ERP/CRM platforms.
\n
• Familiarity with enterprise data catalogs and MDM platforms.
\n
Additional Details
\n
• Location: St. Louis, MO preferred (hybrid, 3 days/week onsite); remote/flexible hybrid considered.
\n
• Employment Type: Contract-to-Hire (strong intent to convert; direct hire considered for exceptional candidates).
\n
• Team & Process: 4-engineer greenfield team (alongside catalog/MDM engineers). Interview process: Lead Solutions Architect screen followed by hiring manager interview.
\n