Data Pipeline Engineer

Job Description

Job DescriptionSalary:

Position Summary:

CCM's Consolidated Intermodal Technologies Group (CIT) is seeking a highly skilled and detail-oriented Data Pipeline Engineer to join our team. The Data Pipeline Engineer is a crucial role in the CIT organization. The position designs, builds, and maintains systems that process and transport data efficiently. Their primary responsibility is to ensure that data flows seamlessly from & to various sources to storage solutions, enabling stakeholders to access timely and accurate information. By managing the data lifecycle, Data Pipeline Engineers support business objectives by enhancing decision-making processes and improving operational efficiency.

Key duties of a Data Pipeline Engineer include managing data operations, supporting cross-functional teams, and supporting areas such as data analytics. They are tasked with troubleshooting data issues, optimizing existing pipelines, and implementing new technologies to improve data processing capabilities. Their work not only ensures the integrity and availability of data but also drives the overall success of data-driven initiatives within the company.

The role is responsible for the ingestion, transformation, loading and distribution of various types of data. This requires knowledge of API standards, EDI protocols, Data pipeline tools and experience using commercially available ETL tools for processing ANSI 301, 310, and 322 message sets. Candidates familiar with data integration to and from tracking systems, terminal operations systems, dispatch and drayage systems.

This position is located on-site in Rockaway, NJ.

Responsibilities:

• Develop and maintain scalable data pipelines to support data processing and transformation
• Collaborate with cross-functional teams to gather requirements and design and document data solutions
• Monitor and optimize existing data pipelines for performance and reliability
• Ensure data quality and integrity through rigorous testing and validation processes
• Implement and manage ETL processes to extract, transform, and load data from various sources
• Document data pipeline architecture, processes, and workflows for future reference
• Stay updated with the latest industry trends and technologies related to data engineering
  

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Business, or a related field (or equivalent work experience).
• 5+ years of experience working in the data pipeline field or in a similar role.
• Proven experience with EDI protocols (e.g., X12, EDIFACT) and integration tools.
• Strong knowledge of SQL and experience with relational databases.
• Proficient in programming languages such as PowerShell, Python, Java, or Scala.
• Experience with data pipeline tools (e.g., Cleo, Informatica, Talend, or Microsoft SSIS) and frameworks
• Excellent problem-solving skills and attention to detail.
• Ability to work independently and collaboratively in a team environment and communicate effectively
• Experience with version control, debugging, and data validation techniques.
• Strong analytical and problem-solving abilities
• Excellent communication skills, both written and verbal.
• Detail-oriented with a strong focus on data accuracy and process improvement.

Benefits:

• HealthCare, Dental, and Vision Insurance
• Life, Short Term & Long Term Disability Insurance
• 401(k) with Company Contribution
• Paid holidays and vacation
  
• Professional development and growth opportunities
• Collaborative and innovative work environment

About CCM

At Consolidated Chassis Management, LLC, our mission is to deliver unparalleled intermodal equipment and fleet management solutions. We are dedicated to optimizing chassis and fleet operations through innovative, reliable, and efficient services. By prioritizing customer satisfaction and operational excellence, we aim to enhance the logistics and transportation industry, driving value for our customers and stakeholders.