Job Description
Job Description
Professional Search Group (PSG) has teamed up with a world class media and entertainment company to find an experienced Sr Data Engineer to join the team, where technology meets storytelling. This is a well-established, US based business that has a long-standing partnership with Professional Search Group. We are looking for someone who is passionate about shaping the future of entertainment through innovation. Are you ready to work your magic with one of the world’s most iconic brands?
Our client is the leading premium streaming service offering live and on-demand TV and movies, with and without commercials, both in and outside the home and they are in search of a Senior Data Engineer on a contract basis. As a member of the Product Performance team, you will work on building foundational datasets from clickstream and quality of service telemetry data – enabling dozens of engineering and analytical teams to unlock the power of data to drive key business decisions and provide engineering, analytics, and operational teams the critical information necessary to scale the largest streaming service. The Product Performance Data Solutions team is seeking to grow their team of world-class Data Engineers that share their charisma and enthusiasm for making a positive impact.
Responsibilities:
• Contribute to maintaining, updating, and expanding existing data pipelines in Python / Spark while maintaining strict uptime SLAs
• Architect, design, and code shared libraries in Scala and Python that abstract complex business logic to allow consistent functionality across all data pipelines
• Tech stack includes Airflow, Spark, Databricks, Delta Lake, Snowflake, Scala, Python
• Collaborate with product managers, architects, and other engineers to drive the success of the Product Performance Data and key business stakeholders
• Contribute to developing and documenting both internal and external standards for pipeline configurations, naming conventions, partitioning strategies, and more
• Ensure high operational efficiency and quality of datasets to ensure our solutions meet SLAs and project reliability and accuracy to all our partners (Engineering, Data Science, Operations, and Analytics teams)
• Be an active participant and advocate of agile/scrum ceremonies to collaborate and improve processes for our team
• Engage with and understand our customers, forming relationships that allow us to understand and prioritize both innovative new offerings and incremental platform improvements
• Maintain detailed documentation of your work and changes to support data quality and data governance requirements
• At least 5 years of data engineering experience developing large data pipelines
• Strong algorithmic problem-solving expertise
• Strong fundamental Python programming skills
• Basic understanding of AWS or other cloud provider resources (S3)
• Strong SQL skills and ability to create queries to analyze complex datasets
• Hands-on production environment experience with distributed processing systems such as Spark
• Hands-on production experience with data pipeline orchestration systems such as Airflow for creating and maintaining data pipelines
• Some scripting language experience
• Willingness and ability to learn and pick up new skills
• Self-starting problem solver with an eye for detail and excellent analytical and communication skills
Company DescriptionProfessional Search Group was originally founded in Australia in 2013. Expanding to the US in 2015, PSG specializes in the recruitment of Finance & Accounting, Information Technology, Salesforce as well as Administrative and Operations professionals across all industries on a temporary, contract and direct hire basis.
Taking a page from our Australian roots, our vision is to change the way people think about working with, and working for, a recruiting company. Adhering to the Aussie easy-going mindset, blending the perfect balance between efficient productive work-lives and fun-filled social lives, allows us to provide a refreshing and personalized experience for our clients, candidates as well as internal staff! With the right culture and attitude in place, we aspire to become a trusted recruitment partner for sought after employers, a trusted resource for top talent, and a firm where the best recruiters aspire to work.
Company Description
Professional Search Group was originally founded in Australia in 2013. Expanding to the US in 2015, PSG specializes in the recruitment of Finance & Accounting, Information Technology, Salesforce as well as Administrative and Operations professionals across all industries on a temporary, contract and direct hire basis. \r\n\r\nTaking a page from our Australian roots, our vision is to change the way people think about working with, and working for, a recruiting company. Adhering to the Aussie easy-going mindset, blending the perfect balance between efficient productive work-lives and fun-filled social lives, allows us to provide a refreshing and personalized experience for our clients, candidates as well as internal staff! With the right culture and attitude in place, we aspire to become a trusted recruitment partner for sought after employers, a trusted resource for top talent, and a firm where the best recruiters aspire to work.