Principal Data Engineer (ADTECH)

Company Description

We invite a well-rounded Principal Data Engineer with extensive experience delivering production-grade Python software.  

If you have extensive experience and technical leadership skills, as well as the ability to set your own work priorities and act as a resource for colleagues with less experience – this opportunity is for you.  

Ready to become a part of our big idea? 

Job Description

Customer

Our client has superior end-to-end technology, a premium marketplace, and best in-market advisory services that power the world’s advertising businesses of the largest media and entertainment companies. For example, Fox, NBC Universal, Viacom in the USA, Sky, Channel 4, RTE, and Mediaset in Europe. 

Project

We invite you to work with the provider of comprehensive ad platforms for publishers, advertisers, and media buyers. This is a new project in the AdTech domain for a team responsible for creating metrics for advertising campaigns and redesigning the solution from Golang to Python.  

Responsibilities:

• Perform engineering tasks such as data manipulation, ETL/ELT, implementation, information storage and retrieval, data flow, and analysis by using technologies like Python, PySpark, Databricks, and SQL  
• Design new software and web applications, support applications under development, and customize current applications. Develop software update process for existing applications  
• Optimize data sources and processing rules to ensure appropriate data quality for all products, utilizing troubleshooting, design, and development skills along with cross-system technical knowledge 
• Provide technical leadership throughout the design process and guidance with regards to practices, procedures, and techniques  
• Create technical documentation using an understanding of internal business areas and processes as well as working knowledge of database design, data manipulation, ETL, information storage & retrieval, and data flow & analysis 
• Display expertise in knowledge of engineering methodologies, concepts, and skills and their application in the area of a specified engineering specialty 
• Use a combination of several cutting-edge AWS services, 3rd party commercial software, and internally developed software to store, process and deliver near real-time data to a variety of the largest media companies in the world

Qualifications

• 6+ years of hands-on work in the field of Software Development and/or Big Data 
• Production experience with Python and excellent knowledge of PySpark and Databricks 
• Working experience with data ETL and ELT tools 
• Strong skills in SQL and proven experience working with major database platforms 
• Strong knowledge of algorithms and data structures 
• Good understanding of Data Structures and Big O notation 
• Experience in cloud (AWS, Azure, Google cloud) as well as deploying software in a cloud environment 
• Confident dealing with technical and business stakeholders, Development Team, Engineering Manager, and Product Managers 
• Upper-Intermediate level of spoken and written English 

Will be a plus:

• Experience with Go 
• Experience with Terraform 
• Experience with Airflow 
• Experience with Hive 
• Experience with Presto and MySQL