Senior Data Engineer

Today we are a global team of technologists and thinkers, who help transform the world with the power of software, enabling our customers to be competitive and disruptive in the marketplace through innovative technology solutions.

DESCRIPTION

Our client connects companies, customers, and communities across the UK, delivering a 'one-price-goes-anywhere' universal postal service to over 29 million addresses. As one of the UK's leading companies, the customer is focused on being recognized as the best delivery company in the UK and across Europe.

We are looking for an Experienced Data Engineer (or ETL Developer) who is proficient in building and maintaining data pipelines in GCP.

On current position, you are expected to have a good match (60-70%) with the following technologies:

• ETL or ELT (end to end understanding)
• Apache Nifi
• GCP (Google Cloud Platform): Cloud DataFlow or Apache Beam, Data Composer or Apache Airflow, BigQuery

RESPONSIBILITIES

• Data ingestion with Apache Nifi
• Data Job scheduling and orchestration with Apache Airflow or Google Data Composer
• Data processing and preparation with Apache Beam or Google DataFlow
• Messaging systems based on Pub/Sub or Kafka
• Data processing with Apache Hadoop/Cloudera Hadoop or Google DataProc (to run Spark jobs)

REQUIREMENTS

• Strong knowledge of SQL
• Proficiency in DataFlow
• Expertise in Data Composer or Apache Airflow
• Experience in Apache Beam
• Good understanding of GCP
• Familiarity with DWH solutions, preferably BQ
• Experience in Agile software development
• Good communication skills
• Intermediate or higher English level, both spoken and written (B1+)

BENEFITS

• Competitive compensation depending on experience and skills
• Individual career path in engineering
• Social package - medical insurance, sports
• Unlimited access to LinkedIn learning solutions
• Compensation for sick lists and regular vacations
• English classes with certified English teachers
• Flexible work hours

EVEN MORE EPAM BENEFITS

• We offer the possibility to work on full product lifecycle -from concept to delivery into production
• We offer mentorship program
• We offer guaranteed professional growth through the technology trainings and technology communities inside EPAM
• We are proactive Agile/Scrum/XP practitioners