Senior Data Engineer

About the Transport Exchange Group

Transport Exchange Group provide best-in-class software solutions for the transport industry. Based in West London we provide the UK's largest freight exchanges to circa 6000 members and counting.

What would you be doing?

• You will join our Development team
• You will help us to migrate from a monolithic architecture to microservices
• Use Domain Driven Design (DDD) to split our main database in smaller, domain oriented, databases
• Identify the best DBMS to be used in the specific context: documental, column oriented, graph, relational, etc.
• You will extract data from sources, minding various data formats and protocols
• Assist in architecting systems and applications
• Be an active contributing part of an agile development team
• You will be responsible for the creation of data models
• Support developers with the creation of tables, views, materialized views, indexes, constraints
• Store, cleanse and curate, land the extracted data into a format or repository for further use
• Expose data to analytical applications for business intelligence, predictive analytics, or machine learning
• Create our big data infrastructure
• Develop or assist in developing products analysis, whether it's a simple spreadsheet reports, to simple webforms, to full featured web or mobile applications
• Benefit from working in a multi-disciplinary team, that uses modern software and product development techniques, and values continuous improvement

What we are expecting from you?

• 5+ years of experience with dimensional data modeling and schema design in Data Warehouses
• Outstanding knowledge of SQL (and DDL)
• 2+ years of with Python, the knowledge of Java is a plus
• 2+ years of experience in custom or structured ETL design, implementation and maintenance
• Demonstrable experience working with MapReduce
• Knowledge of Hadoop ecosystem (HBase, Hive, MapReduce, Zookeeper, PIG, etc.)
• Proven ability to configure and use streaming processing platforms like Spark, Storm, Kafka Streams, Samza, and in particular Apache Flink
• Solid knowledge of Domain Driven Design
• Knowledge of both Relational and NoSQL DBMS
• Basic Machine Learning familiarity is a plus
• You are able to communicate clearly and accurately about technical topics
• Good spoken/written communication in English

We offer:

• A decent level of salary
• Continuous professional competencies development and professional growth opportunities
• Flexible schedule (40h per week)
• Paid trainings
• Paid leave 18 days / Paid sick leave
• Comfortable working place (2 monitors, productive tools)
• Very warm and friendly working environment
• Foreign languages learning in the company office (different levels of language skills; etc.)
• Corporate/ team/ special events
• We always encourage candidates from different locations to apply for our open positions. People who join the team and need to relocate from different regions, except Dnepr, are eligible to relocation bonus Challenging work in international professional environment