Big Data Engineer, Multinational Financial Services

DataArt's specialists participate in data governance and data improvement processes for one of the top three UK insurance companies. Our team helps the client in finding and clearing issues with the data.

The Big Data engineer will be responsible for creating applications and solutions that will manage data flow. The specialist will also participate in the creation of new data pipelines, identify existing data gaps, and offer automated solutions for providing advanced analytical capabilities and enriching data with application APIs.

Required Skills and Experience

• Strong knowledge of Big Data general concepts
• Hands-on experience with the Hadoop ecosystem (Spark, MapReduce, Hive, etc.)
• Knowledge of standard data structures and algorithms
• Experience handling large data using streaming
• Experience with tools like Hive, Spark, HBase, Sqoop, Impala, Kafka, Flume, Oozie, MapReduce, Indexers, etc.
• Hands-on programming experience in Java, Scala or Python
• Experience with end-to-end design and build processes for Near-Real-Time and Batch Data Pipelines
• Strong experience with SQL and Data modeling
• Analytical thinking
• Spoken English

Additional Competencies

• Experience working in an agile environment and understanding of various phases of the software development life cycle
• Skills working with the Cloudera platform
• Skills using source code and version control systems like SVN, Git, etc.
• Knowledge of Hadoop architecture components
• Ability to work with minimal supervision
• Understand the current state and identify and recommend possible solutions
• Natural curiosity and a desire to explore solutions in a highly focused team

DataArt offers:

• Professional Development:
  - Experienced colleagues who are ready to share knowledge;
  - The ability to switch projects, technology stacks, try yourself in different roles;
  - More than 150 workplaces for advanced training;
  - Study and practice of English: courses and communication with colleagues and clients from different countries;
  - Support of speakers who make presentations at conferences and meetings of technology communities.

• The ability to focus on your work: a lack of bureaucracy and micromanagement, and convenient corporate services;

• Friendly atmosphere, concern for the comfort of specialists;

• Flexible schedule and the ability to work remotely;

• The ability to work in any of our development centers.