Senior Big Data Engineer

We are looking for a Big Data expert, who understands what to do with one Petabyte of data, is able to intelligently design the right architecture, and kick it off in the full-scale production.

We are starting a new big Healthcare project for a California-based electronic health record provider. It is a great opportunity for a Senior Big Data Engineer to join us and lead this challenge.

The client is a big EHR provider on the market of California and is expanding to other states. We received a unique chance to redesign their application. Our goal is to create a solution to handle Terabytes of data simultaneously on a daily basis, with vertical and horizontal scalability. At the same time, the solution will provide data processing and analysis features.

The application will have a wide range of data analysis features. It will allow receiving data from different sources and processing it to enable fast decision-making at any business level, ready access to financial information (such as revenues), transparency of any issues related to data, easy passing through government audits, and much more.

Responsibilities:

• Leading the team of developers during creation of the new solution
• Designing, reviewing, implementing, testing, and optimizing ETL (batch and real-time) processes
• Designing and developing big data applications while migrating to the new technology stack
• Sharing technical knowledge with the Teammates and Customer
• Troubleshooting data and/or system issues

Requirements:

• At least 3 years of development experience with Scala and Java or Python
• Strong experience with Spark and its streaming from Kafka
• Strong knowledge of the Hadoop Ecosystem
• Experience in Data Warehousing/RDBMS technologies and back-end reporting systems
• ETL, data parsing, have an experience of working with different data types (Excel, EDI, CSV, etc.)
• Strong scripting skills to perform data/file manipulation
• Experience with automated testing practices
• Low-level performance optimization experience
• Good knowledge of MS Azure or AWS-related service
• At least Intermediate level of English Will be a plus:

Will be a plus:

• Experience with Druid 
• Experience in Microstrategy
• OLAP experience