Regarding the fact our Client is building a new unit responsible for Database, AMP and Reporting, we are looking for an experienced Hadoop Developer In Warsaw.
Role description: Responsible for Database, AMP and Reporting development in the agile team
Project requires to load collateral data (by Sqoop) to relational database on Oracle, then transform them into improved form by Spark, leave them in Hive tables. Next step is to generate reports through Spark using same data plus include additional data that are already on Hadoop cluste
- Deep understanding of the principles of distributed systems
- Knowledge and experience with: Apache Spark, Apache Hadoop, Scala, Hive, Oozie, Pig, Sqoop, Impala, Kafka
- Experience with NoSQL databases: MongoDB, Cassandra, HBase or similar
- Experience with building ETL/ETL pipelines
- Production experience with performance tuning of Hadoop/Spark solutions
- Familiar with OOD and design patterns
- Experience from large-scale system integration projects
- Working with RDBMS and knowledge of SQL Integration towards RESTful and SOAP/OSB-services
- Knowledge and experience with web servers such as Tomcat, Jetty or Weblogic
- Knowledge and experience with tools like Maven, Git, and the Atlassian suite (Stash/Bitbucket, Jira, Confluence)
- Hands on experience with Redhat, Linux shell scripting
- Agile experience.
- Attractive salary (based on your professional experience and technical skills),
- Flexibility to choose the form of contract (UoP/B2B),
- Private healthcare – LuxMed, Benefit system – Multisport card,
- Dedicated administrative consultant.