Key Responsibilities:

• Work as a senior developer for the Hadoop enterprise data platform

• Develop components for big data platforms related to data ingestion, storage, transformations, and analytics

• Test data components and ensure they meet requirements

• Ensure the unit of code and design follow reference architecture standards

• Incorporate new and existing technology platforms with defined use

• Collaborate with other developers and architects to enhance the capabilities of the platforms

• Assist in cross-team collaboration and design discussions

• Provide sandbox functionality to users of the datasets to build models for analysis

Requirements:

• 8+ years of overall experience in Information Technology and Systems working on production applications

• 5+ years of experience with Big data distribution and analytics, specifically in Cloudera Ecosystem. Must consist of hands-on experience with a full project life-cycle implementation with the following features:

  • Data volumes in terabytes with vertical and horizontal scalability

  • Structured and unstructured/semi-structured data

  • High speed querying, highly secure system, Metadata management

  • Analytics use cases including in-memory processing

• 5+ years of experience in data modelling, relational and dimensional

• 8+ years of experience in multiple relational database systems (SQL Server, Oracle, MySQL, PostgreSQL)

• Experience with high volume systems

• 5+ years .NET or Java programming experience

• 5+ years scripting experience: Scala/Python

• 5+ years Hive programming experience

• 3+ year Avro/Parquet experience

• 3+ years of Impala experience

• Experience in Cloudera distributed system

• Experience in Financial projects

• Experience in high volume Data migration projects

• Pre-processing Data Lake datasets by using Hive and Pig.

• Designed the file exceptions by using Python programs.

• Developing data ingestion pipelines using Sqoop/NiFi/Kafka

• Design and creation of Data Models in Hive/HBase

• Unit and Integration Testing

• Experience in Teradata to Cloudera Data Platform migration