Bigdata Engineer Bigdata Engineer …

Comtel Solutions Pte Ltd
in Singapore
Contract, Full time
Last application, 12 Jan 21
90-120k
Comtel Solutions Pte Ltd
in Singapore
Contract, Full time
Last application, 12 Jan 21
90-120k
Posted by:
Sasi Comtel • Delivery Manager
Posted by:
Sasi Comtel
Delivery Manager
The position exists to continuously enhance the efficiency of existing Data pipeline Job/framework and setup a best practice for Big Data projects planned on the BI & Analytics platform.

Responsibilities:

  • Review and tune the code for the Spark framework to increase the throughput.
  • Establish best practices and guidelines to be followed by tech teams while writing codes in Spark.
  • Create a framework to continuously monitor the performance and identify the resource intensive jobs.
  • Able to write code, services and components in Java, Apache Spark and Hadoop.
  • Responsible for systems analysis Design, Coding, Unit testing, CICD and other SDLC activities.
  • Proven experience working with Apache Spark streaming and batch framework.
  • Proven experience in performance tuning Java, Spark based applications is a must.
  • Knowledge of working with different file formats like Parquet , JSON and AVRO.
  • Data Warehouse experience working with RDBMS and Hadoop,  Well versed with concepts of change data capture and implementation.
  • Hands-on experience in writing codes to efficiently handle files on Hadoop.
  • Ability to work in projects following Micro Services based architecture.
  • Knowledge of S3 will be a plus.
  • Ability to work proactively, independently and with global teams.
  • Strong communication skills should be able to communicate effectively with the stakeholders and present the outcome of analysis.
  • Working experience in projects following Agile methodology is preferred.

Skills:

  • At least 3 to 9 years of working experience, preferably in banking environments
  • Expert knowledge of technologies e.g. Hadoop, Hive, Presto, Spark, Java, RDBMS like Teradata and File storage like S3 are necessary.
  • Candidate must speak and write well.
Comtel Solutions Pte Ltd logo
More Jobs Like This
See more jobs
Close
Loading...
Loading...