Job overview:

Design and implement the methods for storing and retrieving the data and monitoring the data pipelines, starting from the ingestion of raw data sources, transforming, cleaning and storing, enrich the data for promptly using for both of structured and unstructured data, and working with the data lake and cloud which is the big data technology.

Responsibilities:

• Develop data pipeline automation using Azure technologies, Databricks and Data Factory.
• Understand data, reports and dashboards requirements, develop data visualization using Power BI, Tableau by working across workstreams to support data requirements including reports and dashboards.
• Analyze and perform data profiling to understand data patterns following Data Quality and Data Management processes.
• Proof of concept and Test solutions of ETL tools for customer relationship management.
• Develop and maintain customer profile data service using Grails framework, Apache Hadoop, Shell script, and Impala-shell.
• Establishes requirements and coordinates production with programmers to control the solution.
• Defines application problems by conferring with users and analyzing procedures and processes.
• Writes documentation such as a technical specification, troubleshooting, and application log to serve as a reference.
  
  Qualifications:
• 3 years+’ experience in big data technology, data engineering, data science, data analytic application system development.
• Have an experience of unstructured data for business Intelligence or computer science would be advantage.
• Java, Groovy, JavaScript, Perl, Shell Script
• Grails Framework, Catalyst Framework, Nodejs
• MySQL, MongoDB, MariaDB, Apache Hadoop, Impala
• Documentation, testing, and maintenance
• Intelli J IDEA, Visual Studio Code, Postman, RoboMongo, MobaXterm, WinSCP
• English communication
• Fast learner, Creativity and Team player