Job Descriptions:

• Design, develop, and maintain data pipelines to extract, transform, and load (ETL) data from various sources into a centralized data warehouse or data lake.
• Integrate data from different sources, such as databases, APIs, and third-party applications, ensuring data consistency and accuracy.
• Create and maintain data models and schemas to facilitate data storage and retrieval, following best practices for data warehousing and database management.
• Implement data quality checks and validation processes to ensure data accuracy, completeness, and consistency.
• Optimize data pipelines and systems for performance, scalability, and efficiency, making sure data processing meets business requirements.
• Implement data security measures to protect sensitive information and ensure compliance with data privacy regulations (e.g., GDPR, HIPAA).
• Document data engineering processes, data lineage, and system architecture to facilitate knowledge sharing and future maintenance.
• Set up monitoring and alerting systems to detect and address issues with data pipelines and systems proactively.
• Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand their data needs and provide the necessary infrastructure and support.
• Stay up-to-date with the latest data engineering technologies and tools, and evaluate their applicability to the organization's data stack.

Qualifications

• Bachelor's degree in computer science, information technology, or a related field. A master's degree is a plus.
• Strong proficiency in data engineering tools and technologies, such as SQL, ETL frameworks (e.g., Apache Spark, Apache Airflow, Apache Beam)
• Experience with Google BigQuery for data warehousing.
• Experience with Google Cloud Dataflow or Dataproc
• Experience with programming languages like Python, Java, or Scala.
• Experience with building streaming data pipelines on Google Cloud
• Knowledge of database design, data modeling, and data integration techniques.
• Familiarity with data governance, data security, and compliance standards.
• Problem-solving skills and attention to detail.
• Strong communication and collaboration skills.