Data Engineer, Data and AI Office (DAO) - 2 Two-Year Contract

What the role is

This role is responsible for data infrastructure and data pipeline development, collaborating with system architects or senior engineers to design and implement suitable data architecture, schema, and data storage solutions that support business requirements.

The role works with vendors and stakeholders on secure API integrations, including API specifications and documentation, to integrate data from multiple sources, and creates end-to-end data pipelines by engaging with cross-functional teams such as analysts and data scientists to understand business requirements, ensuring data is properly formatted and accessible for analysis and reporting.

What you will be working on

Data Infrastructure and Data Pipeline Development

Collaborate with system architect or senior engineers to design and implement the suitable data architecture, schema and data storage solutions that support business requirements. Work with vendor and stakeholders on secure API integrations, including API specification and documentations to integrate data from multiple sources.

Create end-to-end data pipelines from various sources, engaging with cross-functional teams including analysts and data scientists to understand business requirements throughout the development process, ensuring it's properly formatted and accessible for analysis and reporting.

Performance Optimisation

Optimise system and pipeline performance through ongoing monitoring and implementation of scalable solutions designed to handle expanding data volumes and growing organisational demands.

Data Quality and Governance

Implement data quality checks and monitoring systems to ensure accuracy, completeness, and consistency of data at various stages of the data flow, ingestion and processing. Establish and maintain data governance practices and documentation. This includes troubleshooting issues when they arise.

Collaboration & Support

Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet their needs. Provide technical support and guidance on data-related matters.

What we are looking for

1. Demonstrated expertise in data engineering principles and scalable pipeline architecture.

2. Required technical skills encompass advanced Python and SQL programming, cloud platform experience, and working knowledge of big data tools and frameworks.

3. Familiar with cloud technologies such as AWS infrastructure and native AWS tools (CloudWatch and Glue).

4. Familiar with data modelling, data access, and data storage infrastructure like Data Lake, Data Virtualisation and Data Warehouse for efficient storage and retrieval.

5. Familiar with rest API and web requests/protocols in general.

6. Able to apply AI to build of data pipelines and perform data quality related checks.

7. Capable of operating independently whilst collaborating effectively across diverse functional teams.

8. Strong analytical thinking and problem-solving competencies.

9. Excellent communication skills, both verbal and written.

10. Minimum 2 years of relevant experience required.

#LI-CSCNJ

About Civil Service College

The Civil Service College (CSC) is a statutory board under the Public Service Division, Prime Minister’s Office. We are the central learning institution for the Singapore Public Service and we partner government ministries and agencies to develop people for a first-class public service.

CSC brings together public officers from diverse backgrounds and provides them with opportunities for learning and skills upgrading. Our service-wide networks enable public officers to build capabilities, develop shared ethos and gain deeper perspectives, creating a collaborative environment for dialogue and knowledge sharing.

Through our programmes and services, CSC aims to strengthen strategic capacity in governance, leadership, public administration and management for the Singapore Public Service.