Data Engineers are responsible for aiding UK Health in developing a cleaned, validated, and discoverable data foundation to enable new analytical workflows and applications.
They will need to work closely with UK Health subject matter experts to scope new data integrations, design a data model, and perform data cleansing and transformation within the data analytics platform.
Core activities
Establish new integrations into the data foundation.
- The engineer will get to work with ontology experts and UK Health subject matter experts to map out a data model.
- Data cleansing and transformation as defined with subject matter experts.
- Configure links to other datasets in the data foundation.
- Work with UK Health QA analysts and subject matter experts to test and validate the output datasets.
- Set up schedules to regularly update the data in analytics platform.
- Configure monitoring and data health on output datasets.
- Liaise with UK Health business analysts to produce documentation on output datasets to aid understanding of pipelines and the data model.
Provide support to analytical users of the data foundation.
- Once data is integrated into the data foundation, it is important that support is provided to UK Health analytical users to create new analytical workflows.
- The engineer will supply advice to analytical users on how they can access and utilise the new datasets.
Qualities
- Comfortable with Python - ideally experience with Apache Spark and Pyspark
- Previous data analytics software experience
- Able to scope new integrations and translate analytical user needs into technical requirement.
UK based – data analytics system can only be accessed in the UK.