Skip to main content

Tell us whether you accept cookies

We use cookies to collect information about how you use this service. We use this information to make the service work as well as possible and improve this service.

Set cookie preferences

You've accepted all cookies. You can change your cookie settings at any time.

Find apprenticeship training for your apprentice

Beta This is a new service – your feedback will help us to improve it.

  1. Home
  2. Apprenticeship training courses
View shortlist
Apprenticeship training course

Data engineer (level 5)

Build systems that collect, manage, and convert data into usable information for data scientists, data analysts and business intelligence analysts to interpret.

Qualification level
5
Equivalent to higher national diploma (HND).
Typical duration
24 months
Apprenticeship category
Digital
Maximum funding
£19,000
Maximum amount government will fund
for apprenticeship training.
Skills
Skills an apprentice will learn
  • Collate, evaluate and refine user requirements to design the data product.
  • Collate, evaluate and refine business requirements including cost, resourcing, and accessibility to design the data product.
  • Design a data product to serve multiple needs and with scalability, efficiency, and security in mind.
  • Automate data pipelines such as batch, real-time, on demand and other processes using programming languages and data integration platforms with graphical user interfaces.
  • Produce and maintain technical documentation explaining the data product, that meets organisational, technical and non-technical user requirements, retaining critical information.
  • Systematically clean, validate, and describe data at all stages of extract, transform, load (ETL).
  • Work with different types of data stores, such as SQL, NoSQL, and distributed file system.
  • Identify and troubleshoot issues with data processing pipelines.
  • Query and manipulate data using tools and programming such as SQL and Python. Manage database access, and implement automated validation checks.
  • Communicate downtime and issues with database access to stakeholders to mitigate the operational impact of unforeseen issues.
  • Evaluate opportunities to extract value from existing data products through further development, considering costs, environmental impact and potential operational benefits.
  • Maintain a working knowledge of data use cases within organisations.
  • Use data systems securely to meet requirements and in line with organisational procedures and legislation.
  • Identify new tools and technologies and recommend potential opportunities for use in own department or organisation.
  • Optimise data ingestion processes by making use of appropriate data ingestion frameworks such as batch, streaming and on-demand.
  • Develop algorithms and processes to extract structured data from unstructured sources.
  • Apply and advocate for software development best practice when working with other data professionals throughout the business. Contribute to standards and ways of working that support software development principles.
  • Develop simple forecasts and monitoring tools to anticipate or respond immediately to outages and incidents.
  • Identify and escalate risks with suggested mitigation/resolutions as appropriate.
  • Investigate and respond to incidents, identifying the root cause and resolution with internal and external stakeholders.
  • Identify and remediate technical debt, assess for updates and obsolescence as part of continuous improvement.
  • Develop, maintain collaborative relationships using adaptive business methodology with stakeholders such as, business users, data scientists, data analysts and business intelligence teams.
  • Present, communicate, and disseminate messages about the data product, tailoring the message and medium to the needs of the audience.
  • Evaluate the strengths and weaknesses of prototype data products and how these integrate within an organisation’s overarching data infrastructure.
  • Assess and identify gaps in existing tools and technologies in respect of implementing changes required.
  • Identify data quality metrics and track them to ensure the quality, accuracy and reliability of the data product.
  • Selects and apply sustainable solutions to contribute to net zero and environmental strategies across the various stages of product and service delivery.
  • Horizon scanning to identify new technologies that offer increased performance of data products.
  • Implement personal strategies to keep up to date with new technology and ways of working.

Full information on Data engineer (level 5) is available from the Institute for Apprenticeships and Technical Education.

Find training providers for this course

Apprenticeship location

Check if a provider can offer this training for the apprenticeship location.

Back to top View shortlist