Apprenticeship training course
Data engineer (level 5)
Build systems that collect, manage, and convert data into usable information for data scientists, data analysts and business intelligence analysts to interpret.
- Qualification level
-
5
Equivalent to higher national diploma (HND). - Typical duration
- 24 months
- Apprenticeship category
- Digital
- Maximum funding
-
£19,000
Maximum amount government will fund
for apprenticeship training. - Skills
-
Skills an apprentice will learn
- Collate, evaluate and refine user requirements to design the data product.
- Collate, evaluate and refine business requirements including cost, resourcing, and accessibility to design the data product.
- Design a data product to serve multiple needs and with scalability, efficiency, and security in mind.
- Automate data pipelines such as batch, real-time, on demand and other processes using programming languages and data integration platforms with graphical user interfaces.
- Produce and maintain technical documentation explaining the data product, that meets organisational, technical and non-technical user requirements, retaining critical information.
- Systematically clean, validate, and describe data at all stages of extract, transform, load (ETL).
- Work with different types of data stores, such as SQL, NoSQL, and distributed file system.
- Identify and troubleshoot issues with data processing pipelines.
- Query and manipulate data using tools and programming such as SQL and Python. Manage database access, and implement automated validation checks.
- Communicate downtime and issues with database access to stakeholders to mitigate the operational impact of unforeseen issues.
- Evaluate opportunities to extract value from existing data products through further development, considering costs, environmental impact and potential operational benefits.
- Maintain a working knowledge of data use cases within organisations.
- Use data systems securely to meet requirements and in line with organisational procedures and legislation.
- Identify new tools and technologies and recommend potential opportunities for use in own department or organisation.
- Optimise data ingestion processes by making use of appropriate data ingestion frameworks such as batch, streaming and on-demand.
- Develop algorithms and processes to extract structured data from unstructured sources.
- Apply and advocate for software development best practice when working with other data professionals throughout the business. Contribute to standards and ways of working that support software development principles.
- Develop simple forecasts and monitoring tools to anticipate or respond immediately to outages and incidents.
- Identify and escalate risks with suggested mitigation/resolutions as appropriate.
- Investigate and respond to incidents, identifying the root cause and resolution with internal and external stakeholders.
- Identify and remediate technical debt, assess for updates and obsolescence as part of continuous improvement.
- Develop, maintain collaborative relationships using adaptive business methodology with stakeholders such as, business users, data scientists, data analysts and business intelligence teams.
- Present, communicate, and disseminate messages about the data product, tailoring the message and medium to the needs of the audience.
- Evaluate the strengths and weaknesses of prototype data products and how these integrate within an organisation’s overarching data infrastructure.
- Assess and identify gaps in existing tools and technologies in respect of implementing changes required.
- Identify data quality metrics and track them to ensure the quality, accuracy and reliability of the data product.
- Selects and apply sustainable solutions to contribute to net zero and environmental strategies across the various stages of product and service delivery.
- Horizon scanning to identify new technologies that offer increased performance of data products.
- Implement personal strategies to keep up to date with new technology and ways of working.
Full information on Data engineer (level 5) is available from the Institute for Apprenticeships and Technical Education.