Modernising NHS service analytics #

Summary of recommendations #

[TBC]

Introduction #

“At present there is no formal professional body (but some outstanding grass roots organisations); very little formal structure around teaching and training, or ‘continuing professional development’; and a general lack of clear technical job descriptions or qualifications specific to NHS service analytics that managers without technical skills can use to evaluate the skills mix that they have, or need, in their service. This stands in very striking contrast to arrangements around the many other technical professions that drive NHS activity including laboratory technicians, clinicians, nurses, physiotherapists, radiographers, and so on…”

“By contrast, other technical specialties in the NHS have a strong and diverse strategic infrastructure to guide their initial training, career pathways, supervision, recognition, ongoing training, and to help create a technical ‘commons of knowledge’ around their work. This will include Royal Colleges or other professional membership organisations that are typically high status and well resourced; detailed job descriptions at a range of seniorities; formal arrangements nationally and locally around training both at entry level and for continuing professional development; and so on. Similarly other government analyst professions such as the Government Economic Service, the Government Statistical Service, and the Government Operational Research Service each have a head of profession, with clear career paths and progression opportunities, supported by well-curated continuing professional development, and the other features of a strong technical profession. These national organisations set out clear best practice guidance, offer analysts accreditation, and require analysts to adhere to a code of conduct. These models for technical work in both the NHS and government provide a clear template for future work around the NHS analysts service.”

Organisations supporting NHS analytics #

• NHS-R community
  • GitHub
• NHS python community
  • GitHub

“The fact that these organisations have run for so long, on modest resource, with comparatively large impact, demonstrates that there is a strong need in the NHS analytics profession for strategic structures, training, and so on. However they also show that this cannot be delivered solely through ‘inspiration’ to the profession on its own. The individuals involved in driving these organisations have achieved phenomenal outputs but they do not have the scale, voice, access and infrastructure of the substantive structures in other professions. They should be closely involved in all subsequent work around the NHS analyst service, the impressive power of smaller scale voluntary and charitable work has now been clearly demonstrated, but the limit of the voluntary model in this space has also been met.”

Recommendations #

[this is mostly verbatim with a few edits for length]

Professional structures #

NHSA 1: create an NHS analyst service modelled on the Government Economic Service, Government Statistical Service, Government Operational Research Service and Government Social Research Service. These professions each have a head of profession, clear career paths and progression opportunities supported by continuing professional development. They hold their staff to high standards by setting out clear best practice guidance, offer accreditation, and set out a clear code of conduct. This service should be responsible for delivering most or all the following tasks.

NHSA 2: create clear job descriptions for NHS analysts at a range of levels

In collaboration with NHS analysts, Association of Professional Health Analysts, NHS R Community, Royal Statistical Society and the Cabinet Office Central Digital and Data Office, the proposed NHS Analyst Service should create clear job descriptions for NHS analysts from entry level to head of profession. The job descriptions should be underpinned by a clear competency framework outlining the specific technical skills required to complete specific technical tasks.

Such tasks may include:

• data communication
• data analysis
• data management
• statistical modelling
• risk prediction
• service evaluation

NHSA 3. Revise Agenda for Change, and ensure technical staff are paid realistic salaries

NHS analysts, software developers, engineers, and other technical staff should no longer be classified as ‘administrative / clerical’ staff. Technical roles require their own category within the Agenda for Change pay scale framework, with their own job titles, capabilities, competencies, KPIs (Key Performance Indicators), and competitive remuneration packages. The NHS must stop expecting to pay highly skilled technical staff in data science and software development on salary scales devised for low and intermediate level IT technical support. Data scientists and software developers in the commercial sector routinely earn more than their manager, customer, or commissioner: this reflects market value, and is no different to employment of senior clinicians, accountants, lawyers, or other technical specialists by organisations. If barriers are hit when discussing offering higher salaries to senior developers with longstanding experience, the anchor point for negotiations should be NHS clinicians’ salary.

NHSA 4. Support an NHS analyst community

NHSA 5. Develop an annual data conference for NHS service analysts

NHSA 6. Find good staff, and empower them quickly with ‘data pioneer’ fellowships

This should be an open competitive programme where applicants can seek resource to cover half of their salary for 3 to 5 years so that they can spend half of their time spreading and developing their working methods, teaching, developing teaching materials, or receiving analysts for supervision and mentorship on placements.

NHSA 7. Identify 3 ‘data pioneer’ analytics teams in ICSs and trusts

To demonstrate the power of modern open methods in NHS service analytics, NHSX should identify 3 Integrated Care Systems and/or hospital trusts where there are strong existing skills in analytics, informatics, and/or software engineering to act as data pioneer teams.

From each group 2 to 4 individuals should be provided with advanced training in modern, open, computational and collaborative working methods, including RAP, with the rest of the team given training in the foundations so that they can learn from ‘doing’ under the direction of the group leaders with advanced training. These data pioneer teams can lead by example, providing open documentation of their work for others to learn from, make the methods and code local service analytics more visible to the wider community, and feed into the wider programme of modernisation around the NHS analyst service. It may be useful to choose teams and individuals who are close to working with raw NHS records data, as they will have substantial internal knowledge around data management that will be widely applicable.

NHSA 8. Commission intermittent code and analysis audits of organisations and analyst teams for service improvement

In collaboration with academics, and key organisations such as AphA and the NHS-R community, the proposed NHS Analyst Service or Analyst Head of Profession should commission regular code audits of all organisations that have received public funding for health data research or analysis

These audits should:

• follow a set methodology
• be published openly
• be used for the explicit purpose of improving performance, rather than penalising poor performance

Specific criteria should be developed in collaboration with the community but include:

• delivery and use of open code
• open methods
• open data where possible
• sharing insights
• support for CPD in work time
• whether staff meet job descriptions with training, continuing professional development or other proof – good performance should be further incentivised by highlighting best practice examples

HSA 9. Create an Analytical Capability Index

This should be developed independently, and used nationally, to track whether individual organisations have room to improve and signal to leadership where gaps lie in their organisation, how they compare to peers, who they can learn from. Careful consideration should be given as to how best to present the results, and whether this should be made public or not. It is important that the results are only used to drive genuine improvements, and not used for arbitrary contextless performance management.

Training #

NHSA 10. Create an Open College for NHS Service Analysts

This brand will emphasise that the training is open to all interested parties, and that the analytic methods promoted are themselves modern, open approaches to data science. This Open College should contain the following activities, set out in NHSA 11 to NHSA 21.

NHSA 11. Devise the content of a national training programme for NHS analysts: initial and CPD

Clear job descriptions and pathways must be tied to training and, where appropriate and non-onerous, proof of competencies. Health data is complex, as are health services: working as an analyst in this setting requires a range of specific knowledge around practical health data analytics, alongside more general technical skills in data management, analysis, and visualisation. The NHS Analyst Service should be tasked with devising a curriculum and training requirements for the key competencies associated with job roles, with clear recognition of existing experience or training in and outside of health, and so on. This should be facilitative rather than restrictive, and be focused on informing high quality training, rather than imposing onerous requirements to gather paperwork as proof of skills.

NHSA 12. Oversee funding and delivery of training, both open online and one-to-one

Training should be an appropriate blend of openly accessible online training, such as MOOCs, accompanied by formal one-to-one or group work to support feedback, supervised practical work, and evaluations, in the situations and skillsets where this more expensive in-person training can be shown to deliver better outcomes than open online work alone.

CPD courses should award completion certificates, proof of CPD, recognised or even required by managers, and these should be matched where relevant to competencies in analyst job descriptions. This should be overseen by a governing body and developed in close partnership with AphA, RSS (Royal Statistical Society, and the NHS-R Community.

NHSA 13. Establish new core training for analysts

Replace the Health Education England Graduate Management Training Scheme in health informatics and health analysis specialisms with a specific graduate training scheme in health data analysis, that should include: core training; rotation in different parts of the NHS (for example, in primary care vs. secondary care); the opportunity to specialise (for example, in data engineering vs. data management vs. data analysis); and specific training in the use of modern open computational methods. This will require funding and coordination from national arms’ length bodies, local NHS organisations, national funders, NHS Leadership Academy, HEE, academic organisations (where they can demonstrate a specific commitment to practical NHS service analytics) and more.

NHSA 14. Outline clear, non-onerous CPD training requirements for analysts

NHSA 15. Embrace RAP and modern, open working methods

Excel has its place, and training will be required at a range of levels for a range of skills. However, there is a clear need to move away from inappropriate use of inefficient and outdated ‘point and click’ methods for analysis, and towards a model based on Reproducible Analytical Pipelines with modern, open, collaborative approaches to data science.

Intermediate, and advanced analyst training should focus on enabling the workforce to develop skills in modern, open, collaborative computational data science with an emphasis on reproducible analytic pipelines covering concepts and skills such as R, version control, GitHub, Jupyter notebooks, Pandas, and similar. This does not mean that everyone in the system must become an expert software developer: but it does require some changes in skillsets and emphasis. RAP has a proven track record in other parts of government and in Public Health Scotland, with a strong model for spreading change in organisations. These will be new skills for many and so training will entail more than links to external generic data science guides.

NHSA 16. Ensure training focuses on RAP as much as Machine Learning

There is a tendency for training to be diverted into more exotic forms of data analysis such as Artificial Intelligence. These have their place, and there are many existing resources that meet these training needs very well outside of health analytics for those who have already developed outstanding skills in data science. However, the key unmet training need in the service is RAP, and the delivery of analytics using modern, open methods to achieve improvements in efficiency, sharing, quality, transparency, documentation, and reproducibility. This must be the priority for any training programme.

NHSA 17. Create a technical team to house and develop continuing professional development resources

Training in technical skills needs to be delivered by those with technical skills in data science as applied to NHS data. It cannot be delivered by generalist data scientists alone. Training also needs to be kept up to date. The aim should always be to provide training in the most advanced computational data science skills; what these are will change over time.

Providing a team of technical specialists with adequate funding to develop, deliver, share, and curate training, including the development of tools such as sandboxes where analysts in training can practice their coding and analysis against dummy data that reflects real NHS data, will be essential if training is to be high-quality and up to date.

NHSA 18. Ensure all training is open by default

The traditional funding model for training from universities and many other providers is to charge per-attender. Wherever online training resources are commissioned they should be open by default, with all video lectures, training materials, written content, exercises, and code shared openly.

It is necessary to remove access barriers to knowledge and training, in a space that urgently requires up-skilling, and to avoid imposing a requirement on analytic staff to ask permission of generalist managers, who may lack technical skills themselves, for access to training budgets that require onerous engagement with bureaucracy.

Fully open access to all NHS analyst training resources will also create substantial network benefits. It will make these training resources accessible to NHS staff in adjacent specialties who wish to up-skill, including managers and clinicians, enabling them to drive forward better use of data in their own teams and organisations; and to outside elements from the public and private sector making it clearer to them how the NHS uses data to improve care, and how they can interact to offer help and support, or improve analytic work with better tools, algorithms, services, or insights.

NHSA 19. Create and maintain a curated national open library of NHS analyst code

Hire a team of 10 people to create an open library of code and workbooks for key recurring tasks, examples of best practice, ‘how-to guides’, code for common analytical queries, codelists, variables, and so on. It must be unashamedly technical but meet the needs of staff with a range of abilities.

The library should be presented as a flexible open online website, with clear tagging, careful thought around discoverability of resources, and careful curation of individual resources into ‘training arcs’. The library delivery team should be led by an editor experienced in producing good open online technical resources; it should include analysts but also include expertise in technical writing, knowledge management, online education, and publishing.

An MVP (Minimum Viable Product) should be created within 6 months by pulling together the best existing resources from national and local teams in close collaboration with all key stakeholders and teams already listed above. This library should be closely tied to (and feeding into) online teaching and CPD. CPD points should be provided for contributing to the library and there should be an obligation for any analyst developing code with public resources to contribute the outputs to the library for re-use.

NHSA 20. Create training specifically for senior leaders to help them become better customers for data analysis

NHSA 21. Commission a rapid review of medical school curricula and similar

Platforms and data access #

NHSA 22. Improve the provision of data analysis environments

NHSA 23. Revise NHS IT policy for analysts to ensure it is fit for purpose

Analysts need to be able to use modern computational data science tools such as python, GitHub and docker on their NHS computers. Current IT policies often block the use of such tools. A similar challenge has been faced and recently overcome by the analytic community in government outside of health.

This must be addressed in national and local IT policies with clear statements on assurance and risk from the NHS Transformation Directorate to local decision makers, to make it the norm for work to be delivered using modern computational data science tools and avoid the apparently prevalent problem of analysts using these tools outside of the formal permissions and policies of their workplace.

NHSA 24. Rationalise national audits, RightCare, GIRFT, and Model Health System

NHSA 25. Make change practical

The NHS should identify 3 data pioneer ICSs that can move to a full TRE and RAP working style within 6 months; and 3 data pioneer national quality improvement audits (at least one within NHS England) that can move to full TRE and RAP working within 6 months.

External collaborations #

NHSA 26. Commission and promote best practice on outsourcing analytics

The NHS Transformation Directorate should coordinate the development of Best Practice guidance on outsourcing to cover the range of scenarios where such external collaborations are and are not beneficial to the system, boilerplate contractual requirements, and red flags around working methods and delivery.

NHSA 27. Require all outsourced or external work to comply with RAP and open working methods

Currently when analytic projects are outsourced to consultancies, academic collaborators or other agencies it is common for only the results to be reported, for example in a PDF or slide deck, without the accompanying methodology or code used to conduct the analysis. This prevents the NHS from error-checking the work, learning from it, or being able to replicate it internally, whether in the organisation that originally commissioned the work or elsewhere in the system. This creates duplication of work, and the loss of knowledge that can create efficient analyses and drive innovation.

This cannot solely be addressed by asking external partners for ‘training’ or more detailed narrative descriptions of the methods used. As discussed in the chapter on Open Methods, all NHS data management and analysis code should be accompanied by adequate technical documentation alongside the code, as required by the minimum standards of RAP, openly available for re-use and external scrutiny. All outsourced work should adhere to this requirement.

NHSA 28. Support NHS/academic collaborations on RAP data science for NHS service improvement

UKRI/NIHR should consider running an open funding call specifically for academic teams to collaborate with national or ICS NHS data analysis teams on using RAP and modern open data science techniques to improve the quality of NHS care, to deliver specific outputs, and to build mutual relationships and capacity building around applied analytics. The targeted outputs should be a range of Jupyter notebooks or similar with well-documented open code describing – with appropriate technical documentation – how NHS data was prepared, analysed, and used to identify or address opportunities to improve NHS clinical activity or outcomes.