Guidance on producing quality analysis for government #

NB: All copyrighted material reproduced under the OGL. To comply with the licence I am providing a link to the licence as well as the original source

This work was commissioned in 2012 in the wake of the “difficulties with the Intercity West Coast franchise competition”

Roles and responsibilities #

There are three roles in assuring analysis:

• commissioning analysis
• providing analytical assurance
• delivering the analysis itself

Commissioning analysis #

The person commissioning analysis must ensure that those doing the analysis understand the context of the question being asked so that they understand the likely risks and can determine what the appropriate analytical and quality assurance response should be. The commissioner has a role to ensure that there is sufficient time and resource for the required level of assurance to be delivered and that they understand the associated risks when time and resource pressures are unavoidable

Providing analytical assurance #

The person responsible for providing analytical assurance must ensure they receive evidence that appropriate analytical quality assurance activities have been conducted and that residual uncertainties and risks are understood and are communicated. Typically this would be done by a senior analyst or analytical project manager who is not one of the analysts delivering the analysis. This activity takes place throughout the life cycle of the analysis from understanding the problem, through designing the analytical approach, conducting the analysis and relaying the outputs.

Delivering analysis #

The people responsible for delivering the analysis frequently assist the commissioner in structuring the question in order to ensure the appropriate analysis is performed. Some analysis may require external specialists and therefore analysts may also have responsibilities as part of the procurement process. Analysts, including those 3 rd parties providing analysis, should also provide proportionate documentation that outlines the verification and validation activities undertaken and the associated conclusion

Quality assurance #

Quality assurance comes from two main requirements:

• Environment
  • “creating the conditions in which quality assurance processes can operate effectively, facilitated by a culture that values quality assurance and welcomes effective challenge, a well understood chain of responsibility and sufficient time for quality assurance”
• Process
  • “establishing a clear process for every stage of the analytical life-cycle. This includes working alongside the commissioner and any other users to ensure there is a shared understanding about the purpose and any limitations of the analysis”

flowchart TB EAQ(Effective quality assurance)-->MAE(Modelling/ analytical environment) EAQ-->P(Process) MAE-->Culture MAE-->CAC(Capacity and capability) MAE-->Control P-->Guidance P-->Documentation

Environment #

• a culture where quality assurance is highly valued, and there are strong incentives to deliver appropriate quality assurance, backed by effective scrutiny of analysis, supporting models, assumptions and data
• capacity and capability where specialist staff have sufficient time built-in for quality assurance, and are able to draw on expertise and experience across government and beyond
• adequate controls, including a clear governance framework

Process #

• clear guidance that sets out the key considerations driving the approach to quality assurance
• clear documentation about the analysis and the quality assurance process