Evidence-based policy making: Proceed with care

All relevant documents pertaining to the reform of public administration in Serbia are calling for the introduction of evidence-based policy making (EBPM) that is supposed to make policies more strategic, outcome focused, joined-up and inclusive. Yet, the concept of the EBPM is not straightforward and it has been contested for decades.

The EBPM indicates a decisive role of evidence in the process of policy making and is based on several assumptions about the nature of policy making and evidence. Firstly, it assumes organised and cyclist process of policy making (rational model) where different stages orderly come one after another. Secondly, the evidence is seen as incontestable and its role is to decide policy outcome. And thirdly, the EBPM supposes the neutral relationship between evidence and policy making, that is, free from ideology and values. In other words, the EBPM assumes orderly policy making process where a small number of policy makers decide on the preferred policy options using the evidence that relatively simply provides an answer to what policy outcome is the best solution. However, the concept of the EBPM and described assumptions have been a subject of numerous critiques.

First, the rational model of policy making envisages policy making consisting of six stages (agenda setting, policy formulation or problem recognition, options considered, decision made, policy implemented, policy evaluated), often described as a cycle, so after the final stage ends (evaluation of the policy) the agenda is again defined and the steps are repeated. However, this model has been long condemned as naïve and termed as a policy myth that continues to be presented for public and media consumption. Namely, it has been suggested that the rational model is an inadequate explanation of the policy making that is in fact “far messier and unpredictable” and separation of stages is difficult to maintain.

Next, stating that evidence should be a basis for policy making sounds like a reasonable and self-evident claim. However, the evidence itself is proved to be a rather vague term and a subject of a debate. Particularly the well-known notion of the “hierarchy of evidence” has been the most contested issue related to the EBPM. Namely, the hierarchy of evidence refers to the different research designs often placed in a hierarchy to determine the standard of evidence in support of a particular practice or programme, with the randomised controlled trials being at the top (‘gold standard’), and case study reports at the bottom of the hierarchy. The hierarchy of evidence in reality means the primacy of quantitative and experimental research methods and marginalisation of qualitative research. Nevertheless, this hierarchy is based on a narrow concept of study design that neglects the issue of “methodological aptness”, that is, different research questions require different types of studies, and for some areas qualitative research is more appropriate.

In addition, the EBPM also assumes a neutral relationship between research and policy, where evidence is supposed to “provide the answer to most if not all policy questions” leaving all ethical and moral questions out of the process. Yet, this relationship has also been under question as in a democracy, decisions can never solely be made on evidence—they will be informed by ideology and values. More specifically, the relationship between evidence and policy making is not unbiased and technical, but rather influenced by politics and government’s agenda. Decisions about the selection of evidence are shaped by ideological preferences that can leave many relevant facts and information out of the picture, because it does not fit with a preferred set of values. For some it leads to the inversion of the EBPM term to a “policy-based evidence making”, thus recognising the EBPM’s failure to include relevant knowledge for the political purposes.

Having in mind all the shortcomings of the EBPM and said critiques of its assumptions, perhaps the use of evidence-informed rather than evidence-based policy making would be more realistic. The EBPM has been more aspirational than a descriptive term, hence the more accurate approach would be to abandon it and accept the limitations of the use of evidence in policy making. The evidence-informed policy making would entail more modest approach that recognises the importance of evidence for policy decisions, but at the same time takes into consideration other factors that (may) shape final outcomes.

[i] The name of the article published in 2001 by the professor Nick Black as a reaction to the introduction of the EBPM following the election of the Labour government in 1997.

[ii] The blog post is a summary of a longer academic essay written at the University of Bristol as part of the assignment within the MSc in Policy Research.

Image source: https://fcw.com/articles/2018/10/18/evidence-based-omb-gunter.aspx