Policy for Campaigns 04: How to make policy when you don’t have any evidence - Part 2.

Even with a shortage of evidence, you still have options for making effective policy

Let’s assume now that you’ve looked down the back of all the possible sofas, and there really isn’t much or any useful evidence sitting around in your organisation, waiting to be used. What now?

Policy with no or little evidence

Presumably your organisation is, or seeks to be, the authority on its subject? Well then, be authoritative. If you have to speak in generalities, do so: “people in XYZ Situation tell us that things are difficult.” This isn’t ideal or without risk: it depends on your organisation being sufficiently respected that your key audiences will place some trust in what it says; and if you are challenged, you may find yourself exposed, so tread carefully if you know there are hostile actors in the area who have competing or opposing views. But equally, if you’re confident that what you’re saying is correct, this approach can get you a certain distance. The tone and methods of your campaigning will matter, though: being too shrill, or overstating the case, could cost your campaign a lot of credibility, and ultimately doom any chance of securing the change you want.

As touched on in part 1 of this article, one thing you can do is explain the logic of the situation: if you can set out a plausible chain of events that is likely to flow from a particular policy decision, a lack of depth in your evidence base might not be readily apparent. If you’re principally concerned with the future consequences of a possible or planned policy change, can you run some sort of exercise to wargame how it might play out, using knowledgeable people in your organisation? Again, it’s not ideal: you’ll need to root any such predictions in good knowledge of the existing situation, which your thin evidence base might make hard. If you’re challenged you might struggle to find a good answer. But finding a credible way to talk about causation and consequences can be a partial way around a lack of evidence: it should help you develop your recommendations, as well as frame your campaign.

Another approach is to make the message human: this is a classic comms method, but it can be a fair jumping-off point for policy too. Even if you can only get a small amount of it, first-hand testimony is incredibly useful. The “first-hand” bit really matters: you need to be able to present people’s experiences in their own words, not paraphrased by you. When all’s said and done, that’s people’s own experiences of what has happened to them. You might not be able to prove how representative it is, or extrapolate out from it across a wider population, but it’s hard for your audience to gainsay it. That’s not to say they won’t try, or even that others in your own organisation won’t (I’ve had to remove first-hand testimony from policy documents in the past because a charity’s trustees were unconvinced of its representative-ness, when you might have thought they would attach more weight to it). You can build policy analysis on this sort of testimony, provided you are clear about the extent to which it is a bit tentative and provisional. When campaigning using this evidence, emphasising when presenting it that this is someone’s actual, real-world experience can go some way.

Related to this, if you have a limited base of evidence, be open about its limitations. Make clear that it’s from a small and/or self-selecting sample with particular demographic skews, rather than waiting for a critic or opponent to do it. It doesn’t prevent you from drawing conclusions, provided you present them with an appropriate amount of caution. “More research is needed” is an evergreen policy recommendation for good reason: if you make a point of calling on better resourced organisations, like government departments, to commission their own research to validate your findings, you can hardly be accused of over-extrapolating from limited evidence; but you can highlight that someone else hasn’t paid enough attention to the issue.

Looking outside your organisation for evidence

If you’re lacking evidence in-house and don’t have the resource to commission research to find it (or hire staff to do so), you could consider whether anyone else has relevant evidence. There may be other organisations working in the same field, even if they do not do the same thing; if they do do largely the same thing, perhaps they are a competitor to your organisation.

If you have a competitor organisation, this at least means that any evidence they have generated will probably be relevant for you. However, utilising their evidence will probably need to be done with great care, in order to avoid upsetting anyone’s sensibilities or diverting scarce resource into a pointless squabble. Or, if it’s really tense between the two organisations, you might just have to pretend the other one doesn’t exist, which might not be tenable in the long term, and means you don’t get to refer to their evidence.

Alternatively, there may be organisations in adjacent sectors whose research or reports contain occasional nuggets of useful information. Obtaining a few scraps in this way might be enough to put a useful bank of insight together, from which you can start to draw conclusions and offer recommendations. The danger is that you end up cherry-picking or taking items out of context, particularly if there are complexities in the other organisation’s area that you’re not fully across. Also consider whether the other organisation might object to you using their information; but assuming they’ve published it (and that you’re not directly reproducing their graphics or text), they won’t have much cause for complaint at a fair and accurate reference to their work.

Equally, what about work that’s not necessarily directly in the public domain: academic research? Depending on the field you’re in, you may find that academic studies have been undertaken that shed useful light on your issues. Many journals now make their articles available on an open access basis, although some will need either a subscription or access to a library that has a subscription. To get the most out of this route, someone with the skills to do a proper literature search and access to an appropriate library will be necessary, but you might find some good material even with more piecemeal approaches. Working with academic researchers in your field is another candidate for a future article – if you have experience of this, please share it in the comments.

And finally, while we’ve considered hostile or competitor organisations, what about friendly ones? If you’re struggling to get evidence because you’re a small organisation, is there someone bigger in the same or a related field who you can work with? Could they even modify or expand their own evidence-gathering a bit, to produce information useful for you? Going even further: getting other organisations around a table, or signed up to a common goal, is a common campaigning tactic, but you might be able to use it for policy as well. Could you garner the insight you need from a round-table discussion with experts from other organisations and/or academia, or by ringing round individually to sound people out about possible future joint work, and at the same time asking how things look from their perspective? If you use this method, you need to keep the collaborative approach going into the actual campaign: don’t just pick everyone’s brains for their insight and then run off cackling with it to do your own thing.

And don’t forget the people you are ultimately working for: your members, supporters, beneficiaries, or however you define them. You should be involving them in your work throughout (and this article has somewhat assumed that; past and future articles cover it at greater length), but there’s a particular role for doing that in developing your policy recommendations. Of course, that also costs money: if you’re bringing in people to advise you over a period of time, it’s good practice to pay them, and you should definitely be reimbursing expenses as an absolute bare minimum. But if it comes down to it and all you’ve got is the most passionate and committed volunteers who are willing to do the work for no reward at all, well, that’s who you’ve got to work with.

Conclusions

There are no easy answers to a paucity of evidence: a superabundance of clear-cut evidence is always the ideal! But nil desperandum: you probably have more information, and more options for getting it, than you might realise; and you can certainly find ways to make best use of what you’ve got, provided you do so carefully.

Please let me know in the comments what your experience in this area has been: how have you worked with scant or low quality evidence; what have you done to build up evidence from a low base; or if you’ve enjoyed a substantial body of evidence, how did you come to have it, and how did you make use of it?