Working paper 1: A picture is worth a thousand words
MVAin4 All Locations February 21 2021
1. Introduction
This working paper describes the method and progress of using a Theory of Change approach to develop a shared understanding of ‘voluntary action’ as complex systems. Voluntary action is defined as “…the commitment of time and energy, for the benefit of society and the community, the environment or individuals outside (or in addition) to one’s immediate family” (see section 5). Complex systems are comprised of multiple components with additional factors which interact with each other.
The paper is prepared by Jurgen Grotz for the ESRC/UKRI funded project “Mobilising Voluntary Action in the four UK jurisdictions: Learning from today, prepared for tomorrow.” It is the first in a series of working papers reporting from the various work strands of this rapid response project addressing recovery from pandemic. This working paper reports a single part of work strand ‘A’, co-production, months 1 – 12.
In work strand ‘A’, “the research team and Project Partners will co-produce a common analytical framework”. This work has since been divided into four parts:
- developing a graphical representation of the logic model (Theory of Change) to gain a shared understanding of ‘voluntary action’ as complex systems across the UK national teams
- using the logic model to direct the co-production of project survey questions and tools with the national teams
- regular review of the logic model
- using the logic model to direct the analysis and presentation of the data
This working paper summarises the first part, the development of graphical representation of the logic model (Theory of Change) to gain a shared understanding of ‘voluntary action’ as complex systems across the four national teams. The draft Theory of Change as presented here is the product of co-production with all stakeholders in the project.
2. Rationale
This study is a complex four nation project with a UK-wide team of academics and the four key sector infrastructure bodies for each nation, supported by a Project Partner advisory panel from professional networks, organisations and related ESRC investments, to be delivered within 12 months, dealing with complex systems. Co- designing and co-delivering this study, putting principles of co-production into practice, required a tested approach and a researcher with experience of applying such an approach, to rapidly gain a minimum shared understanding between all involved in the study. A Theory of Change approach was therefore the pragmatic choice for developing the analytical framework for the project using a theory-based evaluation technique (Weiss, 1972) with refinements from process evaluation of complex systems (Moore et al 2015). The situation was exacerbated as under COVID-19 restrictions co-production activities such as deliberative workshops were not feasible. Grotz was asked to lead on this aspect of the work as he had recently led a project of similar complexity, also affected by COVID-19 restrictions, which had successfully used a similar approach (Stuart et al 2020).
The immediate task was to establish, within weeks of the project start, a shared understanding about what activities, for example emergency food and medicine deliveries, the project would consider, the complex geographic and temporal context in which those activities take place during the COVID-19 pandemic in four nations, and the mechanisms that are influencing them.
The approach was also useful for this project, in informing evaluation techniques for observing what difference activities make. Given the project’s overall aim to offer early findings which may support ‘recovery’, ‘preparedness’ and ‘lessons for post pandemic settlement’, the approach is relevant as it offers a systematic way to assess what longer-lasting differences, impacts, the activities are making in the context of the pandemic, and again, how context and mechanisms, are influence this.
3. Method
A Theory of Change approach, popularised by Weiss (1972) as a method to assess programme effectiveness, maps out how and why a desired change is expected to happen. It has since been applied in a range of contexts and for this study it is used to provide a visual representation of how and why voluntary action can make a difference in the response to COVID-19, offering a framework to systematically synthesise the complex and varied stands, geographic and temporal dimensions of this study. For this study the Theory of Change is merely intended to offer a visual representation, a logical pathway, and to illustrate how conclusions may be drawn from the available evidence. It helps direct the focus to specific evidence with relevance for specific questions for example how resource allocation affected activities or whether access to digital technology was an enabler or lack of access led to exclusion. It is important to recognise that this Theory of Change is not a final and static description of how changes happen, but rather a living document that will need to evolve with new evidence and resulting new assumptions to be explored. Furthermore, it is essential to acknowledge that the visual representation is a simplification which allows us to understand connections building on the shared understanding of the researchers and stakeholders for this project and that different representations are possible.
The restrictions imposed because of COVID-19 meant that often used and well described activities to collaboratively develop a Theory of Change, such as series of iterative deliberative workshops with different stakeholders, were not feasible for the project.
Instead, the project used an adapted Delphi Exercise as a pragmatic response to the complexity of operationalising the project, reflected in the wide range of stakeholders; COVID-19 restrictions; project length; and the limitations of online communication.
A Delphi Exercise relies on a collective iterative process of drafting, commenting, and redrafting. The Delphi Exercise included presentations to stakeholders 16 November 2020, 10 December 2020, 15 December 2020 for comments on drafting and a presentation of the Draft Theory of Change below, 15 January 2021.
As part of the Delphi Exercise, the language of the Theory of Change, in particular its key terms were presented and reviewed, this is reflected in the DRAFT Theory of Change below. In addition, commonly used definitions were also presented and reviewed, some of which are included below in Section 5, page 9-10, for information and illustration.
4. DRAFT Theory of Change v#2 11.01.2021
The following draft Theory of Change was presented to the Researcher Core Group 15 January 2021 with a view to inform subsequent meetings of the national teams, deliberating on the questions for the Call for Evidence. The Researcher Core Group is made up of the eleven co applicants and the co-ordinator. The Call for Evidence is part of the projects research methods and directed at key stakeholders in the four nations.
The colour coded graphic representation indicates first how it relates to the over aching aim of the project, then how it relates to the three main research questions as outlined in the project proposal and specified in titles of the individual pages of the Theory of Change below.
This graphic represents a ‘Roadmap’ and provides a basis for shared understanding that will now be used to discuss how to answer the research questions in detail with the various research strands and how to achieve the overall project aim.
Further feedback on the Theory of Change will be systematically collected during upcoming national team meetings.
Please note that “social welfare” was removed from the original project aim, with an explanation in the ‘Definitions’ section at the end of the document.
5. Definitions
The following terms used in setting the overall aim of the project, were included in the Delphi Exercise. They will require further review in line with the ongoing review of the Theory of Change and the other work strands of the project but are listed below as a result of the Delphi Exercise.
Voluntary Action
The Delphi Exercise reflected a broad consensus around the definition of ‘voluntary action’ mostly in suggesting it be made as wide as possible so as to also include community participation in groups without registration or a confirmed structure.
Voluntary Action is the commitment of time and energy, for the benefit of society and the community, the environment or individuals outside (or in addition) to one’s immediate family. It is undertaken freely and by choice, without concern for financial gain. It comprises the widest spectrum of activity for example, community development, arts, sport, faith based, education, neighbourliness, youth, environmental, health and direct care. This can include activities undertaken through public, private and voluntary organisations as well as community participation and social action in associations and groups which may not be registered or don’t have a confirmed structure.
This above definition is broadly based on the NI Volunteering Strategy, with formal/informal removed and an addition about unregistered groups without a confirmed structure. The core of the definition can also be found in Kearney (2001).
Community
During the Delphi Exercise there was also broad consensus that the definition of ‘community’ should include place, identity and interest, and circumstance. The following is an extract from an IVR document (Ramsay 2012), suggested by a participant in the Delphi Exercise. The term ‘Volunteering’ is replaced with ‘Voluntary Action’ to reflect definition above.
Voluntary Action [Volunteering] is a situated practice and a collective activity (Brodie et al., 2011). This collectiveness may be social (for instance, taking part in gardening at an allotment) or simply involved (for instance, voluntary action [volunteering] using technology to complete a survey for a charity). Similarly, the situated nature of these activities can be considered in terms of peoples’ lives, their social relationships or the physical and social sites of their volunteering.
Community is not a given or static concept. Indeed, people are widely acknowledged to belong to multiple communities, whether these are geographical or of interest and/or shared experiences. But it is important to recognise that voluntary action [volunteering] happens in space and place, and that the community is simultaneously a location, a site of identification and a set of relationships. (Ramsay, 2012).
Need (pandemic specific and/or general)
During the Delphi Exercise it was suggested that the phrase ‘unmet need’ contains a redundant qualifier. While this was not fully clarified, ‘need’ was recognised as a difficult term but one that required clear definition for use in this project ranging from basic needs such as food and shelter to more complex ones such as wellbeing. For the purpose of the graphic representation above, ‘need’ was replaced with “Who benefits and how” enabling an assessment of how need was met.
Social welfare
During the Delphi Exercise the term ‘social welfare’ was widely rejected, for example, as “not reflective of the common language in the sector”, “quite a narrow term”, “too close to the notion of public service delivery and government” and “there seems to be a bit of mismatch between the typology below and the implications of a definition of voluntary action built around the concepts of social welfare and service delivery which would, for example, exclude culture and sport”. It was therefore removed from the graphic representation above.
6. Discussion and conclusions
This working paper describes the pragmatic response to a complex and changing research environment, in developing a Theory of Change at the very beginning of the project to provide a working account of key related objects, actions and outcomes. It is subject to ongoing changes and to collaborative critical debate. Nonetheless the approach described here does appear to have delivered a useful tool, as intended, in providing a graphic representation that can indicate from an early stage how the project partners conceptualise the research landscape and how the project can address the overarching aim of the project and its three main research questions.
The choice of a Theory of Change approach and the practical development of it exemplify that the entire project is underpinned by a commitment to collaboration and co-production, delivered through partnership between a multi-disciplinary and multi stakeholder research team composed of academics and leading sector specialists, working across the four nations of the UK.
7. References
Brodie, E., Hughes, T., Jochum, V., Miller, S., Ockenden, N., & Warburton, D. (2011). Pathways through participation: What creates and sustains active citi- zenship? London: NCVO/IVR/involve.
Kearney J. “The Values and Basic Principles of Volunteering: Complacency or caution?” Voluntary Action 2001/2007;3(3):63–86, reprinted in Davis-Smith, J. and Locke, M. Eds (2007) Volunteering and the Test of Time: Essays for policy, organisation and Research. London: Institute for Volunteering Research.
Moore, G. F., Suzanne, A., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O’Cathain, A., Tinati, T., Wight, D. and Baird, J. (2015) “Process evaluation of complex interventions: Medical Research Council guidance” BMJ 2015; 350 :h1258.
Ramsay, N (2012) Understanding how volunteering creates stronger communities: a literature review. London: Institute for Volunteering Research
Stuart, J., Kamerāde, D., Connolly, S., Ellis Paine, A., Nichols, G. and Grotz, J. (2020) The Impacts of Volunteering on the Subjective Wellbeing of Volunteers: A Rapid Evidence Assessment. What Works Centre for Wellbeing and Spirit of 2012 https://whatworkswellbeing.org/wp-content/uploads/2020/10/Volunteer-wellbeing-technical-report-Oct2020-a.pdf
Weiss, C.H. (1972) Evaluation Research: methods for assessing program effectiveness. Englewood Cliffs N.J: Prentice Hall.