In this three-part mini-blog series, we will delve into Dr Grace Tidmarsh’s PhD work, which placed a focus on evaluating Positive Youth Development (PYD) programmes.
In the first blog, we will outline the findings from a systematic review that Grace completed, which was published in the Journal of Youth Development.
Grace completed a systematic review to understand the processes and factors that influence outcomes in PYD programmes. PYD programmes capitalise on young people’s strengths to foster long-term positive experiences, environments, and connections. PYD programmes can be assessed by using a process evaluation.
Description: SPRINT Project team members and alumni stood with young people and staff from St Basils at the University of Birmingham, UK.
What is a process evaluation?
A process evaluation helps researchers to find out whether programmes, such as PYD programmes, have been delivered in the way that was intended. They also give information about why a programme is successful and how this can be enhanced; and on the flip side, why a programme might fail or deliver unexpected and unwanted effects.
Process evaluations can focus on several different components. These may include:
The extent that the programme sticks to the intended content and style of delivery
The quality and success of staff training
How the individuals that the programme seeks to help are recruited
Individuals experiences of, and level of engagement in the programme
The systematic review that Grace completed focused on the outcomes of process evaluations of PYD programmes for disadvantaged young people.
What is a systematic review and how do researchers conduct them?
Description: Illustration of a grey pencil to the left of 4 tick boxes with white ticks inside them, on a yellow background.
A systematic review is a detailed and comprehensive synthesis of available evidence on a specific research topic. To plan, complete and write up a systematic review, PRISMA guidelines are followed. This includes a checklist that contains 27 items that relate to the different sections of a systematic review academic paper. Broadly, the items can be broken down into 5 key points:
1. Why is the research topic important and what are the questions that the researcher wants to answer.
2. What processes and criteria are used to find and screen relevant academic papers, and what types of outcomes are focused on.
3. How is relevant data extracted from the academic papers included in the review and how is the quality of the papers determined and reported.
4. What processes and techniques are used to analyse, group, and break down the findings, and how are they presented.
5. What do the findings mean and what implications do they have for practice, policy, and future research.
What was found?
10 PYD programme process evaluation academic papers were included in the systematic review
The overall quality of the PYD programme process evaluations was highly variable
Key factors that helped the delivery of the PYD programmes were community collaboration, meeting the needs of young people, and sustained communication
Key factors that hindered the delivery of PYD programmes were funding and logistical challenges, sessions being too like school lessons, challenges around meeting support and behavioural needs of young people
Key takeaways
PYD process evaluation quality could be improved by collecting both numerical (e.g., surveys) and non-numerical (e.g., interviews) data
Flexibility is vital when delivering and evaluating PYD programmes in complex settings and this should be reflected when assessing the outcomes and quality of PYD programmes
Greater awareness is needed around the importance of key factors that help and hinder the delivery of PYD programmes for disadvantaged young people
Stay tuned for part two of our mini-blog post series on Grace’s PhD work!
Would you like to read the academic paper on the PYD process evaluation systematic review? You can access it here.
You can also read our recent blog focused on a process evaluation conducted by the SPRINT team. We describe how a New World Kirkpatrick process evaluation was used to assess My Strengths Training for Life™ (MST4Life™).
References
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O’Cathain, A., Tinati, T., Wight, D., & Baird, J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ, 350, h1258. https://doi.org/10.1136/bmj.h1258
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., Petticrew, M., Rycroft-Malone, J., White, M., & Moore, L. (2021). A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance. BMJ, 374, n2061. https://doi.org/10.1136/bmj.n2061
Tidmarsh, G., Thompson, J. L., Quinton, M. L., & Cumming, J. (2022). Process evaluations of positive youth development programmes for disadvantaged young people: A systematic review. Journal of Youth Development, 17(2), 106-140. https://doi.org/10.5195/jyd.2022.1156
Photo credit: Canva and Dr Mark Holland.
Written by Dr Sally Reynard, Research Associate in the SPRINT project.