The following is a description of the evaluation that were published in the first phase of the youngballymun strategy (2010-2013).

A Process Evaluation of youngballymun

This report presents the key findings of an independent process evaluation of the youngballymun initiative. 

youngballymun is a complex Community Change or Area-Based Initiative (ABI) which was established in 2007 with joint funding from the Atlantic Philanthropies (AP) and the former Office of the Minister for Children and Youth Affairs (OMCYA). It comprises five child and youth-centred service strategies designed to improve learning and well-being outcomes for all children and young people in Ballymun. These include: Ready, Steady, Grow (RSG); 3,4,5 Learning Years; Incredible Years-youngballymun; Write-Minded; and Jigsaw youngballymun. The five service strategies are located within an ambitious overarching programme that seeks to work with existing statutory agencies and community-based organisations in order to bring about long-term changes in service provision and practice and encourage evidence-based, outcomes-focused approaches. 

The principal aim of this evaluation was to appraise the initiative during an approximate three-year period (2010-2012) and to examine the processes and activities that influenced its development and implementation during that time. Broadly, the evaluation team set out to establish:

  • What worked during that period in terms of the development, implementation and evolution of the initiative and under what conditions/circumstances?
  • What has not worked and why?
  • What key lessons have emerged at this stage of the initiative?

The process evaluation was guided, from the outset, by the above objectives but from within these, the research team identified four overarching themes as being of central importance.  These include:

  1. The early design and development of the youngballymun initiative including institution building and organisational capacity (e.g. including organisational structures that set goals, encourage participation and provide effective leadership and guidance); 
  2. Engagement, partnership working and collaboration (e.g. that allow for the exchange of information, development and achievement of shared goals as well as the building of trust, and promotion of inter-agency working);
  3. Capacity building (e.g. that encompasses selection, training and coaching to improve skills and which encourages best practice and innovation for practitioners and services); and
  4. Systems change and longer term integration of services and practice (e.g. whereby the initiative works toward the establishment of sustainable change across different systems such as local services, administrative layers, policy).

The study was located within a broader implementation science framework. Thus, the four themes above are explored and discussed with reference to the six key stages of implementation including: exploration; installation; initial implementation; full implementation; innovation; and sustainability. 

A multi-method, predominantly qualitative evaluation was carried out between late 2009/early 2010 and mid-2012, although some limited survey work was also undertaken in late 2012. This approach elicited in-depth information on youngballymun that was relevant to the aims and objectives of the study and which was essential for understanding the 'how and why' of the youngballymun initiative.  Purposive sampling was used to recruit a wide and diverse range of key stakeholders (n=170) across the Ballymun community and beyond, all of whom had agreed to take part in a series of interviews and focus groups undertaken during the study period.

A Process Evaluation of youngballymun - full report (PDF)

A Process Evaluation of youngballymun - summary report (PDF)

 

Evaluation of the overall success and impact of Ready, Steady, Grow

This report presents the findings of an independent evaluation of Ready, Steady, Grow youngballymun's area-based infant mental health strategy. The evaluation was conducted between July 2011 and December 2012 by a team of researchers from the UCD Geary Institute and the UCD School of Psychology.

The Ready, Steady, Grow (RSG) service strategy is based on partnerships with the Health Service Executive (HSE) Speech and Language Therapy (SLT), the HSE Primary Care Psychology Services, the Public Health Nursing (PHN) Teams and a number of local community partners. It was originally conceived as comprising three specific components: an antenatal care programme, Infant Mental Health training and service model, and the Parent-Child Psychological Support Programme (PCPS). The articulation of RSG's work has evolved since its inception, and RSG is now clearly articulated as an Infant Mental Health strategy. To this end, RSG seeks to build the capacity of parents, practitioners, services and systems to appropriately support the Infant Mental Health needs of young children and their families, giving particular focus to the parent-infant relationship and children�s social and emotional development. 

There are two separate components to the Ready, Steady, Grow evaluation; a process evaluation and a programme evaluation. The process evaluation aimed to assess how and to what extent the RSG service strategy is promoting the use of Infant Mental Health principles and practice into the existing service structures in Ballymun. Three specific topics were investigated: capacity building, collaboration, and early identification and intervention. Exploratory interviews, designed to investigate these areas, were conducted with individuals identified by the research team and RSG staff as key stakeholders in Ballymun. The findings from these interviews were then used in conjunction with relevant literature to inform the development of a survey, which was administered at two time points to a larger sample of individuals working with young children and their families in the Ballymun area. 

The programme evaluation aimed to assess the impact and effectiveness of PCPS. It examined changes over time in outcomes for PCPS infants and their families (n=333) using measures routinely administered by PCPS staff as part of the programme's design. Where feasible, outcomes were then compared to the outcomes of a group of socio-demographically similar families who did not receive PCPS. This group was sourced from the comparison groups of Preparing for Life, which is another Dublin-based early intervention programme. Finally, the reach of PCPS in Ballymun was assessed, in addition to the level of PCPS participant engagement and attrition. The results of both process and programme evaluations were integrated, where relevant and appropriate, to determine the overall success of RSG.

youngballymun is fortunate to have a very close working relationship with the developer of the PSPS programme Professor M. �ngeles Cerezo. As a primary stakeholder in evaluation of Ready, Steady, Grow Professor Cerezo was invited to write a commentary on the PCPS section of the Ready, Steady, Grow evaluation report. We would recommend that the evaluation report be read in conjunction with Professor Cerezo's commentary. 

Evaluation of the overall success and impact of Ready Steady Grow  - full report (PDF)

Professor Cerezo's commentary on the Ready Steady Grow Evaluation - full report (PDF)

Evaluation of the overall success and impact of Ready Steady Grow - summary report (PDF)

Professor Cerezo's commentary on the Ready Steady Grow Evaluation - summary report (PDF)

Learning Years Evaluation Report

This report presents a summary of the findings of the independent evaluation of youngballymun's Learning Years Service, undertaken by SQW, in partnership with Dr Christine Stephen of the University of Stirling (School of Education).

The Learning Years Service aims to improve holistic developmental and learning outcomes for children in early childhood care and education (ECCE) settings in Ballymun by increasing the quality of service provision through staff professional development and the provision of mentoring and coaching support to enhance practice. To this end, the service provides two integrated strands of activity; coordinated mentoring support to facilitate the implementation of the National Quality Framework for Early Childhood Education (i.e. Siolta) and training and coaching support for practitioners to facilitate the implementation of the evidence based ECCE curriculum HighScope.

In October 2010 SQW was appointed to undertake an evaluation of youngballymun's Learning Years Service. The study was carried out between November 2010 and June 2012. The focus of the evaluation was on assessing the extent to which the Learning Years Service and its coordinators were having an impact upon the quality of ECCE provision and outcomes for children, and the degree to which this model of support is transferable.

The evaluation approach focused on three broad areas of investigation; developing a profile of preschool settings in the community with which the Early Years Quality Coordinator and the HighScope Coordinator work and of the families and children with whom they work; carrying out an operational analyses, including an examination of the modus operandi and effectiveness of the two Coordinators to determine what difference their roles have made to the quality of ECCE provision in Ballymun; conducting an investigation of the impact of the Learning Years Service on practitioners and on the quality of service provision.

This report presents a summary of the findings of the independent evaluation of youngballymun's Learning Years service

An Economic Appraisal of the youngballymun Initiative

This report presents the findings of the independent economic appraisal of youngballymun  undertaken by JustEconomics as a component of the overarching process evaluation of youngballymun being carried out by an international team led by NUI Maynooth.

youngballymun is a complex community change initiative, focussed on the delivery and embedding of evidence-informed prevention and early intervention practice into mainstream service provision across the life cycle of the child and family.

The study is described as a 'value for money' study (VFM) which in this context means the relationship between spending and short, medium and long term outcomes (i.e. the changes that take place as a result of an activity). The study appraises the extent to which in its design and early implementation phase (2007-2011) four services initiated by youngballymun can together be considered to deliver a return on State and philanthropic investment, and providing in those terms, value for money.

Value for money is high on the agenda of the Irish government and this research aims to make a contribution to informed discussion on how investment in prevention and early intervention services can generate greater collective benefit and gain from public spending.

The study was carried out between November 2011 and June 2012. The approach taken involved the collating of primary and secondary data available from service evaluation reports on short-run changes in outcomes (e.g. conduct problems),  using these to predict long-run changes in outcomes (e.g. reduced crime) and its associated costs, whilst gauging the length of time that outcome benefits would last, the level or 'drop off' that was likely and the extent to which observed outcomes were attributable to the intervention in question or other factors.

The final evaluation report (pdf)

A Summary Report (pdf).

Write Minded Evaluation

This report presents the findings of the independent evaluation of Write-Minded undertaken by SQW in partnership with the National Foundation for Educational Research (NFER). Write-Minded is youngballymun's area-based literacy strategy which aims to support schools, families, and community-based initiatives and projects in Ballymun to improve literacy outcomes for children and young people. It fits with and reinforces, at a community level, the work of Delivering Equality of Opportunity in Schools (DEIS), a national initiative, introduced by the Department of Education and Science in 2005. The strategy provides comprehensive, coordinated system-wide supports for the implementation of a mix of targeted and universal evidence-based practices focused on promoting literacy and educational wellbeing for children, young people and families in the community. 

In August 2010 SQW, in partnership with the National Foundation for Educational Research (NFER), was appointed to undertake an evaluation of Write-Minded. The specific purpose of this evaluation was to understand the effectiveness of Write-Minded to date in relation to:

  • child literacy outcomes
  • literacy skills outcomes for families
  • pedagogical outcomes for teachers
  • planning outcomes for schools
  • (primary/ post-primary) transitional outcomes for pupils 
  • literacy development outcomes for the wider community  

A second aspect of the study was to facilitate the development and implementation of a mechanism to allow for the ongoing monitoring and evaluation of the literacy strategy.

Evaluation activity was carried out between October 2010 and January 2012, and comprised of literacy assessments with children in Ballymun primary schools, in-depth school-based interviews and a survey of teachers and principals, as well as interviews with community organisations and a discussion group with local parents. 

Final Evaluation Report (pdf)

A Summary Report (pdf)

Whole-School Implementation of Incredible Years - youngballymun

This report presents a summary of the findings of an action research study to support the ongoing implementation of Incredible Years across six primary schools in Ballymun, focusing specifically on a whole-school approach to the implementation of the Incredible Years (Child Teacher and Parent) series of training programmes.

The idea of an action research study was conceived in order to maximise the gains that had been made in whole school implementation of Incredible Years, to add momentum to the change process and to ensure continuous improvement. It was envisioned that the action research cycles would progress implementation and continuous programme improvement at a school level. The study was undertaken (between September 2010 and June 2012) by an independent research team; Professor Mark Morgan (DCU) with Dr. Karen Espey. 

The overall aim of the study as per youngballymun was to engage with and support the Incredible Years Schools Coordinator and participating school communities in an action research process to progress and add momentum to the implementation of a whole-school approach to Incredible Years. More specifically the study sought to;

  • Identify the key elements of a whole-school approach to children's social and emotional learning.
  • To facilitate a participative process of action and reflection in each of the participating schools, involving all the relevant actors to progress a whole-school approach to the implementation of Incredible Years.
  • To identify the barriers to implementation, and facilitating factors.
  • To facilitate cross-school learning.
  • To document the implementation process and the learning.

Whole-School Implementation of Incredible Years - youngballymun
An Action Research Study  - Full Report (pdf).

Whole-School Implementation of Incredible Years youngballymun 
 - Summary Report (pdf).