Rubicon youth alcohol and drug support services outcome evaluation, Sept 2007 - New Zealand Police

Rubicon youth alcohol and drug support services outcome evaluation

September 2007

A report prepared by Bea Makwana and Lisa McCauley, Evaluation Team: Organisational Assurance, National Headquarters, New Zealand Police

This report has been produced by Bea Makwana and Lisa McCauley of the Evaluation Team: Organisational Assurance, Police National Headquarters. The report is the final evaluation of the Rubicon Youth Alcohol and Drug Support Services. It covers the period February 2003 to 31 December 2005.

First published in September 2007 by New Zealand Police, PO Box 3017, Wellington, New Zealand

Crown Copyright 2007

ISBN 0-477-02985-X


Summary

Rubicon is a youth alcohol and drug support service based in Whangarei that was one of 14 community youth programmes selected nationally in 2002, to reduce youth offending. Rubicon received $165,000 over three years.

This outcome evaluation considers the effectiveness of Rubicon over three years, 2003-2005. During this time Rubicon had 570 clients varying between 11 and 18 years. The majority of clients were M&3257;ori although there was a diverse range of ethnic groups.

The evaluation found:

  • the majority of clients were aged between 13-15 years for a 12 month contract with Rubicon.
  • clients had a good understanding of their goals while they were on the programme, and were reducing their drug and alcohol use.
  • client retention in an educational setting was strong due to a ‘contract&rquo; signed by the school, clients, Police and Rubicon, and the relationship Rubicon shared with the local Campus Cop.
  • there were some challenges for Rubicon, including limited time in schools despite growing demand for counselling and drug testing.

A strength of Rubicon was that it was a specialist service to meet a community need. The relationship with Police provided legitimacy to the programme, the drug testing provided quantitative records for ongoing monitoring, and provided an incentive for clients to not take drugs, and weekly counselling in schools provided consistent support. There is an opportunity for Rubicon to refine their services by developing more accurate demographic and drug testing results. The evaluation also indicated that both Police and Ministry of Justice need to provide more support to community youth programmes during the implementation, assisting with recording systems and clarifying monitoring procedures.

Background

In April 2002 the Ministers of Finance, Justice and Police received additional appropriations for 2002/03 and out-years to enhance the capability of Police and the Ministry of Justice to reduce youth offending and/or general violence. A total of 14 Community Youth Programmes (CYP) were selected nationally to implement programmes aimed at reducing youth offending. This outcome evaluation is of the Rubicon Youth Alcohol and Drug Support Services programme in Whangarei which received a total funding package of $165,000 over three years.

Evaluation Phases

The Rubicon programme has had three evaluation phases: formative, process and outcome. The focus of this report is the process and outcome phase which have been combined into one report, and covers the period from programme inception in February 2003 until 31 December 2005. The aim of the evaluation was to assess the overall effectiveness of the programme by measuring its success in meeting the programme objectives.

The process and outcome evaluation utilised mixed methods, using both qualitative and quantitative information. Qualitative information was collected via interviews, and quantitative data were primarily extracted from the Rubicon six-monthly monitoring reports.

Rubicon Youth Alcohol and Drug Support Services

Rubicon is a specialised alcohol and drug support service working with youth aged between 12 and 18 years in Whangarei. Entry onto the programme required the young persons to have alcohol and/or other drug related issues and either be attending school, or not in an educational setting but living in the Whangarei district; and caught by parents/caregivers/school/Police in possession, or under the influence of alcohol or drugs. The programme had an expected throughput of 150 clients per year.

Rubicon was originally staffed by a programme manager, but increased to have two full time alcohol and drug counsellors, a part time administrative assistant, and most recently a wh?u liaison coordinator and another alcohol/ and other-drug counsellor.

Objectives for Rubicon Youth Alcohol and Drug Support Services

The programme objectives included:

  • to reduce the level of drug and alcohol use by young people;
  • to reduce the level of offending and drug related crime by youth participating on the programme;
  • to reduce truancy; and
  • to encourage youth to remain in an educational setting.

Rubicon Clients

Over the evaluation period, a total of 570 clients were accepted onto the programme, varying in age from 11 to 18 years. Most clients were male, aged between 13-15 years. The majority were M&3257;ori although there was a diverse range of ethnic groups identified on the programme.

Most referrals to Rubicon were from schools, with students becoming contracted clients, self-referrals or undertaking one-off drug tests. Referrals were also received from government agencies, and community agencies but the priority was to work with clients who signed a Police Alternative Action Plan (PAAP) with the Campus Cop and Rubicon Contract for 12 months.

Alcohol and drug counsellors visited schools each week, and random drug testing was conducted by an independent person contracted by Rubicon. The majority of contact by alcohol and drug counsellors was one-on-one with clients as the large client case load made it difficult to work more holistically with families.

Although clients were expected to be on a Rubicon contract for 12 months, the programme did not collect specific entry and exit data of clients, in addition clients would re-enter the programme as self-referrals, or change schools and have to re-enter the programme before the 12 months was completed, making it difficult to assess the duration of client involvement. During the evaluation phases stakeholders developed a good understanding of Rubicon services although there were a small number that wanted more communication and clarification of services.

During the evaluation a number of pitfalls and improvements were identified by clients, stakeholders and staff which are included in this report.

Five clients participated in case studies. Their stories are presented in this report.

Key Programme Objectives

Reduction in Level of Alcohol and Drug Use

Stakeholders considered Rubicon to have raised the profile and awareness of youth alcohol and drug issues in Whangarei, and felt that some clients had reduced their drug use. However, there were concerns that drug testing was not undertaken regularly enough by Rubicon, and there had been difficulties in having school staff present while the caseworkers drug tested students in school.

A six-month snapshot of drug testing results was taken between July and December 2005 to provide descriptive information about client drug use. The majority of clients were aged between 13-15 years, and identified as M&3257;ori. The results were collated by school, rather than tracking individual drug test results, and the results did not include all clients each month. The data showed that increasingly Rubicon completed follow-up tests with contract clients. An unexpected outcome was Rubicon clients liking the drug tests as it provided proof of their abstinence.

Reduction in Offending and Drug Related Crime

Rubicon was a specialist drug and alcohol intervention service that did not require clients to have previous offending histories, or be at-risk of offending. In addition, the large client case load limited the opportunity to focus on offending related behaviour. As a result, it was not appropriate to include any offending analysis into the evaluation.

Reduction in Truancy

At the time of the evaluation Rubicon did not have access to truancy data from each school therefore evidence of a reduction in truancy was not possible. However, there were positive education related outcomes by clients, with Rubicon clients becoming reintegrated into the school system, improving academically, and with some taking leadership roles within the school.

Client Retention in an Educational Setting

The relationship Rubicon had with the Campus Cop and the joint PAAP and Rubicon contract was a critical part of how Rubicon became established in local schools. During the process phase of the evaluation educational benefits included students staying at school and being better behaved. Stakeholders also felt that Rubicon had provided support to schools and helped the relationship between home and school. There were a range of positive outcomes, including clients staying in school, taking leadership roles, achieving NCEA, having career aspirations, part-time work after school, while others were experiencing better relationships at home. There were some challenges, including confidentiality issues within schools, having to work with schools that had different philosophies and approaches to drug and alcohol related issues, Rubicon contracts going missing within the schools, and limited time in schools despite increasing client numbers.

Conclusions

The providers felt the programme was effective because of the confidential, trusting and strong links that were developed with clients, and the counselling and drug testing over the twelve month period. Providers were also positive in having self-referral clients as they were motivated to change their behaviours.

Rubicon was clear about the purpose of the programme and services provided. The five case study clients demonstrated an understanding of the purpose of their involvement, recall of their goals, and reductions in their drug and alcohol use.

Strengths of the Programme

Rubicon has provided a specialist service to meet a community need. A contract with Police, Rubicon and the client has provided legitimacy to the programme, and the drug testing has given quantitative records for ongoing monitoring of drug use, while providing an incentive for clients to not take drugs. Regular staff meetings and professional development has supported staff. Rubicon has built strong interagency relationships reflected in the range of referral agencies that exist and the requests for interventions and assessments. Rubicon has provided a range of counselling services to best meet the needs of clients and has maintained regular visits to schools to ensure an ongoing relationship with clients. The five case study clients have demonstrated an acute understanding of the reasons for their involvement with Rubicon, recall of goals, and reductions in alcohol and drug use.

Area for Improvement

A move towards more accurate monitoring of client demographic details, entry and exit details, drug and counselling data would help to determine how a client is progressing through the programme. Consideration should also be given to ensuring clients receive the intended eight drug tests over 12 months. Rubicon files need to be accessible for monitoring and evaluation, and there were requests from stakeholders for more regular communication. With a move towards wh?u counsellors and holistic measures, Rubicon may need to consider what needs assessments are most appropriate to record information. Rubicon and Police would benefit from clarifying the branding and ownership of ‘Rubicon&rquo; to maintain services and relationships.

Learnings for Police and CPU

As with other CYPs, Rubicon has underscored the need for more support from the Ministry of Justice Crime Prevention Unit (CPU) and Police in the implementation and monitoring of the programme. This includes the setting up and recording of drug tests, which were unique to Rubicon, to ensure the data is accurate, relevant and provides the greatest utility to the programme and key funders.

Top

Table of contents

Acknowledgements
Table of Figures and Tables
Section 1: Executive Summary
Section 2: Introduction
Background to the Community Youth Programmes
Selection of Rubicon Youth Alcohol and Drug Support Services
Report Structure
Section 3: Evaluation Approach
Rubicon Youth Alcohol and Drug Support Services evaluation phases
Methodology
Limitations of the Data
Ethical Considerations
Section 4: Description of Rubicon Programme
Rubicon Youth Alcohol and Drug Support Programme History
Section 5: The Programme and Clients
Aims of the Rubicon Youth Alcohol and Drug Support Services Programme
Client Information
Programme Content and Delivery
Stakeholders
Case Studies
Section 6: Outcomes of the Programme
Aim 1: Reduction in Level of Alcohol and Drug Use
Aim 2: Reduction in Offending and Drug Related Crime
Aim 3: Reduction in Truancy
Aim 4: Client Retention in Educational Setting
Conclusions
Strengths of the Programme
Areas for Improvement
Learnings for Police and CPU
Section 7: References
Section 8: Appendices
Appendix A: Process Evaluation Programme Manager Interview Guide
Appendix B: Process Evaluation Stakeholder Interview
Appendix C: Process Evaluation Rubicon Trust Representative Interview Guide
Appendix D: Process Evaluation Stakeholder Questionnaire
Appendix F: Outcome Evaluation Alcohol and Drug Counsellor Interview Guide
Appendix G: Outcome Evaluation Stakeholder Interview Guide
Appendix H: Current and Exited Client and Family/ Parent/ Caregivers Interview Guide
Appendix I: Entry Interview With Child/Young Person
Appendix J: Referral Form
Appendix K: Contact Sheet

Top

Download full report