Ethically and Culturally Sensitive Evaluation Plans
This discussion offers you an opportunity to identify the ethical and legal issues that will be most relevant to my program evaluation. Address how I will meet the guidelines adopted by the American Counseling Association, the American Evaluation Association, and other applicable ethical and legal standards, as you gather information for your evaluation.
Additionally, consider how you will evaluate the cultural sensitivity of the program by examining the following questions. What data will you need to collect in order to answer these questions?:
1. Are the intended services being delivered to the intended persons? Do particular groups find the services problematic because of their cultural values or practices?
2. Are there needy but unserved persons the program is not reaching? What is known about these unserved persons and why the program is not reaching them?
3. Once in service, do clients complete service? Who drops out, and why? Do particular groups drop out more often than other groups? Could this mean that the services are not sensitive to their cultural needs?
MY PROGRAM IS LISTED BELOW
Program Evaluation & Logic Model:
The SMART Recovery Teen & Youth Support Program
According to Wall (n.d.), "An evaluation is a purposeful, systematic, and careful collection and analysis of information used for the purpose of documenting the effectiveness and impact of programs, establishing accountability and identifying areas needing change and improvement." There are many ways to do a systematic evaluation and the kind of evaluation used is influenced by the purpose, timing and procedures used in the program. These include summative evaluation (analysis of outcome) and formative evaluation (analysis of specific aspects of the program while it is being implemented to see about necessary improvements), among many others. Then there are specifically designed evaluations like the accountability-oriented program evaluation proposed by Chyung, Wisniewski, Inderbitzen, & Campbell (2013). This can be used together with various approaches (perspectives used by evaluators when they undertake an evaluation - think of it as a kind of 'lens') like the empowerment evaluation by Fetterman and the responsive evaluation by Stake. For the purpose of this evaluation, I feel that Robert Stake's responsive evaluation is most appropriate since the program I have chosen to look requires 'response' in terms of its program activities in relation to the program's impact in helping teens and the youth recover from addiction.
The Program: SMART
Smart Recovery is a non-profit organization established and run by volunteers since 1991 to help people recover from various addictions and to support and advocate for them in the process. While initially geared for adults, a program was established to focus on specific issues on teens and the youth as the problem of addiction and substance abuse have tragically come to include this particular age group. An important element of the program are group counselling and meetings which are designed to be informational and supportive. The organization now has operations worldwide, holding 1,500 weekly group meetings outside of daily online meetings in virtual environments to support people struggling with addictions. SMART is especially active in the US, UK, Canada, Sweden and Denmark. According to SMART Recovery (2015), their teen and youth recovery program aims to help teens and the youth take control of their lives by learning new ways of coping and managing themselves so that they control their behavior. Smart Recovery (2015) explains, "SMART Recovery is a self-help program that offers a place where teens can get together to try to look into and change behaviors that hurts themselves and others like smoking, drinking, fighting and using drugs, to name a few." It is a 4 point program centered around group counseling to build and motivate the youth, help them cope with urges, manage thoughts, feelings and behaviors and reach a point of sobriety to establish goals, find healthy interests and achieve a balanced life.
Program Evaluation
Program link - https://www.smartrecovery.org/teens/
Following Chyung, et. al. (2013). where the evaluand is the thing under evaluation, I have identified and provided insight on the rest of the program evaluation steps suggested by Chyung, et. al. (2013):
1. Evaluand - the SMART Recovery Teen & Youth Support Program
2. Purpose - the evaluation is worth-focused following the responsive evaluation approach because of the extrinsic value of the program to society since it is volunteer-ran with the organization being non-profit. All involved take responsibility for specific roles not for monetary gain but to contribute to the betterment of others - in this case to help teens and the youth recover from addiction following a simple 4-point program.
3. Stakeholders - the teens/youth, their families and community (downstream) and the Smart Recovery NGO (upstream)
4. Program logic model (see simplified logic model attached)
5. Dimensions - this is largely focused on particular processes and their importance
6. Evaluation methodology - this is largely text-based from the text resources that can be derived from SMART Recovery's website
7. Validity & reliability of measure - being text-based, validity efforts will focus solely on finding descriptive or quantifying information on the processes used in the program, while reliability efforts will focus on finding similar information across similar cases and if they measure or present the same function (processes)
8. Data Collection - data/information collected from the SMART Recovery website showed that processes of importance are simplified within the 4-point program aimed at behavior changes.
9. Data Analysis rubrics - content alignment (processes aligned with the 4 point program for SMART Recovery for adults with a good grading), participation (teen and youth participants were enthusiastic and took part grading it good), retention (due to the problematic nature of adolescence, retention is a problem in some group counselling activities, rating it good in the beginning and barely adequate in the end) and leadership (program leaders provided good leadership while leadership qualities among teen participants are barely adequate even when some participants did show initiative)
10. Synthesis - of the 4 point system, I feel that all areas or points of the program are essential but what is fundamental in terms of the 'counseling' process is Point 1 - where those participating through group therapy employing CBT via 'talk' establish their reasons and motivations to get over their addictions. Without this process, the participants are not anchored to the program in terms of commitment to recover.
Logic Model (see attached document)
Word Count:
Resources:
1. Chyung, Seung Youn, Wisniewski, Annette, Inderbitzen, Brad and Campbell, Debbie (2013). " An Improvement and Accountability Oriented Program Evaluation: An Evaluation of the Adventure Scouts Program," from Performance Improvement Quarterly, Vol. 26, no. 3, pp. 87-115. Wiley Online Library. URL (Abstract): https://onlinelibrary.wiley.com/doi/10.1002/piq.21155/abstract
2. Wall, Janet. (n.d.). Program Evaluation Model 9-Step Process. Sage Solutions. URL: https://region11s4.lacoe.edu/attachments/article/34/(7)%209%20Step%20Evaluation%20Model%20Paper.pdf