- implement multi-dimensional educational, vocational, economic, and recreational opportunities for youth on multiple health and development topics that introduce them to new situations, ideas, and people, and challenge them to build or learn skills; and/or
- implement mechanisms to refer individuals to other federal, state, county, city, school district, and local community service providers for physical, social, emotional, educational, and developmental support and services as necessary.
Best Practices for Youth Development ProgrammingEffective youth development programs build competencies in many areas, increase self-efficacy, expand opportunities and recognition for youth, and increase healthy bonding with others. Generally, research has established that effective interventions last nine months or longer. These findings are reflected in the features of effective youth development settings, which currently serve as a guideline for youth programs.
To help CAPP agencies design effective Component Two programs, Jutta Dotterweich presented findings from youth development research in a learning community meeting in 2017:
Webinar Recording (4/17/17)
Outcome MeasuresWorking together, CAPP agencies and ACT for Youth developed logic models to narrow down the outcomes we are trying to achieve, which enabled us to identify the core outcomes we will measure: self-efficacy, healthy decision making, and youth-adult connectedness. These three measures appear to be relevant to Component Two proposals across sites and will be used to evaluate efforts.
Webinar Recording (6/16/17)
Webinar Recording (11/16/17)
EvaluationComponent Two programs that meet certain criteria will be evaluated using pre-post surveys. CAPP Component Two youth surveys ask questions about self-efficacy, healthy decision making, and youth-adult connectedness. By asking these questions before the program begins and again after the program ends, we hope to identify positive gains in these core outcomes for youth participants.
Component Two Survey Questions (sample only)
Permission must be secured from each program site before the surveys can be administered, and passive consent from a parent or guardian may also be necessary (as described below). Surveys will be administered using electronic tablets and a process that guarantees anonymity.
Selecting a Program for Evaluation
It's important to note that not all Component Two programming needs to be evaluated. Each agency will select a program for evaluation that fulfills the following criteria:
- A group of youth attends programming for a minimum of two months (8-12 sessions). Shorter programs are excluded because a program of longer duration is more likely to achieve Component Two outcomes than a shorter, more concentrated program. For example, a one-week program that meets five hours a day is likely to be less effective than a program that meets once a week for two hours over several months.
- Programming is focused on skill building.
- The same program leader is present for all program sessions. A supportive, stable relationship between youth and an adult program leader is a critical feature of effective youth development programming.
Webinar Recording (5/18/18)
Usually, if a site administrator has granted permission for you to implement a program, he or she is likely to be interested in the results of the surveys. Before you begin, be sure to secure permission in writing from the school principal, program director, or other authority.
For programs in sites OUTSIDE New York City public schools:
- Discuss the use of surveys with the site administrator or principal. ACT for Youth has developed a template you can use to initiate this conversation or to draw talking points from. The letter may be sent with the parent information/opt-out form below. Feel free to modify the template to fit your needs.
- Send documentation of the site administrator's or principal's permission to use the survey to your ACT for Youth evaluation contact. This can be as simple as a forwarded email exchange. This is required.
- If desired by the site, send the parent information/opt-out form (PDF) home prior to survey administration. This is a "passive consent" process: parents will need to return a form only if they are opting their children out of the surveys. If a signed form is returned, be sure the youth participant returning the form does NOT complete surveys. Note: a youth participant without permission to complete surveys may still participate in programming.
In New York City public schools, the New York City Department of Education (NYC DOE) requires additional steps.
- Ensure all educators who will administer the surveys have been fingerprinted (PDF) by the NYC DOE. (Fingerprinting by other organizations is not accepted.)
- Send documentation of fingerprinting to the NYC DOE Internal Review Board. This is required.
- Discuss the use of surveys with the principal. ACT for Youth has developed a template you can use to initiate this conversation or to draw talking points from. Feel free to modify this document to fit your needs. Use of this document is optional.
- Once approved, have the principal complete the Approval to Conduct Research Form and send it to your ACT for Youth evaluation contact. This is required.
- Send the Component Two survey parent information/opt-out form home prior to survey administration. This is required. If a signed form is returned, be sure the student returning the form does NOT complete surveys. Note: a student without permission to complete surveys may still participate in programming.
Purchasing and Preparing Tablets
If you are already using tablets for CAPP Component One pre- and post-surveys, you can simply add the Component Two survey to your list of available surveys.
If you still need to purchase or set-up tablets, ACT for Youth has equipment purchase recommendations (PDF) for everything from the type of tablets to the bag to transport them.
- Each tablet must be set up with a) the Qualtrics Offline Survey app, a free download from app stores, b) minimal access to other apps, and c) parental controls. This needs to be done only once. This document provides instructions for setting up Kindle Fire tablets for survey use (PDF).
- Be sure the tablets are charged and any necessary updates are completed the day before using them to collect survey data.
Administering the Surveys
Administering the surveys takes planning and time -- time that may already be tight in any given program. Preparation is key.
Well before surveys are to be used, educators should review the guidance for survey administration (PDF) so that they are ready for a smooth process. For example, it is important for youth participants to understand that their participation is voluntary and their responses will be kept anonymous. Before they administer surveys, educators need to become familiar with the script regarding informed consent, which is highlighted in the survey administration document.
The survey will be completed twice by Component Two youth participants: once at the start of programming, before any content has been delivered, and again after the last program activity. No participant names will be recorded on the survey, so it is important to assign and keep track of a unique ID for each participant. (This ID needs to be entered at the start of each survey and should be unique for EVERY participant; this ID CANNOT be repeated.) We will use the unique participant ID and the time the survey was completed to identify which surveys were completed before programming, after programming, and which surveys were completed by the same person.
Uploading Survey Responses
Once you return to your home organization and tablets are connected to Wi-Fi, upload survey responses recorded in the Qualtrics Offline Survey app. Note that neither educators nor ACT for Youth staff will see names associated with survey responses at any time in this process. Launch the Qualtrics Offline Survey app and tap the button to upload surveys.