Why use pre-post surveys?The goal of evidence-based programs (EBPs) in adolescent sexual health is to change behaviors that lead to unhealthy or unwanted outcomes. How do we know that what we are doing is working? Unless we actually measure change within individual participants, we can only make educated guesses. Surveys that ask the same questions of students before and after the program content is delivered can give some indication of program success.
What questions are asked on the CAPP pre-post surveys?CAPP surveys ask about past behaviors as well as intent to have sex, use a condom, and use birth control. The CAPP pre-survey is slightly different from the post-survey; a few questions about experience are asked at the start of the cycle, and participant satisfaction questions are asked at the end. However, most questions appear on both surveys.
Note: The surveys ask sensitive questions; it is important for youth participants to understand that their participation is voluntary, and their responses will be kept anonymous. Before they administer surveys, educators need to become familiar with the script regarding informed consent (see "Informed Consent" below).
When and with whom should I use the surveys?
- The pre-survey is given to participants by the educator at the beginning of the first session of the cycle, and the post-survey is given after the last activity in the EBP is completed.
- CAPP pre-post surveys are not used in every cycle. Your ACT for Youth evaluation contact will work with you to determine the CAPP cycles in which you should use surveys.
Why are we using electronic surveys?Surveys are completed using secure URLs or QR codes that link to Qualtrics. Participants must complete surveys on a device that is connected to the internet. Collecting data electronically speeds up the data collection and analysis processes while maintaining confidentiality and reducing costs. This means ACT for Youth can more quickly summarize findings for more providers.
For questions regarding survey administration, including internet/device access, URL/QR code distribution, and survey data reports, please email your evaluation contact in your TA Support Team.
Will schools and other host organizations allow us to use the survey?Usually, if a site administrator has granted permission for a provider to implement an EBP, he or she is likely to be interested in the results of the surveys. Before you begin, be sure to secure permission in writing from the school principal, program director, or other authority. There are detailed steps for this process below. The survey asks sensitive questions, and it is important that school and program administrators understand and support this process. See "Obtaining Permission" below for ideas on how to approach administrators.
Administering Surveys: An OverviewAdministering the surveys takes planning and time -- time that may already be tight in any given program cycle. Preparation is key.
Educators administer surveys to participants through URL or QR codes on a device that is connected to the internet. Here are some points to consider when preparing to administer surveys:
- Do participants have internet access?
- Do participants have cell phones?
- Do program facilitators have an electronic method to contact participants in order to send links such as email, social media messaging, or texting?
- Tablets previously purchased for survey administration can be used to access survey links as long as they are connected to the internet.
- Mobile hotspots are an option to help with internet access - please contact your DOH program advisor for more information.
It is important that youth program participants know why you are asking them to complete this survey. Be sure to verbally state this information and check to see if anyone has questions. Here is a sample script for informed consent:
"We want to learn more about how well this program works, and that means we need to ask you, the experts, about how your thoughts and opinions may have changed from the beginning to the end of this program. I'll ask you to take a short survey now and again on the last day of the program. We want to learn how your thoughts and behaviors may be different at the end of the program. Both surveys will take just a few minutes. Both surveys are anonymous. That means your name does not go on the surveys and we will never be able to link your answers to you. I will not know which answers are yours, and your teacher/program leader and parents will never see your responses. Your answers will be combined with those from other young people. Both surveys are voluntary. While we hope you will finish both of them, you do not have to participate. You can skip any question you do not want to answer. The information you tell us will be used to help make this program better in the future."
In order to conduct the surveys, you will need permission from the organization that is hosting the EBP, and you may also need to allow parents to opt their children out of the survey.
When introducing the survey, be sure to let administrators know that this survey and protocol have gone through rigorous review processes by Cornell University's Institutional Review Board and the New York City Department of Education Institutional Review Board. These boards have high standards for protecting the privacy, rights, and welfare of participants in research and evaluation projects, and they have approved this survey and protocol.
You may wish to emphasize:
- CAPP is a statewide initiative; survey responses from your sites will be combined with those from around the state as part of the evaluation of the CAPP initiative.
- Surveys are anonymous. Names are never collected with surveys. Program facilitators will not see individual survey responses, only aggregate data.
- Individual participation is optional. Parents and youth participants can decide NOT to take the surveys. Youth participants may also skip any question on the surveys.
For cycles in sites OUTSIDE New York City public schools:
- Discuss the use of surveys with the site administrator or principal. ACT for Youth has developed a template you can use to initiate this conversation or to draw talking points from. The letter may be sent with the parent information/opt-out form below. Feel free to modify the template to fit your needs.
- Send documentation of the site administrator's or principal's permission to use the survey to your ACT for Youth evaluation contact. This can be as simple as a forwarded email exchange. This is required.
- If desired by the site, send the parent information/opt-out form (English; Spanish) home prior to survey administration. If a signed form is returned, be sure the youth participant returning the form does NOT complete surveys. Note: a youth participant without permission to complete surveys may still participate in EBP programming, if permission to attend the EBP was granted.
For cycles WITHIN New York City public schools:
In New York City public schools, the New York City Department of Education (NYC DOE) requires additional steps.
- Ensure all educators who will administer the surveys have been fingerprinted by the NYC DOE. Any educators who have not yet been fingerprinted by the NYC DOE will need to submit copies or photos of both a signed social security card and driver's license to the Institutional Review Board (IRB) Personnel Eligibility Tracking System (PETS) system via email (PETSAdminSupport@schools.nyc.gov). The PETS team will enter the educator's information into the IRB system and the educator will then receive a nomination email within 24 hours. All instructions will be in the nomination email. If you have any questions, please contact Vicki Baum.
- Send documentation of fingerprinting to the NYC DOE Institutional Review Board. This is required.
- Discuss the use of surveys with the principal. ACT for Youth has developed a template you can use to initiate this conversation or to draw talking points from. Feel free to modify this document to fit your needs. Use of this document is optional.
- Have the principal complete the approval to conduct research form (Word), making sure that the school ATS code is included. Send the completed form to your ACT for Youth evaluation contact.
- Send the parent information/opt-out form (English; Spanish) home prior to survey administration. This is required. If a signed form is returned, be sure the student returning the form does NOT complete surveys. Note: a student without permission to complete surveys may still participate in EBP programming.
Data, Reports, and Feedback
Once an organization has reached a threshold of 50 completed pre- and post-surveys, ACT for Youth can create a report from the results. For questions about survey reports, or the number of surveys completed at your organization, please reach out to your evaluation team contact.