Linking Research with the Practice of Youth Development

Practice Matters, April 2007

A publication of the ACT for Youth Center of Excellence

PDF

by Mary Agnes Hamilton and Stephen F. Hamilton

This paper draws from a working symposium organized by the authors and Larry Pasti, Linking Research to the Practice of Youth Development, held in Albany, NY, September 25-26, 2006. About 50 invited participants represented a cross-section of New York State agencies, community agencies and organizations, intermediary organizations, higher education, and national experts. The keynote speaker was Reed Larson, professor at the University of Illinois and member of the National Research Council and Institute of Medicine Committee on Community-Level Programs for Youth. The NYS Office of Children and Family Services, the Cornell Youth in Society Program, and the ACT for Youth Center of Excellence sponsored the symposium. Larry Pasti contributed substantially to this essay.

Youth development practitioners are increasingly expected to base their programs and practice on research. Evidence-based practice, drawn from medicine and public health, is considered the ideal, yet applying it to youth development is daunting. The amount of research that has been done on critical questions is limited. Moreover, nothing in the practice of youth development is analogous to prescribing medication. The most severe limitation is that, in the health field, evidence-based practice is supported by a comprehensive system for reviewing research, extracting practice guidelines, and communicating those guidelines to practitioners. No such system exists for youth development practitioners. Despite these and other obstacles, the vitality and efficacy of youth development practices require stronger links between research and practice.

Evidence-based Practice in Medicine and Public Health

Nearly all Western medical practices are based on research and theory about the human body and the threats of trauma and disease. What is new about evidence-based practice is reliance on systematic reviews for quite specific guidance. One advantage of such reviews is that they can identify matches between specific conditions and treatments, indicating, for example, which heart attack survivors are most likely to benefit from which statin drugs.

In medicine and public health, evidence-based practice draws on web sites including The Cochrane Collaboration and The Community Guide (see Resources and References at the end of this article for web addresses). These are repositories of systematic reviews of the research literature relevant to specific health topics. At their most elaborate, the reviews take the form of meta-analysis, a technique that combines findings from multiple studies, treating them as parts of one large and very powerful study. Even when data cannot be pooled in this manner, the criteria for including studies in the review are quite rigorous. These reviews are conducted by multidisciplinary teams of eminent scholars, who must agree on the boundaries and methods of the search, the criteria for inclusion, the conclusions drawn, and the implications or guidelines for practice. The methods section of The Community Guide (from the Centers for Disease Control, CDC) gives an overview of this process.

Youth Development Programs and Practices

Policy makers, state and local government, and funding agencies increasingly demand the use of evidence-based practice to decide which youth development practices they should support. Service providers as well have an interest in being able to show that their programs are effective and make a difference. However, systematic reviews of the research and generation of recommendations for practice comparable to those done for health have not been done for the field of youth development. Moreover, the youth development research base is minuscule in comparison to the research in health, which results from an annual investment exceeding fifty billion dollars. Randomized controlled trials (experiments), the core of evidence-based practice in health, are almost non-existent in youth development. Nonetheless, new research supports specific youth development practices and a growing number of youth development programs can claim to be validated by research.

Programs that have been validated by research are not the same as evidence-based practice. Programs and practices are related but not identical. A program embodies a set of practices. Some of the same practices might be found in programs that differ drastically. In medicine, practices include drug treatments, diagnostic procedures, and recommendations such as altering diet and exercise. The equivalent of a program in medicine would be the entire course of treatment delivered in relation to a health condition, such as a heart attack. The term "program" in youth development is commonly used to refer to one-time events. For example, a youth organization teaching public speaking might sponsor an evening "program" in which youth give speeches. We use the term here also to mean a linked set of activities, such as the organization's multi-faceted and continuing "program" in public speaking, or a set of activities run by an organization, such as a school, a teen center, a youth bureau, a church or mosque, or a 4-H club.

In youth development, practices refer to how programs are carried out, that is, what youth do and what adults do with youth. Examples include activating youth voice, demonstrating that adults care about youth, setting and enforcing rules of conduct, and involving parents. Practices may be identified with particular organizations as well, such as schools, juvenile courts, libraries, and boys' and girls' clubs. Obtaining and disseminating evidence about the efficacy of such practices is very challenging. Even when a program has been carefully evaluated and found to be effective for its intended purposes, we may not know which of the practices embedded in the program are most responsible for the program's effects.

Youth development practices may be implemented as components of many different programs. Youth participation in decision making is an example of one best practice. In addition, youth development practices may be part of community-wide initiatives or of whole organizations rather than youth programs per se. The practice of involving youth in decision making can be implemented in different kinds of programs and in different organizations, for example, in faith-based organizations and in community government as well as in youth organizations. Youth development practices are embedded in programs but they may also be separate from programs.

Evidence-based Youth Development Programs

Many youth development programs have been validated by evaluation research. Youth workers and policy makers can now access a growing number of program reviews on the web. Communities That Care is a well-known group promoting prevention strategies for youth problem behaviors. The New York State Office of Children and Family Services' web site links to this and other sources of effective programs and practices.

PROSPER is a major program currently underway to promote and test the adoption of evidence-based youth development programs. Such programs deserve careful attention. Indeed, when practitioners wish to achieve goals that have demonstrably been achieved by "off-the-shelf" programs, they should use those programs in the absence of compelling reasons not to. There are two limitations to the use of these programs. First, most programs that have been carefully evaluated and "packaged" for wide dissemination are aimed at problem prevention and treatment, not at positive youth development. Second, it is often difficult or impossible to implement an entire program with high fidelity to the original, validated specifications. Adaptation of youth development programs is far more common than exact replication. But when programs are not replicated with high fidelity, they cannot be assumed to have the same results. When high-fidelity replication is not possible, we need to consider how to base practices in evidence, not just programs.

Evidence-based Practice in Youth Development

Basing youth development practices in evidence requires research that opens up what happens inside programs and assesses specific practices rather than programs as undifferentiated wholes. The best example of this type of research, which is also the best example of the use of meta-analysis in youth development, was conducted on mentoring programs by DuBois, Holloway, Valentine, and Cooper (2002). The authors pooled the data from 55 evaluations of youth mentoring programs and found statistically significant positive, but modest, effects. They also found that certain best practices, in combination, led to stronger results. Those practices include: continuing training of mentors, structured activities for mentors and youth, frequent contact of youth with mentors, parent involvement, and monitoring of program implementation (pp. 187-188). This review provides guidance for mentoring practices at a more specific level than simply affirming that mentoring programs have positive outcomes. See also the evaluation and research syntheses of youth development programs by Roth, Brooks-Gunn, Murray, and Foster, 1998, and Roth and Brooks-Gunn, 2003a, 2003b.

The most authoritative source for evidence related to youth development is Community Programs to Promote Youth Development (Eccles & Gootman, 2002). The "Features of Positive Developmental Settings" summarized in Table 4-1 (pp. 90-91) convey a sense of the kinds of practices (which they call "features") that research demonstrates promote youth development. The seven features are: physical and psychological safety; appropriate structure; supportive relationships; opportunities to belong; positive social norms; support for efficacy and mattering; opportunities for skill building; and integration of family, school, and community efforts. However, the question of scale remains. For example, once we know that "supportive relationships" are beneficial, youth development professionals must still take many steps to design programs that operationalize these features in real-life settings, to enable adults to act in a manner that yields supportive relationships.

"Efficacy and mattering," also identified by Eccles and Gootman (2002) as a key feature, refers in part to practices of youth development known as youth participation or youth voice. "Efficacy" means the sense of being able to get things done, to act on one's own. "Mattering" means feeling that one's actions make a difference and that one is valued by others. Youth can experience efficacy and mattering in many different situations. There is no single way to operationalize this feature. There is widespread agreement that youth development means young people should have every possible opportunity to participate in decisions, not only about their own involvement in youth development programs and activities, but also in shaping those programs and activities. We are strong advocates of this principle, yet we recognize that it is based more on belief than on evidence. Research has begun to work out just how young people participate in decisions and whether and in what ways they gain developmental assets from the experience. See also Pearce and Larson (2006) and Zeldin, McDaniel, Topitzes, and Calvert (2000).

Strengthening the Links between Research and the Practice of Youth Development

Strengthening the links between research and the practice of youth development cannot simply await an adequate research base and an effective system for reviewing that research and disseminating the findings. Rather, the field must proceed as far as possible under the circumstances. Following are some recommendations for that purpose.

  1. Evaluation research is critical to the accumulation of a research base in the field. Evaluation is typically so meagerly funded and planned so late in the program development process that results of evaluation research have little validity and less impact. High-quality evaluation research is the best source of evidence for the field. More and better evaluations are essential, as is better dissemination of evaluation findings.
  2. Participatory action research is especially appropriate to youth development because it extends the principle of youth participation to evaluation. Practitioners and youth should participate fully as partners in research projects, ideally in all steps of the process. Too much remains to be known to leave all the work to researchers. Moreover, much wisdom resides in those who live youth development day-to-day.
  3. Researchers and practitioners should collaborate to develop better means of reciprocal communication. Practitioners need to know more about what research has found, but researchers also need to know more about what issues practitioners struggle with that research might help to illuminate.

Resources and References

Following is a list of the works cited plus additional youth development resources that link research with practice.

Web sites

Publications