Program Evaluation and Research
EDNA MCCONNELL CLARK FOUNDATION EVALUATION SUMMARY
Excerpt from "Evaluating The Results", Keeping Families Together and Children Safe: Facts on Intensive Family Preservation Services, October 1994.
"With two decades of experience, intensive family preservation has a solid track record. It is among the best-defined services in the field, and research shows it works on both an individual and family level, helping parents learn the skills to keep their families together safely.
Success in the social services world is difficult to measure, however, and researchers have yet to find scientific "proof" that states can successfully use intensive family preservation to lower their overall foster care rates. Part of the problem may be lack of adequately designed evaluations to test the methodology. Another source of difficulty is targeting the appropriate families to receive the service. But it is also genuinely difficult to document the impact of an intervention when each family is different and when placement decisions are made by individual caseworkers and judges. In addition, social and economic factors beyond the control of any social service can have a major impact on the numbers of children removed from their homes.
Evaluation is, therefore, a much-debated topic in the family preservation field. There are three key areas to review 1) safety of children; 2) improved family functioning; and 3) prevention of unnecessary placement and the resulting fiscal benefits of the program. On a subjective level, client satisfaction should also be included.
Although no social service can guarantee 100 percent safety, IFPS programs have a very strong safety record, precisely because of the characteristics of the service itself: trained caseworkers are on the scene within 24 hours of referral and begin delivering services immediately. Services take place in the home, where IFPS workers can help parents minimize the threat of violence. Workers are available daily, around the clock, to provide optimum support and protection for all of the family members. And workers have only two cases at a time, giving them the flexibility to respond to emergencies and the time to stay long enough to stabilize the household when a crisis occurs.
COST-EFFECTIVENESS AND PLACEMENT PREVENTION
The Targeting Conundrum
As the programs matured, IFPS advocates recognized a need for more rigorous, random-assignment evaluations. Does intensive family preservation really reach the families for whom the services are intended-that is, those families whose children would actually have been placed if intensive family preservation had not been available? If services are not reserved for these families, the case for a reduction in overall foster care numbers and an impact on social service budgets is weakened.
Several attempts to evaluate this aspect of family preservation have revealed a targeting problem in states' use of the program. Just as decisions to remove a child can be shaped by the personal criteria of individual caseworkers, decisions to refer a family to IFPS can be equally variable. If, in order to obtain the best services for their clients, caseworkers refer families with children who would not have been removed from the home, then there is no way to determine whether the services would actually have prevented placement if they had been properly targeted.
It is becoming increasingly evident that reducing states overall foster care placement rates requires more than installing a single program to prevent out-of-home care. For IFPS to be successful, administrators must reorient other parts of the system so that the system as a whole supports families and prevents unnecessary breakups. This means at a minimum: creating new fiscal incentives for use of intensive family preservation services; developing new accountability approaches; establishing clear expectations about the use of preventive programs and the need to strengthen families and ensure that stays in out-of-home care are minimized; retraining workers to keep families together and safe; and connecting IFPS to follow-up programs that build on the benefits of an intensive, short-term intervention.
California and New Jersey Evaluations
In New Jersey, where placement was defined to include any stay outside the home, even with other family members and for any length of time, a preliminary report showed significant placement avoidance for up to nine months among families receiving IFPS when compared to a control group. After nine months, the difference diminished and because of the small number of families involved, was no longer statistically significant. The final New Jersey report, released in early 1992, included an additional county and showed that family preservation did result in statistically significant placement avoidance over a one-year period, when compared to the control.
Both of these evaluations had other limitations. They used small sample sizes and were based on relatively new programs. And in California, the family preservation category grouped together HOMEBUILDERS-type IFPS programs with other, quite different, home-based programs.
Six geographic areas around the state were selected to be part of a random assignment study, comparing families receiving family preservation services with those receiving traditional child welfare services. Investigators found little difference in placement rates or reoccurrence of abuse and neglect between the two groups. But because the family preservation programs varied, it is difficult to interpret the findings. And since only 21 percent of the children receiving traditional services were placed the programs clearly missed their target group. Dissatisfied with the results or the various programs it supported under the aegis of family preservation, Illinois decided to start over in Chicago and has specified that family preservation programs in the city must use HOMEBUILDERS' intensive approach.
The Bottom Line
OTHER METHODS OF EVALUATION
The Institute for Family Development is a 501(c)(3) not-for-profit organization.
Copyright 2021, Institute for Family Development, All Rights Reserved