About Outcomes-Based Evaluation
Outcomes-based evaluation is a powerful tool that can help you tailor your programs to the needs of your community, report the difference you make, and improve library service. Outcomes-based programs are designed with specific outcomes in mind, which helps you reflect on the reasons you offer summer reading, the type of summer reading programs you plan, and the results you want to achieve with your programs. In addition, outcomes-based programs incl ude an evaluation component which enables you to capture data about the impact of your work.
Outcomes evaluation is the process of identifying desired outcomes from a particular program or service and then conducting some kind of data collection to determine if those outcomes have been achieved.
Building on the output data that most librarians collect about their summer reading program, outcome measures answer the question: “so what?” What changes in attitude, knowledge, skills, or status happened to your patrons as a result of taking part in the summer reading program? Outcome evaluation is most effective when it is part of a systematic planning process. Therefore, the California Summer Reading Outcomes Project is based on the four-step process that is outlined in the book, Dynamic Youth Services through Outcome-Based Planning and Evaluation (American Library Association, 2006), by Eliza T. Dresang, Melissa Gross, and Leslie Edmonds Holt:
- Gathering information.
- Determining outcomes.
- Developing programs and services.
- Conducting evaluations.
Each step builds on the step before. First you gather information from and about your users that will help you determine what their needs might be for a particular service. Based on what you learn, you decide the particular need the library can fulfill and express that need in terms of one or more outcomes as defined above. The third step is to plan and implement a program or service designed to achieve the desired outcome for your patrons. Finally, you conduct an evaluation study to find out if you really did achieve the outcomes.
It takes more thought, time, and effort to determine outcomes than it does to document outputs. Among the outputs of summer reading programs are numbers of people who sign up, who attend programs, the number of books read during the summer (or the weight of all the books read in the summer). It is really just a matter of counting.
Outcome measures are a little trickier because you are trying to figure out what change happened to an individual as a result of some service or program or intervention provided by the library. Usually the only way you can know this is to ask the person to tell you. Therefore, the typical methods used to collect the data that yields outcome measures are those used by social scientists: pre- and post-tests, surveys, interviews, and focus groups.
Libraries often include a budget for evaluation in large grant-funded projects. This allows them to bring in an outside consultant with special skills in this area. Increasingly, however, libraries are learning how to do it themselves and applying simple outcome measures to their ongoing services and programs. And this sums up the California Summer Reading Outcomes Initiative.