What Works Literacy Partnership: Making Data Work For You
Literacy Partners, New York, NY
Adult basic education programs
collect large amounts of data. In many instances programs tend to
collect more data than they know what to do with. Most often this
data is used for reporting purposes and has limited impact on a
program's day-to-day operations. However, as adult educators we
recognize that accurate, complete data is essential to remaining
a viable and credible organization. We also know that it takes time,
financial support, committed personnel and patience to create a
data system that informs and is fully integrated into an agency's
The What Works Literacy Partnership
(WWLP), led by Literacy Partners in New York City, was founded in
1996 with a grant from the Wallace-Readers Digest Funds. It brought
together 12 exemplary adult literacy programs from across the country
who were interested in building their capacity to collect, manage
and analyze data, before results and accountability became the driving
forces behind educational policy in the United States. WWLP represents
a discerning and proactive response by national leaders in adult
education, the Wallace-Reader's Digest Funds and local programs
to launch a state-of-the-art effort to dramatically improve the
ability of adult education programs to collect data and report on
student achievements. Over the past five years these programs have
worked diligently to identify effective practices that lead to using
data for program improvement and decision-making. These programs
believe in the fundamental power of quality data.
The programs that comprise the Partnership represent the diversity
of adult education providers. They are urban, rural and somewhere
in between. Budgets range from $250,000 to $4 million. Together
the partners engage the services of 1,837 volunteers and employ
270 paid part-time and full-time teachers. The programs include
eight that are community-based, two school district-operated, and
two community college-based. They are located in Illinois, Virginia,
Pennsylvania, North Carolina, Massachusetts, New York, Wisconsin,
Arizona, Ohio, Vermont and Montana. The total number of students
served by Partnership agencies exceeds 28,000.
For the past five years the WWLP Partners have been on a journey,
sometimes frustrating sometimes exhilarating. When Partners joined
WWLP they were at various stages in their data collection practices.
Some had been collecting data for years as part of centralized urban
networks and had sophisticated management information systems in
place. Others lacked even rudimentary databases and many had no
uniform assessment practices. All were committed, however, to making
substantial changes in their programs and to improving their approaches
to collecting, using and analyzing data. They were eager to find
out how their students were doing by employing more uniform assessment
practices; they were willing to administer standardized tests as
well as performancebased measures. Furthermore, they were convinced
that, if they began asking better questions about their programs
and collected appropriate data, they would uncover new and vital
information that could lead to improved instructional and administrative
practices. The challenges that each program has faced have been
uniquely its own. The "lessons learned" as a result of this collaborative
effort, however, have begun to create a remarkably consistent picture
of just what it takes to make data "work" for an agency.
As WWLP enters its final year, the Partners are writing case studies
to document and share their "lessons learned" with the field. One
"Prior to joining WWLP we had been providing educational services
to adults seeking to increase their basic academic skills, increase
their English language proficiency and rind sustainable employment.
We were providing these services largely without a system of student
assessment, which resulted in inaccurate data, no reliable system
to asses the cost of specific outcomes, and few opportunities
to reliably promote and advocate for the organization in terms
of its efficiency and effectiveness.... Today our program has
worked quite diligently to create a system of assessment that
meets external requirements yet is flexible enough to accommodote
the needs of staff and students. From the identification of assessment
tools, to training instructors, to revamping the database system
entirely, to creating all new forms that capture essential information,
we believe we are now better equipped to meet the challenges that
adult education faces in the 21st century."
The What Works Literacy Partnership represents a unique model of
professional development that merits examination and replication.
Partner programs have had opportunities to receive training from
expert researchers and educational evaluators; they have been able
to experiment with the development of their assessment systems and
have had the benefit of sustained interaction with colleagues around
shared goals. Another Partner states,
"Our early association with the What Works Literacy Partnership
brought the issues of assessment and evaluation to the forefront.
By interacting with other agencies throughout the country, we
were able to see the benefits that involving teachers and learners
in creating a formal assessment process would have. The importance
of systematizing and standardizing assessment processes was revealed
through our interaction with the partners and from the expert
training that we received. We learned from WWLP the importance
of asking the right questions and analyzing the correct data to
present a rich and detailed picture of our agency, its programs
and accomplishments to funders, trustees, staff, and learners."
WWLP has identified key findings for developing assessment and
data collections systems, including:
Understanding the multiple purposes for assessment -- including
documenting program impact, finding ways to improve programs
and monitoring individual student progress -- will help in designing
an effective system;
Involving staff in every stage of assessment and achieving
staff buy-in are key ingredients to the success of any evaluation
and assessment plan;
Investing in staff development is essential;
Administering standardized assessment measures correctly yields
valid and reliable data;
Program managers, students, tutors, funders and policymakers
share the responsibility to provide high-quality adult literacy
programs and to gather the evidence necessary to demonstrate
that these programs acutally work;
Students need to be involved in and understand each phase
of the assessment process; and
Asking good questions and gathering good data enable a program
to analyze successes and to make improvements when necessary
Out of the WWLP effort will come project materials that can help
others design effective and efficient assessment and evaluation
plans. The products include:
Self-Assessment Survey of Agency Resources and Skills
This instrument is designed to assist programs in identifying
current data collection procedures and areas that need improvement.
Indicators of Data Proficiency: Three Stages of Growth
This model identifies three levels of program proficiency with
corresponding descriptors assigned to each level. Programs can
use this document to assess what systems and practices are currently
in place that support the collection and to determine what needs
to be done to move the organization to the next level.
Data Bytes Guide Sheets
This series of information sheets answers the most frequently-asked
questions about data collections, management, and analysis.
Sample sheets respond to questions such as: How do I build a
data collection system? How can I train teachers and tutors
to collect data? How do I involve students in data collection
Model Data Reports
Sample reports from WWLP agancies will provide models for effectively
using data to tell an organization's story.
Each WWLP agency tells the story of how it resolved an issue
related to data collection and assessment. They represent the
"lessons learned" from the field.
The past five years have been an exciting time of discovery and
challenges for the WWLP initiative. Through a process of support,
training, and experimentation, the partners have developed a broad
body of knowledge about what it takes to build data collection systems
and develop effective assessment practices. Agencies have been able
to identify the skills they need and to focus on honing those skills.
They have learned what they can do with the new knowledge they have
gained and are sharing that knowledge with funders, policymakers,
and adult educators across the country.
The case study that follows describes
how one WWLP partner -- Literacy Volunteers of America-Chippewa
Valley -- identified a challenge and worked together to find solutions.
For more information on the What Works Literacy Partnership or on
its upcoming publications, contact 212-802-1113 or go to its newly-designed
Web site at: www.wwlp.org
Originally published in Adventures in Assessment,
Volume 13 (Spring 2001),
SABES/World Education, Boston, MA, Copyright 2001.
Funding support for the publication of this document
on the Web provided in part by the Ohio State Literacy Resource
Center as part of the LINCS
Assessment Special Collection.