Assessment tools in the multi-level ESOL classroom

The More Things Change, the More They Seem to Stay the Same

Maria Elena González
Adult Literacy Resource Institute, SABES, Boston, MA

Last Fall I returned to the classroom for the first time in five years. My
absence from the classroom correlates with my time in staff and program development, during which I undertook a different kind of teaching, that of presentations or trainings for my peers.

During that time I thought a lot about what constitutes good teaching and learning while observing and talking to teachers. The conversation had been getting a little too abstract for me, however, a bit too removed from what I had actually experienced in the classroom. So I chose as my own staff development project for the year to teach ESOL once a week at a small community-based organization near my home.

The English Class
“La clase de Inglés,” as my students referred to the class, was entirely composed of women, mostly from the Dominican Republic. They averaged about eight per night, but always began at about 20 during the first weeks of each cycle. A wonderful thing about this class was that it offered childcare to each woman (or man) who came. The class was never designed to be just for women, but available, no-cost child care was definitely a big draw for the immigrant women who lived nearby.

A feature of the class that had attracted me initially also proved a challenge as time went on. The class was offered twice a week but the teaching was shared between me and another teacher. I taught one night and she taught the other, which worked wonderfully for our schedules, but made
it challenging to plan classes together. We did meet regularly with the counselor, to talk about the students and to plan initial assessment at the beginning of each cycle.

Initial Assessment
The first planning session was at the ALRI library where I offered a sample of various assessment tools and past copies of Adventures in Assessment. I also had samples of tools I had used as an ESOL teacher, including one I had designed for an advanced class that never took place. This self-assessment was modeled on one developed by teachers at Read/Write/Now Adult Learning Center in Springfield, Massachusetts. I always liked how their assessment is based on principles of whole language and referred back to it several times over the years. It is not so much one tool, but a series of tools that are done over a whole cycle of classes. (You can find it and an accompanying article about it in Adventure in Assessment, Vol. 1.)

The version I developed focuses on a learner self assessment of the different ways s/he may be using English across the four major skill areas of comprehension, speaking, reading and writing (see “ESOL Assessment” and Self Assessment Tool at the end of this article). The learner not only lists those areas in which s/he has difficulty, but also thinks about the ways in which s/he already knows and uses English.

In the first session, for example, the learner fills in the following sentence: I understand English at (work, my children’s school, health clinic, etc.) by checking off one of four categories: a little, some, a lot, need more.” The same is true for the skill areas of speaking, reading and writing. The student is then invited to look over what s/he has checked off and notice the different ways s/he may already understand, speak, read, and write English.

After the student has a sense of his/her areas of strength, then s/he chooses one area from each skill category to work on in class. For example, a student may choose to focus on “understanding English at church,” “speaking at the children’s school,” “reading in English for a job application,” “writing notes to teachers.” The end goal is that the learner will come up with an individual plan for the class.

Mixed Results
As I write this article and look over the assessment forms filled out by the learners, I realize that they did give us some of the information we wanted, such as literacy levels and a broad survey of the learners’ use and understanding of the English language. Yet I was disappointed because it failed to give us and the learners, clear and definitive information.

First, the design of the tool is a bit unwieldy: it’s too long and has too many crooked lines for the learners to navigate easily. Second, even though I explained each category in Spanish and invited the women to answer in Spanish, it was still difficult for the learners to understand how the form was organized. It is somewhat artificial to separate listening and speaking skills, especially for beginning students. It was even more difficult to differentiate between “a little,” “some,” “a lot,” and “need more.” As an ESOL teacher, I usually don’t get into adverbs of frequency until well into a cycle and normally not with absolute beginners. The difficulty in this case was compounded by the fact that “poquito” and “algo” (“a little” and “some,” respectively) mean the same thing in Spanish.

Clearly, this tool would work better with more advanced learners, for whom I had designed it in the first place. Even with an advanced group, though, there needs to be plenty of time allotted for learners to fill it out, ideally with the help of more than just one teacher. The learners need someone to walk them through the instrument and to talk to them about what they check off and why.

My co-teacher and I still used the results of this initial assessment tool to learn about each student’s literacy levels as well as to get a general idea about which topics learners were interested in. For the end-of-cycle evaluation, however, I wanted a tool that would be more interactive and dynamic, something that would jog people’s memory about what had actually taken place in my Monday class.

Top

End of First Cycle Evaluation
On newsprint paper, I wrote the theme for each class next to the date it occurred. Samples of the lesson were stapled on the left hand side. Each student was given five stars to place next to their favorite lessons. I left space on the right hand side for written comments which they could write in Spanish if they wanted. Once I explained the process and gave them their stars, I left the room for as long as they needed.

The women liked the task and gave me some predictable as well as unpredictable comments. For example, they rated highly a couple of lessons that I thought had bombed. They also wrote some positive comments about me as the teacher, which is typical of many ESOL classes, but those comments were also peppered with some positive criticism, such as “I would like for the next class to start like if we were kids, starting with simpler sentences, at a lower level.”

When I asked for clarification of their comments, a couple of learners told me I had gone a bit too fast for them, that I erased things too quickly from the board, and that they wanted more traditional types of lessons. Clearly, doing the evaluation had given the learners the space and perhaps even the confidence of telling me what they needed to learn better. The ultimate, albeit small sign of their progress was the one comment actually written in English: “I liked is (sic) class because, se compartia (we shared)…”

Second Cycle
For the initial assessment of the second cycle, we decided to totally change gears from what we had done in the first cycle. This time we decided to separate the assessment into three distinct parts: the intake (with the personal information), the literacy, and the goal setting. For the first two parts we used an assessment tool developed by the state of New York for use in their adult basic education ESOL classes. It comes in various languages and tests literacy through a series of short readings in English at various levels. After reading them, learners are encouraged to write a short paragraph about themselves. Unfortunately, doing the personal piece and the goal setting (more about that later) took all of the first day, and the following class was cancelled due to a holiday. We lost momentum and never went back to the reading piece.

We did get a small flavor of what it would be like to do the entire assessment from one learner who had done the Spanish version. She actually wrote an entire paragraph where she revealed she only had a second grade education (she had left that question blank in the first part of the assessment). She wrote:

“si llo quiero aprender por que me es mui dificil cuando sargo a buscar trabajo. cuando voi a los apoime que tengo que usar una tercera persona isi no aparese interprete. me sucedio unabes, tube tres hora es perando por un interprete. me asido dificil por mi nibel de estudio segundo de primaria gracia”

(“yes, I want to learn because it has been very hard when I go looking for work. When I go to “apoime” (appointment) I have to use a third person if there is no translator. It happened that one time I was three hours waiting for an interpreter. It has been hard because my level of studies is second grade primary. Thank you.)

In this one example, we can see how the tool is meant to work from soliciting personal data to actually gauging reading and writing levels.

Breaking the Ice, Setting Goals
For the goal setting, I decided to do a variation on a “breaking the ice” group activity Elsa Auerbach used a long time ago and which I have used many times since. The learners split into two lines facing each other. I brought in my tape recorder with contemporary African music and told the students they had to walk or dance moving to their left until I stopped the music. Then, and only then, could they stop and face the person in front of them. They had to interview the other student and ask them their name, their hometown (since most were from the Dominican Republic), how many years they were in this country, and why they wanted to learn English. I told them to pay special attention to the latter question since they would be asked to introduce me to one other person at the end of the activity. It was a lot of fun to see the women dancing, including the grandmothers, to the beat of the infectious African beat. I let it go on for a couple of rounds and when we stopped, I asked each learner to introduce me to one of the people they had talked to and tell me why they wanted to learn English.

As each woman introduced another, I wrote what each said on newsprint. We went over it the next class, adding the voices of those who had not been present. I then asked the learners to look for common themes or goals in the list of reasons why they wanted to learn English. It took us a while to make the distinction between general goals like “I want to learn English because I like it,” to more concrete goals like “I need English when I go to the doctor.” We came up with a short list that included doctors, children’s schools, work, shopping, and citizenship tests. The group finally decided to start with doctors as a class topic for that cycle.

Top

Cycle Evaluation
The last day of the cycle was suddenly upon us and the agency organized a celebration to which members of the Board of Directors were invited. I had to design something that could be completed in fifteen minutes or less, so I asked myself what did I really want to know? What questions would give me the bare bones of information I needed to evaluate the class and plan for the next cycle? The questions also had to be as simple and direct as possible to minimize the amount of explaining I would have to do. The following questions, typed out in big print on a single sheet, fulfilled those requirements with fairly good results:

1) This cycle I learned:

2) I used my English at:

3) Next cycle I want to learn about:

4) The teachers help me to learn when they:

5) The teachers do not help me to learn when they:

For the first question, I asked them to be as specific as they could, encouraging them to write down anything at all that they remembered from the class. Many wrote the parts of the body we had been learning and showed themselves in a very tangible way that indeed they had “learned something.” I was hoping for the same degree of specificity in the second question, “I used my English at...” but it was too hard for most to go beyond a list of “work, bus, etc. “I tried to get verbal anecdotes from some as I went around. One woman wrote “may (sic) house” as a place where she had used her English. Since I knew she had children, I kidded her by saying, “Oh, Alba, you are now talking to your kids in English.” “No, with my neighbor,” she replied, then told me how she translated a letter from the clinic to her neighbor and felt that she had been able to do so because of the health theme in our classes. This is the kind of detail I wanted but which is hard to get from a written evaluation unless you ask the “right” question and have the time to prod.

Question #3 was also helpful in giving us information about the next cyle but #4, “The teachers help me to learn when they...” was a mixed bag and #5 did not work at all. I speculate that either it was too close to #4 (the same words, except for “do not”) or it would have meant being critical of the teachers, which adult learners usually don’t like to do. Even after explaining the question in Spanish, most wrote variations on “my teachers are good” and a couple showed that they had not undestood the question at all. Overall, I liked the simplicity of this form but would have liked more time to really get into the questions with the students and try to get them to give me (and themselves) as much information as possible.

I developed an even simpler final evaluation tool for the end of the third cycle. We simply asked the students what they liked most about the class and what they had not liked and any recommendations for a future class. We got much more direct information such as recommendations to use the book in class (we had picked a self-study text of English for Spanish speakers) to the by then ubiquitous “don’t erase so fast.”

Third Cycle
Simplicity and speed were the motivating factors for the short third cycle. Because we already had information about returning students, I had come up with a “test” that included materials from the last cycle, mainly in response to the women’s request for more traditional exercises such as dictation and quizzes. It worked very well because it provided a sense of continuity for those who had been coming most of the year and it was in an assessment format that felt familiar to them. For me, the teacher, it was also a familiar way in which to assess how folks had progressed and whether they had engaged with the material.

Top

The “John’s Test”
I chose the true-and-tried “John’s test” for the initial evaluation of new learners because we needed a tool that would give us the maximum amount of information in the quickest amount of time. There were only two of us to assess 20 people, half of whom were brand new. My co-teacher took over the assessment of returning learners, while I met with the new learners, which included two young men. We only had about two hours in which to do it all and the John’s test was perfect for that kind of situation. I could conduct the oral proficiency piece, for example, while one student was filling out the intake piece and another could be finishing Part II, the literacy screening.

The John’s test, as many teachers of ESOL to adults know, was widely used in programs until fairly recently. It was one of the first tests (1975) originally designed to test English oral proficiency in adults at basic education programs. It was also not copyrighted which further encouraged its use but mostly, it is a quick, easy-to-use tool that concentrates on testing listening and speaking skills. Through the years, teachers have adapted it, mostly to include a reading/writing component.

The version I used is so changed that it hardly can be called the John’s test. For one thing, “John” is gone. In the original version, a series of seven pictures showed a day in the life of John. Learners were expected to try to describe what was happening with John in those pictures. At the Boston Workplace Collaborative, where I first taught ESOL, we got rid of John and used the personal data questions as a “test” of oral profiency. A “0” indicated no comprehension, while a “3” indicated an advanced level. A two-part literacy screening followed that goes from very simple matching of words to pictures of common, everyday things like “fire,” “house,” etc., to more advanced tasks of reading a short letter and answering comprehension questions.

It had been many years since I had last used it, but I found it worked fairly well in answering the kinds of questions we had about a brand new learner. Could they understand and answer simple, personal questions such as “what is your name, how many years of formal schooling have you had in your country, can you read or write at all, can you read in English at all?

It also helped us deal with our biggest challenge, which was to assess folks as quickly as possible. Like most assessment tools, it was less effective in giving us accurate levels of people’s knowledge and skills but it did give us elementary knowledge about each student.

The lesson in all this?
This is the question I always ask myself after I’ve been faced with a challenge where I have had to reevaluate assumptions and change a course of action. I think the biggest realization for me after this experience, and which I had forgotten in my years out of the classroom, was how time-consuming assessment is. It was the biggest factor in my decision to go from using a “cadillac” of a tool to a true-and-tried placement test. Ideally, initial assessment should be done at a special time with up to thirty minutes allotted per learner, so there is enough time at the end for the teacher or counselor to talk to the student about the process.

A casualty of the time crunch is on-going assessment. There simply wasn’t enough time in our three cycles to include a formal assessment even though there were a couple of quizzes and frequent checking in between my co-teacher, the counselor, and me. For the most part those conversations tended to focus on other basic needs our students had, many of whom were facing cuts in social welfare benefits such as food stamps. This was especially disturbing for us when we realized that for at least one mother, the snack we gave to her kids during childcare was her biggest incentive for attending class.

A colleague asked me why I tried different tools instead of sticking with one throughout the year to see how it worked with different learners. It struck me that I had not even considered that option, which is a good one given that in my experience, every learner and situation in ESOL is different. I believe my motivation for changing tools was simply the need to find one that fit certain criteria that were not very clear at the beginning but which evolved as the year went by.

Through this experience I have come to believe that developing criteria for assessment should not only include standards for measuring skills and levels but other factors as well, such as composition of the class, frequency of meetings and length of cycles, and ultimately, what really makes sense to do. I had started out by being overly concerned with assessing people’s skill levels in English and getting a picture of literacy in their lives. Those were good goals but I needed to start with some simple, basic questions such as, what do we really need to know about a new student when they walk through the door, how can we assess returning students (especially when there are no other levels), what is realistic given the time available for the assessment as well as the class itself, what can we find out about a previous class to inform the next one, what do learners want to learn.

Ultimately, I don’t believe any one language assessment tool can give us the whole picture of a learner. At best, an initial assessment tool can give the teacher a snapshot of the student and the final evaluation is the end result of an ongoing conversation about what is going on in class, what needs to be better, and what the learner’s needs are. Getting answers to some of these questions became more important to me than the exact tool I was using.

ESOL ASSESMENT TOOL

SELF ASSESSMENT TOOL

CYCLE EVALUATION

Top

Originally published in Adventures in Assessment, Volume 11 (Winter 1998),
SABES/World Education, Boston, MA, Copyright 1998.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.