Improving Standard Assessment in Adult Education

           When considering the implementation of a professional development initiative, the context and culture of the school must be the first consideration. At best, it is futile to offer training that is needed if it is not wanted. At worst, it is harmful to the social health of an organization. This paper proposes the use of a standard assessment in order to measure and improve the accuracy of student placement. The challenge will be successfully launching this initiative in the context of Adult Education (AE) at the Vancouver School Board (VSB).

MA ELM 570, October 2013

Contextual Information

Adult Learning Centres

The AE program at the VSB was established in the 1970s as a self-paced program for single mothers and newly unemployed workers facing long-term changes in BC’s resource economy. The program was run in a single room, and the goal was to empower students by facilitating graduation through instruction and encouragement responsive to the learning needs of adults. A meager budget meant that the program was developed by underpaid instructors with no access to professional development. A philosophy that traditional teacher training did not necessarily prepare one for teaching adults meant that many instructors had no accredited training. Over the years, the program has grown to six centres and eight outreach programs. Teachers are unionized and certification is now mandatory. Fair wages and some professional development have been gained, but an organized approach to school improvement remains elusive.

Standard Assessment Practice

For the most part, AE teachers work independently to help their students achieve success. Although qualitative evidence indicates that they are successful in this endeavor, no reliable data is collected to support this understanding. Although one may be inclined to fault the schools for lack of due diligence, it is important to note that there were no formally established PLO’s for upgrading courses until the BC Ministry of Education drafted the Literacy Foundations Curriculum in 2008 and formally implemented it in 2010. Further promoting individualized assessment is the absence of standardized testing in AE. In fact, most adult students are not even required to write provincial exams.

There is little motivation to change AE’s culture of professional independence from within because an established lack of bureaucratic oversight has allowed the schools great freedom to teach to the student instead of the curriculum. Even if there were a standard measure of assessment, sporadic student enrollment and widely varied student needs would make the results difficult, if not meaningless, to interpret. Furthermore, many adult students did not graduate in the K-12 system because of adverse reactions to traditional assessment strategies. The AE teacher’s freedom to subvert these strategies has lead to a great deal of personal and academic success for students. These factors work together to create a pride in service that is integral to AE culture. Although this culture of independence is generally constructive, any proposal for school-based improvements, not to mention district-based improvements, is challenged as a result. Nevertheless, student learning can benefit from district-wide improvements to the only test that every AE student takes: the intake assessment.

Many prospective students arrive with transcripts, but they are often out of date, inaccurate, or irrelevant to AE programs. The intake assessment circumvents academic history by providing data that allows learning centre teachers to gauge the actual level(s) of instruction a new student needs. The assessment, however, does not offer reliable data. It consists of fragments of the Test of Adult Basic Education, an unstructured writing sample, and in some centres, a math test developed in-house that covers all levels from numeracy to grade 12.

Although intake assessment results are accepted at all of the centres, the test is inconsistently administered between and even within centres. In response to the perceived needs of students, some teachers will time the test while others do not, citing time as an inaccurate measure of ability. Some schools administer the entire test, while others only use part of it in order to avoid overwhelming prospective students with a lengthy intake procedure. For the most part, it is thought that new students are properly placed, but again there is no consistent data to inform this understanding because mistakes in placement are resolved by classroom teachers. If student placement data generated in the learning centre were more reliable and accurate, classroom assessment could be calibrated district-wide and student learning could be improved. The question then, is how to make this happen in a culture that values the individual student’s immediate needs over the success of the program as a whole.

Facing the Challenge of Change

District-wide professional development seems an obvious solution, but there are some unique challenges to be considered. Although the proposal is to update a single assessment, there is no precedent for this type of initiative in AE, so it is difficult to judge the commitment to change or predict the best way to proceed. Strict limits on funding challenge access to resources required to successfully complete the initiative. There may be confusion or disagreement about who will lead this initiative because it involves collaboration between and within schools and between and within teachers and administration. Leaders must be prepared for the possibility that staff will undermine the initiative. “It takes energy, stamina and a bold heart to stand in front of teachers and encourage them to change their professional practice” (Neal, 2008, p.7). Regardless, “the primary task of management is to get people to work together in a systematic way” (Christiensen, Marx, and Stevenson, 2006, p.73). The first step in any change management system must be to assess the level of agreement in an organization. The “Agreement Matrix” (p. 74) can be used to understand the level of agreement in AE.

Previous district initiatives in AE have been mandated by administration. They were connected to funding concerns and did not offer a vision for improved teaching and learning. They were developed with little to no collaboration with teachers, communication was non-reciprocal, and they were implemented by the use of what Christiensen, Marx, and Stevenson describe as “power tools” (2006, p.75). The resulting culture of professionalism lacks meaningful engagement with meaningful school reform and encourages teachers to approach school planning from the lower quadrants of the Agreement Matrix. In contrast, teachers work both independently and collaboratively to support specific and immediate student learning needs, putting AE instruction in the top two quadrants of the Agreement Matrix. The difference lies in the perceived degree of meaningfulness: AE instructors work collaboratively when they see a direct benefit to relevant student learning.

The challenge of the proposed professional development initiative is that it requires teachers to collaborate on a broad scale to develop a standard procedure in a working environment that prides itself on individualism. To facilitate the personal transformation required for this professional transformation, teachers will need to understand how a reliable intake assessment can benefit individual students and inform their practice if this initiative is to be a success. The active social process of a constructivist approach (Royal Roads, 2013, p.9-10) will aid this transformation by subverting the historically unpopular use of power tools and encouraging more intellectually engaging “management tools” (Christiensen, Marx & Stevenson, p.76). As the initiative progresses, teachers will respond to the change and move into the upper quadrants of the matrix. Then the use of “leadership tools” (p. 77) will become more appropriate and effective.

Professional Development Plan

Step 1: Outline Plan

In an email, describe the long-term vision for improvements that can be achieved through more a more reliable and informative intake assessment. Present a timeline of the proposed steps in the process. Emphasize the benefits for all stakeholders: students, teachers, and district. Solicit collaborative input and emphasize the core goal of student success. Ensure that leaders are prepared to listen and respond to all questions about the initiative, including dissenting opinions: “Active critics are a great asset in any undertaking [because] we can learn from adversity and difficult situations” (Neal, 2008, p.5).

Step 2: Technology Mediated Environment

This proposal is a long-term project that will affect all schools and requires extensive communication between people who work asynchronously. Access to balanced formal and informal communication needs to be continuous as the work develops. “Meaningful reform will only occur in schools in which educators are able to access and deploy information rapidly to solve problems and then to use that information to adjust to changing needs and demands” (Kowalski 2007, p.159). Principals need to take a leading role in facilitating communication via memos, meetings, and conversations. I also suggest establishing two technology-mediated environments:

  1. a Google document for each centre, to be established on the school-based professional day. This type of interface will be mediated by the contributors. Brief training and ongoing support will be required to ensure equal and productive access.
  2. A WordPress website to address ongoing communication about assessment issues relevant to the district. Two people, ideally a teacher and a principal, should be delegated to take on the administration of the site.

Step 3: Reading Club

Some people prefer learning through discussion with peers. The VSB has a reading club to discuss texts relevant to teaching. I propose that AE choose a text related to assessment practice, and send out invitations to all adult educators. Participation would be voluntary but a summary of the discussions would be posted on the website, and reported back formally in staff meetings. Informal reporting would also be encouraged.

Step 4: District Professional Development

Three optional days of professional development are provided annually for teachers in AE. One day is school-based, one is district-based, and the third is self-directed. Ideally, the task of refining the intake should be a district-wide concern. However, getting approximately 200 teachers to refine one test in one day is not a reasonable goal. I propose instead that the district day focus on quality assessment in the classroom, specifically issues related to the second and third of “seven actions to ensure student success” (2010, p. 5) proposed by Chappuis, Commodore, and Stiggins:

  • Refine achievement standards to reflect clear and appropriate expectations at all times.
  • Ensure assessment quality in all contexts to support good decision making. (p. 5)

Step 5: School-based Professional Development

Teachers in the learning centres need to collaborate to improve the quality of the intake assessment, but first they need to find out what their schools think of the assessment. I propose that the school-based professional development day address the strengths and weaknesses of the intake assessment as it exists in their school. The purpose(s) and goal(s) of the test should be identified and a list of suggestions for how test could be used to improve teaching and learning should be made. Discussions in Step 4 should reflect key learning from Step 4.

Step 6: Learning Centre-based Professional Development

This step is ambitious because it moves outside of the standard delivery of professional development, but does so at no additional cost. In order for learning centre instructors to develop a plan for an improved intake assessment, all learning centres should be closed for a day so learning centre teachers can develop the feedback from Step 5 into an outline of an improved centre-specific intake assessment. Involvement of the principals in this day-long meeting would help align communication and content between centres and ensure that their interests are represented. Principal participation would also indicate board commitment to the improved assessment and thereby encourage teacher buy-in.

Step 7: Self-directed Professional Development Day

This step requires a concerted effort to recruit participation. This third professional day is self-directed, which means that the only person who can set the agenda is the teacher. If the learning centre instructors will not unanimously agree to devote their self-directed day to the intake assessment, the board will have to shut all learning centres for a second day.

The agenda for this day will be to share the centre-based outlines for an improved intake assessment with the goal of creating a single assessment that suits the needs of all centres. Principals should set the tone for collaboration to help overcome possible unwillingness to let go of ideas developed in Step 6. Improvements to assessment should also include a plan for using the resulting data.

Step 8: Begin Using the Assessment

Collect data and share findings on a schedule determined in steps 6 and 7.

Step 9: Celebrate Success

This step should actually occur throughout the process. Because the established culture is resistant to bureaucratic intervention yet very responsive to improved student achievement, continued emphasis on success and the potential benefits of improved assessment will assist teacher buy-in and create the conditions for successful implementation.

 

References

Chappuis, S., Commodore, C., Stiggins, R.(2010). Assessment Balance and Quality: An Action

Guide for School Leaders. Boston, MA: Pearson.

Christensen C, Marx M, Stevenson H. The tools of cooperation and change. Harvard Business

Review [serial online]. October 2006;84(10):73-80. Available from: Business Source Premier, Ipswich, MA. Accessed October 4, 2013.

Kowalski, T.J., Petersen, G., & Fusarelli, L.D., (2007). Effective Communication for School

Administrators: A Necessity in an Information Age. Lanham, Maryland: Rowman & Littlefield Education.

Ministry of Education. (2008). Literacy foundations curriculum program guide, response drafthttp://www.bced.gov.bc.ca/irp/drafts/literacy_foundations.pdf Accessed October 4, 2013.

Ministry of Education. (2010). Literacy foundations curriculum http://www.bced.gov.bc.ca/irp/pdfs/literacy_foundations/2010literacyfoundations.pdfFullAccessed October 4, 2013.

Neal, Mary-Anne. (2008) Leading sustainable change in an education system: published by the Australian Association for Research in Education. Retrieved October 4, 2013.

Royal Roads University. (2013). Learning and teaching model. Retrieved from http://faculty.myrru.royalroads.ca/sites/default/files/RRU%20Learning%20and%20Teaching%20Model%20White%20Paper%20-%20June%202013.pdf