Ten Critical Ideas About Assessment to Benefit Every Educational Professional Tris Utschig, Georgia Institute of Technology and Daniel Apple, Pacific Crest

Abstract

This paper presents 10 critical ideas about assessment that, when placed into practice, can significantly advance and add value to every academic professional's work. Together, these ideas form a set of principles and practices whose impact can be measured to provide a comprehensive picture of the assessment culture in an organization. Each idea presented here is supported by the literature and informed by the use of the idea in practice. Key terms and schema relating to assessment are defined, related resources are noted, and ways in which assessment can enhance student learning and growth, enhance faculty performance, and improve institutional effectiveness and quality in its collective learning environment are shown. The ideas presented here fall within three categories. The first category addresses vital issues in the relationship between assessment and evaluation. Specific topics addressed in this category are the role of measurement in assessment, common misconceptions about assessment, using assessment to enhance performance on an evaluation, and turning evaluation into assessment. The second category deals with assessment values, practice and mindset. Specific topics addressed in this category are key steps in the assessment process, writing clear performance criteria, assessing assessments, and self-assessment for growth. The third category deals with conditioning and historical cultural norms in higher education that prevent change toward an assessment culture. Specific topics addressed in this category are the role of assessment in the mentoring process and the assessment mindset. The paper closes with a brief description of issues regarding cultural change that must be confronted when an organization in higher education begins to embrace and implement these ideas.

Table 2: Definitions

1. Role of Measurement within Assessment and Evaluation

Measurement is a process that is often poorly understood. First and foremost assessment is not just measurement. The practice of measurement of teaching and learning throughout Higher Education holds significant opportunity for improvement (ref). Nonetheless, in either quality assessment or quality evaluation processes, one must always collect quality data to understand at what level the performance has been delivered. From this data, an assessment can be produced to help improve the performance. Also from this data an evaluation can be made to determine the acceptability of the performance. In either case, it is critical that the areas of quality in the performance are clearly stated (see idea number 6, writing clear performance criteria) so that one can identify what really matters and then locate or construct the means to accurately and reliably measure those areas. In order to produce the needed analysis in assessment or the judgment in evaluation, it is critical that the measurement is done in timely, effective and meaningful ways where the participants in the process can be assured that the data is reliable, targeted, and comprehensive in order the carry out the intended purpose (Stiggins, 1987).

Among the community of scholars currently advancing this knowledge area are the Evaluation Centers at colleges like Washington State University and Western Michigan University, those involved in placement of individuals like ACT and ETS. From this community, tools to measure many areas of performance critical to student learning in higher education have already been developed (Faculty Guidebook: A Comprehensive Tool for Imrpoving Faculty Performance, 2005; Hatry, Houten, Plantz, & Taylor, 1996; Schrock, 2006).

2. False Assumptions about Assessment

There are several key assumptions that many educators have made about assessment that make it very difficult for them to elevate the use of assessment and its quality in daily practice.

Assumption 1: A need for improvement implies poor quality. Within an evaluation culture, if someone is working to get better it is assumed that they are sub-standard. For example, a faculty member who is recommended to go to a teaching workshop implies that they are not teaching well enough. Once it is understood that everyone can improve and that stating a need for improvement implies that someone is committed to making that improvement then areas for improvement become a positive sign and not a negative stigma. An area for improvement is very much different than a weakness, but people frequently interchange these words.

Assumption 2: High quality implies no need for improvement. This assumption is almost the converse of assumption one. Star performers have found that improving all areas of performance is productive in advancing overall performance, not just focusing on the least effective areas (reference would help here). The focus at any point in time should be in the area that can contribute to the greatest growth. Sometimes the camera should be on a strength, because you want to understand better how this strength works and why it is contributing so much to a performance. At other times the camera should be on an area where you can see that you are being held back from achieving the level of performance you would like. Thus, it is important not to bias the selection of focus away from strengths because they are equally important to areas for improvement.

Assumption 3: The person receiving feedback will know how to make use of the feedback to improve. For example, most faculty believe that students are used to getting evaluative feedback and therefore know what they need to improve to meet a benchmark. However, most people who get feedback are not capable using it effectively for two major reasons. First, they are not emotionally prepared to take an evaluation and transform it into an assessment (see turning evaluation into assessment) (need reference). Second, and more importantly, the analysis skills for the given context are nearly always beyond the performer’s skill set to provide meaning and direction for improving future performance. For example, in a composition course, students who get feedback on their paper see the issues but have little clarity of what to change in their writing process to make this improvement. Therefore, it is important to understand that the assessor needs not only to provide feedback on how to improve the content, but also how to improve the process.

These ideas are echoed by Entwistle when discussing faculty development to improve student learning. "Evidence on conceptual change suggests that, first, the individual has to perceive a reason for change. If the existing conception still feels adequate [high quality 􀃆 no need for improvement] and comfortable, change is unlikely. A variety of experiences have to be provided which both challenge existing conceptions and suggest interesting alternative conceptions [suggested improvement does not mean low quality in current performance]. But these experiences have to be carefully managed within an encouraging and supportive climate. The experiences also have to be designed to fit the previous knowledge and learning habits of the learners involved [such that they can make use of the feedback]" (Entwistle, 1995).

3. Using Assessment to Advance Performance in an Evaluation

A very important relationship between assessment and evaluation is the way in which evaluation can create motivation for assessment. Everyone wants to perform well, and when there are well-defined means in place to determine level of performance, the use of assessment to continually improve performance against these means in preparation for an existing evaluation is a very motivating force.

Now, it is important to realize that the assessor cannot drive the relationship. Despite the fact that the assessor wants to assist in increasing the performance of the assessee, a hidden motivation to help themselves is often present as well. For example, in the teacher-student relationship, most often the faculty member desires to improve the student’s performance so they get strong reviews as an educator. The department chair wants their junior faculty to grow and prosper so they get tenure and strengthen the department. The college wants their programs to get better so they are reviewed as world-class. The advisor of the Ph.D. candidate wants their mentee to pass their dissertation defense and expand their community of established expertise. In each case, the person who is in a position to provide assessment to help improve performance in the evaluation process also has a stakeholder interest in the results of the evaluation. If these assessment relationships are led by the assessor rather than the assessee, this leads to force-fed assessment, or the desire of the mentor/teacher/administrator wanting success more than the mentee. When this occurs it severely reduces the potential for growth of the assessee because long term behavioral change comes from within and not from the outside. Effective, internalized change cannot be mandated (Fishbein & Ajzen, 1975; , "Recruiting Supportive Coaches: A Key to Achieving Positive Behavioral Change", 2003) (need additional refs). Thus, it is much more effective to use a mentoring process to set up the desire for assessment and then let the assessee play the central role in the assessment process where they initiate and implement assessment. In this way, preparation to perform in evaluations will by much more effective due to internal forces driving the desire to perform well in the evaluation process.

4. Turning Evaluation into Assessment

The fundamentals of turning evaluation into assessment are to ask more of the feedback, respond non-emotionally to the feedback, and to structure the feedback in three forms – strengths, improvements, and insights.

Asking more of the feedback: as informed by best practices in the literature, over the years we have synthesized the information to develop many profiles and performance measures such as the profile of a self-grower, quality learner, and a quality faculty member as well as performance measures for level of self-growth, level of learner performance, and level of educator performance (Collins & Apple, 2003; Krumsieg & Baehr, 2000; Nancarrow, 2003) (have profile refs, need refs for each performance measure). In each of these profiles is the ability to effectively reflect and self-assess in order to move up the scale and reach the highest levels of performance. In other words, quality self-assessment is a must. Therefore, the opportunities that each person has of using evaluative feedback as the basis for transforming this feedback into assessment is a critical factor in becoming a "star" performer. In order to do this, one must view evaluative feedback as benchmark data to use and expand upon within a flexible, proactive self-assessment process to improve performance.

Non-emotional response to the feedback: most individuals who receive evaluative feedback are not ready to use this feedback for growing future performance because they are too emotionally wrapped up in how well they just performed. Responses to evaluative feedback may be to ignore it in the case of satisfactory performance, or to feel that the evaluation process was flawed and that it was unfair in certain aspects in the case of unsatisfactory performance. However, if the feedback is taken and analyzed through the lens of assessment, evaluative feedback can be useful. Furthermore, by focusing on performance rather than performer it becomes easier for the evaluator to provide more information and analysis and for the performer to use this feedback since it is removed from emotionally laden ties to the performing individual.

Organizing feedback into strengths, improvements, and insights: evaluative feedback can identify strengths and help build those strengths through analyzing the reasons behind why those strengths contributed to the performance, identify areas for improvement and lead to action plans for achieving those improvements, and provide insight about the underlying elements of the performance that has just been evaluated. By organizing evaluative feedback in this way, "the evaluation no longer serves as a punitive measure or a justification document, but instead serves as a … useful tool by which the performer’s progress is measured within the assessment paradigm (Watson, 2005)."

Current practice of assessment can often be enhanced by recognizing and incorporating some commonly missed fundamental tools, concepts, and alignment with basic principles. The most important areas to consider include writing clear performance criteria, evolving an assessment mindset, and importance of assessing assessments.

5. Key Steps in the Assessment Process

As learned through assessing thousands of different assessment processes and systems over the last 15 years, effective assessment always involves several common steps (Apple & Baehr, 2005). Observation has shown following to be the most frequently missing steps: ensuring the assessment request and purpose come from the assessee, establishing the rules of engagement between the assessor and assessee, ensure assessment is delivered only in the desired areas, and formatting feedback in a structured manner for the assessee.

The first step is determining who has requested the assessment and for what purpose. Clearly, we understand that no individual can control the behaviors of someone else. Thus, assessment must be learner-centered around the assessee, where they desire to make changes for a desired outcome. Whether an observation that a change could be beneficial originates with the mentor or the assessee, the assessee must want and agree to the purpose of the assessment (Chism, 1999).

Another critical step that is often skipped is the establishing of rules of engagement between the assessor and assessee. This helps to create an atmosphere of trust within the assessment process and relationship. Creating simple guidelines for collecting assessment data and delivering the feedback lead conditions where the feedback can be used effectively to produce an enhanced future performance. Conditions to consider include setting proper boundaries and conventions, agreeing upon the way in which the performance will be viewed, and finally, how and when the feedback will be delivered (Arreola, 1995; Chism, 1999).

Checking that the criteria used for the assessment align closely with the focus areas chosen by the assessee is another step that is often skipped or poorly addressed. This may arise from a situation where the assessor feels they are in a better position to determine what the assessee needs and thus uses their values and expertise to make these decisions. However, the truth is that the person or program being assessed is actually in a much better position to know what they need. Here are the conditions that must exist for the criteria to be valuable; the assessee must a) want to improve in the areas subject to the criteria, b) have an opportunity to improve, and c) feel they can benefit significantly from the assessment because the criteria are important to the overall performance. The assessor needs to understand that their role is one of servant leadership, often providing a recommended list of potential criteria, but must let the assessee chose the criteria that meets the above conditions (Angelo & Cross, 1993).

Finally, assessment feedback is often left underutilized despite intentions to bring higher levels of understanding, comprehension, and meaning to that feedback. As mentioned in idea number four (turning evaluation into assessment), organizing feedback into three forms: strengths, areas for improvement, and insights, is a simple but effective way to elevate the quality and usability of assessment feedback. The first area of analysis is clarifying the strengths in a performance as to their value and in how those strengths were produced. Often, individuals or programs produce value, but don’t realize it and are unaware of how they did it. If a future performance is going to leverage off of demonstrated strengths, we need to understand them. The next area of analysis is the production of well developed action plans that describe how the most important improvements are going to be made. This is where the assessor "earns their money" or produces significant value. Their expertise can provide much better direction and more quickly develop practices to gain these improvements. Finally, since assessing something allows you to be in a very special learning situation, you can share your new understanding and learning with the assessee through well developed insights (Gardner, 1997; Parker, Fleming, Beyerlein, Apple, & Krumsieg, 2001; Wasserman & Beyerlein, 2005).

6. Writing Clear Performance Criteria

Fundamental to quality assessment and evaluation is having clear performance criteria. These criteria form the foundation for building and improving performance in any type of assessment setup. Simply put, performance criteria are the areas of quality encompassing an action or task to be performed. Another definition might be "a description of the quality requirements" during and as a result of a performance (Faculty Guidebook: A Comprehensive Tool for Imrpoving Faculty Performance, 2005; Rogers, 2002; , "What is a performance criterion?" 2006).

The writing of performance criteria is one of the more difficult tasks for faculty and staff. It requires skills in identifying and prioritizing the key qualities that are sought in a performance or a set of performances, an analysis of these qualities, seeing the pattern in the analysis, visualizing the performance, and finally writing the clear statement of what this performance looks like. Additionally, the performance criteria should be described in an explicit and motivating way such that buy-in to these challenging expectations is generated for all stakeholders (Airasian, 1991; Arter & McTighe, 2001).

Performance criteria are a must for improving and document performances such as learning activities, courses, and programs. They are also critical for administrators, determining the level of institutional effectiveness, effective faculty tenure and promotion process, and accreditation. The performance criteria provide an excellent foundation for identifying a set of measures that really matter and correlate strongly to the integrated performance desired. The more performance criteria are imbedded in daily practices, the stronger the assessment culture evolves and the more likely effective feedback will be both generated and integrated for improving future performance.

7. Assessing Assessments

Any person involved in assessment knows that assessment is used to advance performance in many areas of educational practice and processes. The one practice/process that rarely gets assessed is assessment. It is critical for assessment systems, processes, and practices get assessed on a frequent basis to improve their performances. The current quality of design, implemented processes, and individual practices can be improved significantly. The identification of clear criteria and measures for assessing assessments with timely data collection and analysis can provide feedback that can be incorporated in improving the performances of the assessments.

The assessors and the designers of assessment systems and processes must initiate the desire and mechanisms for this assessment to occur. The assessment experts are normally in control of the assessment systems and processes and are the least likely to initiate the improvement of these assessment of assessments. A practice that is very effective is for the assessee to assess the assessment they receive.

8. Self-assessment to Produce Self-Growth

The enriched learning environment in higher education is based in an assessment culture where each person wants to improve and practices self-assessment as a means of mentoring self-growth. This basically requires support for stopping to take a few minutes after each performance – teaching a class period, finishing up a course, designing a new activity, writing a proposal, facilitating a committee meeting, etc. to assess the performance to see how future performance can be improved. The pressures of performance (doing) has increased over the last ten years and as a result leaves even less time to be reflective and the discipline to practice self-assessment. This is the fundamental basis for shifting a culture to self-growth.

The shifting of a Higher Education institution towards a continuous quality improvement environment is a non-trivial transformational change. Many forces are lining up to encourage, challenge, and support this change such as regional and professional accreditation, state legislature, professional association, funding sources, and students. There are many things to consider in expanding assessment systems, processes, and practices within Higher Education.

Higher Education has conditioned faculty with years of experience for both receiving and providing evaluative feedback. Faculty are familiar with giving (or getting) feedback to improve content to meet a benchmark – not in giving (or getting) feedback on improving the process. A significant value in Higher Education is being critical of thinking and knowledge of others. DeBono called this the Intelligence Trap – it is much easier to find fault in others, their ideas, and performance than putting oneself in a risky venture of presenting your own ideas, trying out new innovative practices, or present new research or ways of improving something established. A quick inventory of practices – course grades, tenure and promotion, student evaluations, peer review of research proposal grants, publishing critique, evaluation of text materials, and merit pay.

9. Role of Assessment in Mentoring

The valuing of growth in performance is fundamental in a mentoring relationship. The key role of a mentor is to help improve future performance by proving quality feedback. The key question then is - If you want to improve future performance do you need feedback on content of the current performance or process of current performance? This is the issue – need to intervene on process and not content. The assessment feedback is associated with how to improve the performance and not on content of the performance. For example, in the writing process, often feedback is given on how to improve a certain paper that is being written versus looking at the issues behind the performance and use this understanding of how to improve their writing process.

10. Evolving an Assessment Mindset within an Evaluation Culture

The whole mindset of assessment shifts philosophy of how feedback is to be given and received. Evaluation is an important process and should improve also, but most of the time when feedback is really being structured for helping improving performance, the feedback still comes across as evaluative. Most people who give feedback do not differentiate evaluative versus assessment feedback and only really differentiate it when they receive the feedback. It is important to set up conventions and practices that create an assessment culture. One of the most important reasons is that faculty are familiar with giving (or getting) feedback to improve content to meet a benchmark – not in giving (or getting) feedback on improving the process. Language becomes important, understanding other’s values and perspectives is fundamental, and finally wanting others to improve performance must be an important goal.