Thread:Rubrics for Peer Evaluation (3)

I have had excellent experiences with peer evaluation and believe that if the task to be marked in this way is designed with the peer evaluation in mind you can have great outcomes. I think that personal reflective assignments are very important and relevant to learning and there is no problem with using peer evaluation so long as the reflection is structured more clearly. The reflection task example used to guide this project could - i think - be improved - rather than one blanket open ended question, it needs to be broken down into smaller pieces with rubric/marks to be allocated for each by the peer evaluator. For example, we have used the CARL framework for some time (and i cannot put my finger of the academic reference for this right this moment!) but it is eg: Context: what happened? Action: what did you do? Reflection: what happened next? where you effective? did you get the result you intended? how did you handle any unexpected outcomes? Learning: insight into your strengths and weaknesses and plans for action emerging from this learning experience

As you can imagine, each of these questions above has the potential to have a sliding scale of marks assigned to them via a rubric which provides guidance and reduces the need for subjective judgement on the success of the task/learning.

In summary - i think the design of the task is very important, and it can make or break peer evaluation. In designing tools for it i think we need to have model assignments and model criteria/rubrics that go with it.

On a slightly different tack - relating to the distribution of marks. I like to have a mix of criteria/marks for outcome AND for process.Sometimes students work hard but still don't get a good outcome. Sometime students wing it with a pretty loose process but know enough to get a good outcome. If we want to help students establish good life-long learning skills and self-directed learning skills then i think it is useful to have a series of questions/criteria/marks focussing on the PROCESS of the task rather than just on the outcome. For example you can ask students to submit information about the "how" of their process eg, did you allow yourself sufficient time to undertake this task? how many sources of information did you consider? If you got stuck, did you seek help and if so with whom? What feedback did you get from your co-learners and how did you incorporate this into your work? How does this learning relate to your prior experience at work or in your personal life? Extra marks can be allocated in the rubrics by peer evaluators relatively simply.

In regards to variation in marks - the kinds of methods discussed seem reasonable. I am very keen on the rating the feedback mechanism however. One common frustration from students feeling hard done by peer marking is "i feel like they did not read my submission properly. they said i didn't address X but it was right there on page Y". It would be great if those receiving feedback could rate the feedback and if a summary/aggreate of that was sent back to the evaluator. It might make these students allocate a little more time and care to their evaluations. If we have evaluators who are regularly getting low scores - could we remove them from the pool of assessors? Similarly, if a student had more than one unfair pieces of feebdack could they request via the system to have an additional evaluator assigned instead?