Search help
How do I decide which Personal Result method to apply in my peer assessment activity?
The choice of calculation method for determining a team member’s personal result is determined by the teacher's preference for compensating more strongly team members who have contributed significantly to their teams, and under-rewarding team members who are peer assessed as weak contributors. The figure illustrates the statistical features, such as team average, range, and standard deviation, associated with each method.
Alternative calculation methods for Personal Result (PR) illustrating effect on team average and spread for a given Team Result
The teacher can select either the Peer Assessed Score (PA Score) or Peer Assessed Index (PA Index) if they wish to exclude a team result in calculating the Personal Result (PR).
More usually, the Peer Assessed Score and Team Result (TR) are combined mathematically to produce a Personal Result. There are three alternative methods. As the figure illustrates, the Indexed Personal Result (IPR) is the least discriminating method, whilst the Rank-Based Personal Result (RPR) is the most discriminating in terms of favouring significant team contributors and penalising weak contributors. Most teachers select the Normalised Personal Result, often with a spread factor of 1.5 to 2.0.
In contrast to the graphical illustration earlier, the following table summarises the example calculations presented through a series of FAQ that present the mathematical definition and example calculations for each method.
Comparison of Personal Results calculated by several methods in a team of four members
|
ASSESSEE |
||||||
|
ASSESSOR |
Bridget |
Julian |
Lydia |
Nigella |
Mean |
Range |
|
Rank Reversed |
1 |
2 |
4 |
3 |
||
|
Peer Assessed Score, PA Score |
54 |
74 |
82 |
78 |
75 |
28 |
|
Peer Assessed Index, PA Index |
66 |
90 |
100 |
95 |
88 |
34 |
|
Team Result, TR |
50 |
50 |
50 |
50 |
50 |
0 |
|
Indexed Personal Result, IPR |
33 |
45 |
50 |
48 |
44 |
17 |
|
Normalised Personal Result, NPR (SpreadFactor = 1) |
39 |
51 |
56 |
54 |
50 |
17 |
|
Normalised Personal Result, NPR (Spreadfactor = 2) |
28 |
52 |
62 |
58 |
50 |
34 |
|
Rank-Based Personal Result, RBR |
20 |
40 |
80 |
60 |
50 |
60 |
Source: FAQ: How are peer assessment and personal results calculated and defined mathematically?
Definitions and features of calculation methods used in Peer Assess Pro
|
Attribute (X1) |
Abbreviation (X1) |
Definition (X1) |
|
Peer Assessed Score |
PA Score |
A relative measure of the degree to which a team member has contributed to their team's overall achievement, team processes, and leadership. The Peer Assessed Score (PA Score) is calculated for each team member directly from their Average Team Contribution (ATC) and Average Leadership Contribution (ALC). That is, from the ten components of Team and Leadership contribution survey in the peer assessment.
A Peer Assessed score is generally used to compare the relative contribution of students WITHIN the same team, rather than BETWEEN teams. The Team Result has NO impact on the value of the Peer Assessed Score. Values for the PA Score range from zero through 100. |
|
Peer Assessed Index |
PA Index |
The Peer Assessed Score (PA Score) is indexed upwards so that the person in the team with the highest Peer Assessed Score is awarded a Peer Assessed Index of 100. All other team members receive a proportionally lower PA Index in the ratio PA Score / max(PA Score). The Team Result has NO impact on the value of the Peer Assessed Index. |
|
Team Result |
TR |
The result awarded to the team for the outputs of their work. The teacher typically derives the Team Result (TR) from grades for team reports, presentations, and results of Team Readiness Assurance Tests.
The teacher may select to combine a student's Peer Assessed Index (PA Index) with their team's Team Result (TR) to calculate a Personal Result (PR) for each student, reflecting their relative contribution to the Team Result as assessed by their peer team members. Peer Assess Pro enables the teacher to select from several methods to combine the Team Result and Peer Assessed Index (PA Index) to produce a Personal Result: the Indexed Personal Result (IPR), the Normalised Personal Result (NPR), and the Rank Based Personal Result (RPR). |
|
Measures of a student's personal result |
||
|
Personal Result |
PR |
A student's personal result gained from combining their Peer Assessed Index (PA Index) and, optionally, their Team Result (TR).
The teacher selects from one of several Calculation Methods to calculate the Personal Result that incorporates the Team Result. These methods are Indexed Personal Result (IPR), Normalised Personal Result (NPR), and Rank-Based Personal Result (RPR).
The choice of method is determined by the teacher's preference for compensating more strongly students who have contributed significantly to their teams, and under-reward students who are peer assessed as weak contributors. Figure 1 illustrates the statistical features, such as team average, range, and standard deviation, associated with each method. The IPR is the least discriminating method, whilst the RPR is the most discriminating in terms of favouring significant team contributors and penalising weak contributors, as the figure illustrates. |
|
Indexed Personal Result |
IPR |
The Indexed Personal Result is calculated from the Team Result (TR) combined with the student's specific Peer Assessed Index (PA Index). The Indexed Personal Result method awards the Team Result to the TOP RATED student in the team, since, by definition, their Peer Assessed Index is 100. All remaining students in the same team earn the Team Result downwards, directly proportional to their PA Index.
The Indexed Personal Result calculation means that NO team member can earn an Indexed Personal Result greater than the Team Result. That is, values for the Indexed Personal Result range from zero up to the Team Result. |
|
Normalised Personal Result |
NPR |
The Normalised Personal Result is calculated from the Team Result combined with the student's specific Indexed Personal Result (IPR). However, in contrast to the IPR method, the Normalised Personal Result method awards the AVERAGE student in the team the Team Result (TR). All remaining students are awarded a Personal Result ABOVE or BELOW the Team Result depending on whether their IPR is above or below that team's average.
Features of the Normalised Personal Result are that (a) In contrast to the IPR method, the Normalised Personal Result method calculates a Personal Result ABOVE the Team Result for the above-average peer rated students in the team (b) The average of the team's Normalised Personal Results matches the Team Result (c) The spread of the team's Normalised Personal Results matches the spread of the Indexed Personal Results (IPR) that is calculated for that team. Spread is measured by the standard deviation statistic. .
Optional feature: To enhance the effect of rewarding high contributors and penalising weak contributors the tutor can increase the Spread Factor (SF) from the default value of 1.0. Increasing the Spread Factor increases the spread of the results centred around the Team Result. However, an increase in the Spread Factor will maintain a team average NPR that matches that team's Team Result. A Spread Factor of 1.5 to 2.0 is recommended, especially in classes where team members are reluctant to penalise weak contributors and/or reward the highest contributors through their peer assessment rating responses.
Values for the NPR range from zero to 100. Calculations that exceed these ranges are clipped to fit within zero to 100 |
|
Rank Based Personal Result |
RPR |
The Rank Based Personal Result is calculated from the Team Result combined with the student's specific Rank Within Team based on that student's Peer Assessed Score. Like the Normalised Personal Personal Result the RPR method awards the AVERAGE student in the team the Team Result. All remaining students are awarded a personal result above or below the Team Result depending on whether their Rank Within Team is above or below that team's middle-ranked student.
Features of the Rank Based Personal Result (PR) calculation method are that (a) A team's RPR values are spread over a MUCH WIDER range than the NPR and IPR methods. Small differences in PA scores within a team are amplified significantly by this method (b) In contrast to the IPR method, the RPR method calculates a Personal Result significantly ABOVE the Team Result for the top ranked student in the team (c) Like the NPR method, the average of the team's RPR values matches the Team Result. Values for the Rank Based Personal Result range from zero to 100. Calculations that exceed these ranges are clipped to fit within the range zero to 100. |
Note that in the Xorro PAP app, we have renamed the following Personal Result Methods
Renaming of terms for Xorro Peer Assess Pro
|
Xorro Peer Assess Pro |
Abbreviation |
Google Peer Assess Pro |
Abbreviation |
|
Peer Assessed Score |
PA Score |
Team Based Learning Score |
TBL Score Score |
|
Peer Assessed Index |
PA Index |
Team Based Learning Index |
TBL Index |
Also see: Participant Teams and Teamsets Group Peer Assessment
What is the Purpose of Peer Assessment? What questions are asked in the peer assessment survey? How is the Peer Assessed (PA) Score calculated? How are peer assessment and personal results calculated and defined mathematically?
Running Peer Assessments from Xorro-Q Importing Participants, Teams and Groups How do I view or correct the Team Composition in a running peer assessment activity? How do I decide on which Personal Result method to apply in my peer assessment activity?
Peer Assess Pro website Peer Assess Pro Reference Guide (PDF)
Categories: Xorro-X, Facilitators.
Tags: peer assessment.