Rehearse is a collaboration between University of Auckland and Xorro Solutions. It uses the Xorro Q participation platform to provide a new and fresh approach to teaching and learning core principles in structures.
Rehearse was first deployed in the second semester of 2015 with the Civil Engineering 211 course on Structures. Students in that course were asked to interact with a initial set of practice activities. Their response to that experience was surveyed. They were also asked for suggestions on how the product should be improved for the following year. Based on their feedback, a group of those students interned over the summer to produce a comprehensive set of content. This content has now been released to the 2016 cohort of students for this course, and initial data from their feedback is now to hand.
2016 Cohort Responds
In 2016, the students in the 211 course experienced the use of Rehearse in both live lecture settings as well as in practice activities. Five lectures used Rehearse intensively to review and consolidate basic skills, prior to moving on to more advanced structures concepts. The students were tested in their basic skills following this module of the course. In addition, students were asked to complete a range of practice activities; their completion of these to a 75% success standard is a requirement for passing the course.
Rehearse is designed to help students who are otherwise under-achieving their potential in the course. Although all students “consume” Rehearse, it is anticipated that those students demonstrating difficulties in understanding or applying core concepts would be the ones most benefiting from it. To determine whether this is the case, students were grouped into six categories based on their performance in a prerequisite course. Data on how students responded to the tool both in class and in practice activities, is analysed in context of these categories.
| Group code | Description |
|---|---|
| Low-R | Students repeating from 2015 |
| Low | Students who received a C- or lower grade in the prerequisite course |
| At-Risk | Students who received a C or C+ grade in the prerequisite course |
| Med | Students who received a B-, B, or B+ grade in the prerequisite course |
| Top | Students who received a A-, A, or A+ grade in the prerequisite course |
Survey Methodology
All students were surveyed a week after the test, to contribute to an assessment of the impact that Rehearse has had on learning. The survey asked students to rate the usefulness of Xorro-Q in learning concepts, and to indicate the extent to which they feel the tool has improved their expectation of achieving a high grade in the course. The survey also included a series of questions based on Lichert scales gauging students’ agreement or otherwise with statements about how Rehearse can be best utilised (in class or in practice activities). For each of these statements, an “Agreement Score” was calculated by awarding 0 points to a middle choice, -1 or +1 point to “Disagree” or “Agree” respectively, and -2 or +2 points to “Strongly disagree” or “strongly agree” respectively. The outcome was then divided by the number of respondents in the group to compensate for the varying sizes of each group. The resulting “Agreement Score” is represented as a % value, and could be considered as equivalent to that proportion of the group concerned selecting “Agree” where over 100% indicates overwhelming agreement.
Results
Responses to the first three questions confirm that the 2016 students feel even more strongly than the 2015 cohort did, about the value they receive from using Rehearse. The improvement relative to 2015 is undoubtedly a result of the wider range of activities available plus the emphasis on feedback in the activities (this being a key are for improvement identified by 2015 students).
| Question | 2016 Response | 2015 Response |
|---|---|---|
| Q2 | 99% | 81% |
| Q3 | 97% | 84% |
| Q4 | 93% | 66% |
Questions 6 through 13 were all Lichert scale questions providing an equally balanced choice for respondents. The Table below indicates how the Agreement Score for each question varies across the sub-groups.
| Agreement Scores | Low-R | Low | At- Risk | Med | Top | All |
|---|---|---|---|---|---|---|
| Q6: Xorro-Q practice activities help me to understand difficult concepts | 75% | 60% | 108% | 62% | 42% | 67% |
| Q7: Xorro-Q should be used as a platform for learning core engineering concepts in earlier years | 75% | 120% | 142% | 85% | 42% | 89% |
| Q8: Xorro-Q should be used in other subjects as well | 0% | 60% | 92% | 74% | 42% | 67% |
| Q9: Using Xorro-Q in class wastes valuable learning time | -50% | -20% | -33% | 28% | 58% | -15% |
| Q10: Using Xorro-Q interactively in class has been helpful | 125% | 40% | 75% | 49% | 8% | 50% |
| Q11: It’s helpful to use Xorro-Q in class every 10-15 minutes to apply what is being taught | 100% | 40% | 75% | 44% | 25% | 49% |
| Q12: It’s helpful to see how my responses in Xorro-Q compare with other students’ responses | 50% | 100% | 117% | 79% | 50% | 81% |
| Q13:I would be more likely to attend lectures that I know will make use of Xorro-Q | 0% | -20% | 0% | 0% | -67% | -13% |
Conclusion
There is no question that this group as a whole, as well as every sub-group, is finding the Rehearse activities very helpful. All groups agree that Rehearse should be used in earlier classes (especially strong opinion expressed by “RISK” and “LOW”). Opinion in using Xorro Q in other subject areas is less pronounced but still strongly positive. All groups agree that they would be motivated by comparing their scores (not just question answers) with generalised class scores.
There is significant divergence in responses among sub-groups aligned with academic history:
Members of the “TOP” group indicated the least positivity about Rehearse (at 42% overall agreement to Q6) while members of “RISK” indicated the most perceived value. The group as a whole returned a 67% Agreement Score to this question, which is equivalent to 67% of participants indicating Agreement and 33% indicating no opinion.
The two back-to-back questions testing opinion on use of Rehearse in class produced the widest divergence of opinion, and the most use of strongly agree” and “strongly disagree” selectors. The proposition that using Rehearse in class wastes time, received strongest support from members in “MED” and especially “TOP”, and strongest opposition from members in LOW-R, LOW and RISK. Paradoxically, all sub groups indicated in both Q10 and Q11 that using Rehearse interactively during class was helpful.
There was split opinion or neutrality on the last question (whether students would be more likely to attend lectures where Rehearse is being used), with overall disagreement. This was consistent for all sub-groups.
