One of the compromises in automatically assessed numeric (and text) response questions is that they demand exact responses from participants. For example, in asking a group of participants “Who was the leader of Nazi Germany?” – we might demand “Adolf Hitler” as the “correct” answer, but we will in fact receive variations such as “Hitler”, “AdolfHitler”, Adolf hitler”, “Hitler”, “hitler”, “A.Hitler”, and so on. Most of us would be inclined to accept several of these responses as correct or at least partially correct. The same often applies to the numeric answer to a multi-step calculation. What shall we accept as the perimeter of a circle with radius 3cm? A “correct” score depends on the participant’s choice of significant places, and of units. In the past, the range of acceptable alternative answers makes the automated assessment of these types of questions problematic. No longer: Xorro’s latest text and numeric questions now provide for a range of possible responses, and for tolerance of variations in the submitted response “string”.
Numeric response questions:
When creating (or editing) a numeric response question, the author can now specify an acceptable tolerance for the response. This can be a % value, or it can be set as an incremental value of the target “answer” value. In addition, the author can set a preferance for such tolerances, which will then apply by default to all of that author’s questions (except where over-ridden by the author at question level). A new feature permits the author to set a prefix and/or a suffix to the answerr field on the participant screen. A prefix might be (for example) “$” or “US$”, while a suffix might be “.00”, “N”, “kN”, or “%”. These make clear to the respondent the intended format of the response: the units being used, perhaps the significance level to be applied.
EXAMPLE: Consider the question: “Using a discount rate of 12% per annum, calculate the Net Present Value of a payment of £35,000 at the end of 36 months from today.”
In setting the correct answer ($24,707) the author might decide to permit a variation of +/-0.1% which will recognise those respondents who enter $24,000. By setting a prefix “£” to display in front of the answer field, the respondent is encouraged to not enter the “£” character in the answer.
Text response questions:
When setting the model (“correct”) answer for a text response question, authors are no longer restricted to a single “string” of text. Now, multiple alternate “answer terms” can be listed, each of which may or may not be identified as “correct”. Each specified term can have feedback associated with it, and each may attract a score if used by the participant. In addition, the author can choose the rigour to be applied to grammar: for example whether to enforce capitalisation, punctuation and use of spaces. Lastly, the mere inclusion of a taregt string in a participant’s response can be deemed sufficient, for example (in replying to the question in the first paragraph) a participant who answers “”probably Hitler” would still get a positive score associated with the term “Hitler”.
EXAMPLE:
Consider the question “What is the term used to describe a graph of the form y=ax2+bx+c ?”
Specified terms might be: “parabola” (correct, 5 points); “parabolic” (correct, 5 points); “quadratic” (partially correct, 3 points); “hyperbola” (incorrect, 1 point); “line” (incorrect, 1 point). Each of these might have specific feedback for the respondent. There might also be feedback and a score offered for the case where none of these terms features in an answer. By default, a participant who enters “linear parabola cubic hyperbola” would not get any points; otherwise the author might choose to allow mere inclusion of the word “parabola” in the submission to attract its points.
