I have overseen two Questionmark Perception exams in the last two weeks and this has highlighted some interesting behaviour. Here is the first.
The lecturer created their own exam. In the results, all students scored zero for one question. The lecturer told me that on looking at the question, all the 1s in the correct responses are now “f”, so no marks were given as no student entered f in their answer (the system wouldn’t let them anyway). This is very strange. It was working in mid-December, when the lecturer entered a full set of correct answers but was not working by the exam a month later.
What could have caused this? Well, I had a play and came across the following behaviour. The numeric input boxes (“choices” in QMP) are numbered by default. If you change the label from a number to something more identifible, it changed the reference to that choice in the logic that determines whether the answer is correct (the “outcome”). But watch what happens:
So I hypothesise that the lecturer renamed choice 1 to choice f between mid-December and the exam, and the system did the rest.
I realise this could be demonstrated even more straightforwardly. If the answer contains zero and choice zero is renamed the following happens.
Peter
Many thanks for posting this to your blog – I was directed toward this by sgwarnog, a follower of yours in Twitter.
I have raised this issue with QM support who have escalated it so I hope we will see a fix eventually – in the meantime, the workround is to change the outcome back to the original value – not a genuine fix, but it does work.
By the way, if you are a regular user of QMP and work in the UK HE sector, you might be interested in joining our usergroup at questionmark@jiscmail.ac.uk
Thanks again
John Dermo
e-Assessment Adviser
Academic Development Unit
University of Bradford
Comment by John Dermo — 29 January 2010 @ 2:03 pm
Hi John,
Good to hear from you. I reported it to our central support in the university as well.
You’re right – the solution is to make sure renaming the choices is not the last thing you do to a question, or to not change them at all.
I will look at the mailing list you suggest, thank you. It would be nice to make contact with others.
Peter.
Comment by peterrowlett — 2 February 2010 @ 10:14 am