Technology can offer the opportunity to create learning moments even when the teacher is not there to grant feedback. In one course, quizzes have been utilized in a unique way so students can practice their course materials while getting formative feedback through the quiz tool before going to the instructor.
Practice Makes Perfect
Wanting to improve her skills with public speaking, Heather Kossilla began teaching in the Construction Management program in 2014. Teaching both Construction Plan Reading (CMGT 3011) and Managing with Building Information Modeling (CMGT 4003), she now enjoys teaching and says she has fun interacting with students.
“In order to learn how to read plans: one must read plans. The more you do it, the better you get at it.”
As a part of her Construction Plan Reading course, she had weekly homework assignments to reinforce the practice of reading plans. But for students new to reading real construction plans it can be quite difficult at first. Yet the only way of getting better at anything, especially in regards to plan reading, you have to keep reading plans. Thus the question was: how can you have homework assignments that both challenges students and also guides them if there isn’t an instructor immediately present?
Her solution was to use the quiz tool to have students answer questions in the fill in the blank or short answer format that could be checked immediately in the quiz offering quick feedback on incorrect answers. At the time Heather designed the homework assignments, Moodle quizzes had the ability to grant multiple tries with successive feedback on each attempt helping nudge students closer to the proper answer they could find in the plans.
With the benefits of successive feedback being obvious, there is one elephant in the room: the choice of using the fill in the blank or short answer format for the homework assignment quizzes. This format in any learning management system has its inherent challenges. Short answers are assessed on matching a string (sequence of characters) or series of strings exactly to be considered a correct answer. Possible strings are case sensitive as well as sensitive to punctuation that may not be inappropriate for the answer. This means that unless students are able to exactly match up character-for-character the correct answer, students would have questions marked incorrect. This format can be rather rigid with students. Thus instructors are required to constantly intervene to check what was entered and override the grade given by the computer. For instructors that don’t want to have to manually evaluate and grade questions, having the computer try to evaluate what a student is entering is considered the “correct” answer is the biggest reason why instructors opt to use multiple choice or true/false questions for formative quizzes.
“I wanted to make sure they were finding the answers. It meant that they were reading the drawings.”
In Construction Plan Reading, the problem of having many different forms of a correct answer is especially true. When talking about materials something like “Ponderosa pine lumber” can be entered as “ponderosa pine” or even just “pine” as well. When trying to give an answer that is a measurement, there are so many ways to represent it. Four feet can be represented as:
- 4 feet
- 4 ft
- 4 ft.
- 48 inches
- 48 in
- 48 in.
This answer will not only accept all the options shown above, but will also accept the answer whether the student puts a space between 4 and ' or not, or even if the student gives a long winded answer like "the answer is 4ft and 0 inches." Having a variety of possible answers, Heather’s concern was that students were finding what was required in the plans. Answers that had grey areas such as typos and slight misspellings was unimportant in regards to an answer being considered correct.
In Moodle there was a solution with the Wildcard feature. In short, the Wildcard feature would allow you to enter a part of a string that was considered essential to the correctness of the string and use an asterisk (*) in areas to omit sections that were not considered vital to a string being marked as correct.
New Opportunities and Challenges
Moving to Canvas, for many, was a great upgrade. In regards to Heather’s homework assignments, it was at first a giant step backwards. Canvas developed quizzes anticipating that instructors would create most quizzes utilizing more basic forms of questions including multiple choice, true/false and matching type questions with the option of essay questions that would need to be manually graded by instructors. The other difficulty is that quizzes were developed to have students sit for a finite amount of time, to complete the quiz in one sitting, and to be evaluated at the end. These quiz characteristics placed huge barriers in front of Construction Plan Reading’s transition from Moodle to Canvas. This was until Canvas launched it into Beta Quizzes.Next (Now called “New Quizzes”) with a new fill-in-the-blank type that offers four different ways for the computer to evaluate questions. The evaluation method most appropriate for Heather was the Regular Expression Match.
A Regular Expression or RegEx is a language used in some programming applications to create a sequence of characters that defines a search pattern. Regular Expressions are considered more robust than using Wildcards but also have a steeper learning curve since it has a very rigid syntax. The other part is when creating a regular expression you need to anticipate potential variations in a proper answer since those variations need to be accounted for in a regular expression. For the example cited above of the answer of four feet for the answer of four feet, a regular expression statement would look like this:
Heather worked with me during the Summer of 2018 on transferring her questions into Regular Expressions in new quizzes, I had her read up on resources on regular expressions as well as regular expressions in Ruby since that was the variety that Canvas used. We tested out our regular expressions using Rubular to see if they worked according to a list of possible answers we thought students might answer and placed them in the quiz questions. Knowing that there might be the chance that we couldn’t account for all potential answers, we created a document for Heather to track answers students have entered that she considered correct so that by the next semester she could update those questions.
According to Heather there were some bumps and needed changes for the quizzes in the first semester. To account for these variances from the regular expression she talks to her students during in class sessions about the homework and gives them the opportunity for them to dispute how the computer evaluated their question. Every subsequent semester afterwards the incidences of answers that were not matched with the regular expressions became less and less as variances were accounted for.
While the challenge of evaluating short answer questions was met, the ability to have gradual feedback on incorrect answers was not in the same way as offered in Moodle. Quizzes in Canvas are set up to give feedback for correct answers, incorrect answers and general feedback when the entirety of a quiz is submitted. This is unlike Moodle that had the ability to grant successive feedback for each attempt at the question. This was accommodated with allowing for multiple attempts on the quiz but is far from the ideal method.
Are Regular Expressions Right for You?
When I asked Heather if she had any tips or suggestions if faculty are interested in using regular expressions in their quizzes she said “Yeah. My tip is to contact Paul.” (Thank you Heather.)
"If you can find that extra bit of patience, if you can afford just a wee bit more time, the payback for you and your students could be twofold. Homework questions are quite forgiving, and yet require the student to think a bit more, AND grant a more robust automatic grading system for you!"
While I welcome the challenge to create regular expressions in quizzes but there are two reasons I feel this worked:
- Dedication: Heather was tenaciously dedicated in regards to getting the quizzes setup the way she wanted. She was so dedicated that she even taught me a few things about regular expressions in Canvas. Regular Expressions is not an easy topic. Even for me. It is something that even the best computer programmers can find room for improvement in regards to setting up expressions.
- Patience: Patience with yourself and your students. There is a lot of time you will have to initially invest in when creating the expressions which also means testing out variations. This also means accounting for the fact that students will surprise you with variations when they eventually take the quizzes. These need to be accounted for in the future semester course instances since the recommendation is to not make any changes to quizzes which have already been taken by students. Thus expect there to be wrinkles that will need to be ironed out over time. Communicate these bumps to students.
If you have any more questions or you need assistance with making additions or changes, we welcome CCAPS faculty to reach out to us for assistance through the CCAPS-ATD Learning Management Assistance form.