From giving quizzes at the end of a lecture or module to evaluate student learning to midterm or final exams to test their cumulative knowledge, the Canvas Quiz tool can be an effective assessment tool. While quiz building and implementation can be facilitated with the assistance of a CCAPS-ATD team member, this tutorial provides a brief introduction of what to consider when building your quizzes in Canvas. This guide can be useful in discussions with your instructional designer about what to consider when designing quizzes.

What Are the Goals for Your Canvas Quiz?

Before creating the quiz, you will want to consider what kind of quiz you want to employ. There are two types of assessment approaches to consider: formative and summative.

Formative quizzes, in general, are meant to check understanding. After a lecture or a reading, you want to be able to evaluate key pieces of knowledge students were able to digest. A formative quiz will allow you to ask questions and see how students respond. If there is a particular question in the quiz that students are having trouble with, it opens up the opportunity to give more focus in reinforcing that knowledge as opposed to questions that students were able to answer correctly.

Summative quizzes are meant to test cumulative understanding during certain milestones or at the end of the course. Midterm or final exams are examples of summative assessments.

Since formative quizzes are meant to check knowledge to help shape understanding, the quiz design will typically be low stakes. This means its weight in regards to the final grade is lower. In a low stakes quiz, fewer questions are typically asked and there may or may not be opportunities for students to retake the quiz. These quizzes can typically be built very straight forward. Multiple choice and True/False questions can typically be sufficient and depending on the goals of the formative quiz, a robust quiz bank may not be necessary.

Summative quizzes, on the other hand, are high stakes since the grades have more weight, more questions are asked, and they typically don’t have opportunities for retakes. For these types of high stake assessments, cheating can be a problem. For high stake assessments, there is the option to use Proctorio for remote proctoring. Other options that help lessen the chance of cheating include having more essay quiz questions since you can check for more genuine/unique answers that way. Creating a robust quiz bank so that each student doesn’t get the same quiz can also help mitigate cheating risk.

Question Types

In Canvas quizzes there are two broad categories of question types to consider: Automatically graded and Manually graded.

Automatically graded questions include Multiple Choice, True/False, Multiple, Fill-in-the-Blank, etc. While these are considered “automatic,” they do require instructors to provide a predetermined answer when building the quiz. For instance, Multiple Choice questions, how students should answer and be graded is straightforward since they have a discrete number of options from which to choose. This is the case for True/False, Multiple Answers, Matching, and Multiple Dropdowns. Question types such as Fill-in-the-Blank and Fill-in-Multiple-Blanks are trickier since they compare the student’s text response to a predetermined string or multiple strings of words. If what students entered doesn’t exactly match one of the predetermined strings, the answer will be marked as incorrect. This means even an extra space or alternate formatting that was not entered would be considered incorrect even if you would deem it acceptable. To make any Fill-in-the-Blank question work you either need to anticipate every potential answer in the question, guide students in the question on proper answer formatting or manually override the grade on a case-by-case basis.

In Canvas New Quizzes, there are additional features for how to text match for Fill in the Blank questions which are more robust. The simpler “Contains” and “Close enough” text match can approximate responses in different ways. The most interesting text match is the robust Regular Expression match which has the benefit of creating code to assess the question. That said, using this feature requires a high-level understanding of Regular Expressions as well as an understanding that whatever expressions you write might not always account for all variations. For more information on these types of questions please read the Faculty Story Construction Plan Reading Homework.

Manually graded questions include the Essay or File Upload type questions. You will need to review and grade any text entered by students or any files students upload, These question types are used for essays written during exams of any length. The file upload feature is useful if you want students to have a place to submit worksheets, if you have an exam where showing the process, like showing your math, is important to the evaluation of the exam. In many cases where instructors want to have Fill-in-the-blank type questions, we typically recommend using Essay type questions since instructors may have to override those grades.

Quiz Banks

Quiz banks are a very useful tool in Canvas. Other than being a repository to pull questions from, a robust quiz bank where you can randomly pull questions is one tool in your arsenal that can deter cheating. There are many ways to use banks. You can create a single bank that has a pool of all the questions for a quiz or you can create multiple banks each containing a different question type that you can use in the same quiz. Banks can also be used in many quizzes in the same course. Here are a few examples of quiz bank configurations that are available:

  1. One instructor created a formative quiz with a quiz bank containing 20–50 questions. For each individual student quiz, 10 questions were randomly pulled from each bank pool of 20–50 questions. No two student quizzes were alike. The Final Exam was then built by pulling 6 questions from each of those quiz banks for each module.
     
  2. One instructor wanted to have a set of banks setup so that one would contain all of the multiple choice and true/false questions pulling a certain amount for the Final Exam at the beginning and then at the end there was a pool of essay type questions where three questions were selected from the pool. Using this quiz design method, the exam was guaranteed to have a set number of randomized multiple choice type questions and a set number of randomized essay questions.

Some people may be daunted by the idea of creating a large number of questions for quiz banks but in reality, there is nothing to fear. The initial time investment doesn’t have to be that big. To start, you can just make a bank to have only a few questions over the number of questions you want to have for a particular quiz if you want to pull them randomly. The rule is whatever number of questions you are pulling from the bank, have one or more question in the bank. Thus, if you want there to be five questions randomly pulled from a bank for a quiz, just have six questions in the bank. Over time you can add more questions to that bank making it more robust as questions are added. In some cases, textbook publishers provide banks that can be loaded into courses so you could have a class built entirely of questions you created, questions the publisher created, or a combination of the two.

Proctorio

Proctorio is the remote proctoring tool that has been selected by the University of Minnesota. Proctorio works as a plugin to Canvas Classic Quizzes and its function is to make sure the correct student is taking the quiz, ensuring the environment is not conducive for cheating (e.g., Protorio prevents students from using second screens, printing the exam, or opening other browser windows/tabs, etc.). Proctorio records the student and the student’s screen while the test is being taken. The Proctorio AI analyzes what the student is doing during the quiz and creates a risk of cheating score based on incidences it flags. Only the instructor of the course can easily navigate to and analyze the flags. Proctorio is not a replacement of the Canvas quiz tool so whatever quiz you build in Canvas Classic Quizzes can have Proctorio turned on for it. Therefore, making changes to the quiz which uses Proctorio is no different than a normal quiz.

Proctorio should generally be used with summative exams and not used where dishonestly or cheating is less of a concern. There are three considerations for understanding the risks and why we also recommend limiting its use:

  1. The software can be finicky for students. Typically, we create a practice test early in the semester for students to complete to test the functionality of the tool on the student’s computer. The software should be able to assist students with using the correct browser (Google Chrome) as well as getting the correct plugin. While complications are becoming rare, they can happen. As such, problems can usually be dealt with at a less stressful time, as opposed to trying to troubleshoot the problem during a narrow window when an exam is being offered.
     
  2. Time and planning for the correct environment may be needed. The demands of streaming video may be difficult depending on the student’s access to a good internet connection. Students will also need to find a place and time where their environment is quiet and has limited distractions that can trigger the Proctorio AI. Planning for the students to sit for their quiz in the correct conditions needs to be considered.
     
  3. The software can be invasive. While the idea of proctoring is the ability to monitor what a student is doing during an exam, these traditionally happened on campus in a set, neutral area. Proctorio, while limited to the instructor’s view in theory, potentially opens up privacy issues. Since Proctorio is recording the student on their webcam, having a glimpse into the student’s home/personal space can be unintentionally transgressive. There is also a growing sentiment of mistrust in technology in regards to data security. Some students may not be comfortable at all using the software, much less using it on a regular basis. You might need to consider alternatives for students that are unwilling to use Proctorio such as doing a proctoring session over Zoom.

New Quizzes or Classic Quizzes?

Before 2020, Classic Quizzes was the only quiz tool available in Canvas. Instructure (the company that owns and updates Canvas) decided to update the tool to change the interface, add new types of questions, as well as many other features. The intent was to replace Classic Quizzes. While New Quizzes may sound like an attractive option, there are problems with it which is why Classic Quizzes still exist in the UMN Canvas distribution. Some of these problems include:

  1. The old quiz banks do not work with New Quizzes. New Quizzes uses its own new quiz bank system. 
  2. Problems with functions of the New Quizzes quiz bank includes:
    • It is not possible to move multiple questions to another bank. It must be done one question at a time.
    • There isn’t a search across quiz banks.
    • Description fields do not exist for quiz banks. Teachers, unless they know the purpose of the bank by the title, might put questions in the wrong banks .
    • There is no way to export quiz banks.
  3. The Rich Content Editor is not as robust. Certain pieces of media cannot be implemented properly in some cases.
  4. Navigation to some of the quiz settings is convoluted and requires a round-about way of access.
  5. There isn’t a clean way to create a survey, like you can with Classic Quizzes.
  6. Proctorio cannot be used.

For the time being we recommend using Classic Quizzes unless you want to utilize some of the unique New Quizzes question types.

Final Details

After considering the types of questions you want to use, whether you will use a bank, or even if you plan on using Proctorio, here is a list of other details that needs to be considered for building any quiz:

Points per Question 

Quizzes are graded using the total of points you assign to each question. Therefore, it is good to know the point value of each question. Typically, Multiple Choice and True False will have a lower point value opposed to an Essay question.

Time Limit

It is important to know how long you want a student to have in the quiz once they start taking it. For formative quizzes this might not be a factor, but it might be for summative quizzes. 

Number of Attempts

For formative quizzes, you may want to give additional attempts for students to retake the quiz. For summative exams you want to stick to the default 1 attempt. If for any reason you want to give students another chance at taking the exam, you can grant discretionary attempts. The tutorial Adding Attempts in Quizzes will show you how. Canvas has three attempt grade settings to consider: 

  • Highest: gives the student the highest grade out of all the attempts
  • Latest: gives the student the grade for the last attempt
  • Average: gives the student points based on the average of all the attempts

Due Date

This informs the student when the quiz is due and shows that date and time. With any assignment or actionable item we recommend adding a due date. For more information, see our tutorial on Due and To-do Dates. 

Note: Students will be able to submit questions past the date unless an “Available Until” date is set which closes student access to the quiz.

The "Assign to" section of the assignment, discussion and quiz tool settings. It shows off the "Due" date field as well as the "Available from" and "Until" fields.

Available from and Until

As mentioned previously, the due date is meant to communicate an expectation of when an assignment or quiz is due. The Available from and Until dates and times give a window when students can access materials in the course, in this instance the quiz. If you want to close access to a quiz at the time of the due date, place the same date in the “Until” field to close off access to the quiz. Use these settings for blocking access to quizzes rather than using the publishing/unpublishing feature.

Feedback 

Each question has a place for feedback that can be given for each question after the quiz has been taken. You can give prewritten general feedback that is always given and you can give prewritten positive or negative feedback which is given either if the question was answered correctly, or incorrectly. There are also various settings when feedback can be given. Feedback can be granted right after a quiz has been submitted or it can be granted at a certain selected time, typically after the due date. If you use the “Until” date feature to close off access to the quiz, and grant the feedback after that date, the feedback will not be accessible to them.

Accessibility

When making questions, accessibility is a very important consideration. Please review Start with the 7 core skills for online accessibility. In general:

  • Don’t use raw URLs, use descriptive titles and text for any links, if utilized.
  • Screen readers can not read images.
    • Images need descriptive alt text. Describe what is being seen in the image in such a way that a person can get the information they need from it without being able to see it. 
    • Do not use images that have text or tables in them. Use the text editor field to type out the text and tables that can be read by screen readers. 
  • Videos need captioning.