Test Design Strategies

The strategies below are generally focused on making it more difficult to violate academic integrity in an online exam. No set of strategies is going to be cheat-proof, even in a face-to-face class. As noted in the first page of this section, being clear about academic integrity and modifying assessment strategies will be the most effective approach.

Assume that publisher test banks are all on the internet

The first thing students may do is Google the question. If your question is from a publisher test bank or other resource, it’s probably there, with an answer. In addition, some students use "study help" sites like Chegg.com Links to an external site.. For a monthly fee, students can post their homework questions and get "help" in the form of someone else answering the question for them. Answered questions are made available to other paying students.

Strategies

  • Rewrite the test bank exam questions.
  • For questions you've written, periodically do a Google search to see if they’re online.
  • If you find your question on Chegg, request that they remove it Links to an external site. as a copyright violation; they've posted your content on their website without your permission. This approach may work with other sites as well.
  • Leave a few questions out there to identify students who are searching for test questions.

Use large question pools

With a pool of questions, you could have Canvas randomly pull a set of questions for each student. Student A may get questions 1, 2, 8, 17, and 23. Student B may get questions 3, 4, 10, 16, and 22. Student C may get questions 7, 9, 15, 19, and 25. If Students A, B, and C all work together to answer the questions, they would have had to figure out the answers to fifteen different questions. The quiz becomes a tool for helping students learn the material, not just an assessment of the result of their learning.

Don’t grade everything

Essay questions require higher order thinking skills on the part of students. They are harder to game, especially if you have other samples of the students’ writing in a quarter. They might be harder to grade, but…

Just because each student answered five essay questions, that does not mean you have to score all five. You can randomly choose one or two questions to score. Students have to do their best to answer all of the questions because they don't know which one(s) you'll select. It is just like working customer service. While not all of the employee's customer interactions will be observed, all of them need to be high quality.

Configure the test to make sharing answers harder

The two best strategies for this are randomizing the order of questions, and randomizing the order of the answers presented in multiple choice questions.

Some instructors set “Show one question at a time” as well. It helps, a bit, but is hard on students with shaky internet connections, since the student has to wait for each question  to download. It’s also hard on students with cognitive or motion disabilities, as progress through each test requires more clicks and more interruptions.

What about time limits on exams?

There are two kinds of time limits that apply. The first is when the exam is available. If the course is online, or the exam is being given during the online portion of a hybrid/virtual course, we recommend making the exam available for at least 24 hours, and preferably 48 hours. Students are taking classes in this mode because they need schedule flexibility. This window provides enough flexibility while minimizing opportunities for integrity issues.

The other time limit is how much elapsed time students will have to complete the test. The idea is that shorter time limits make it harder for students to look up answers. We don’t recommend this approach. If the answers are easy to Google, then even the shortest time limits are not going to stop this. It favors students with quick recall or information processing skills. It’s particularly hard on students with cognitive or motion disabilities, as they can’t move as quickly through the questions. In other words, the exam will be testing things that aren’t necessarily on your class outcomes.

What about watching students take a test on Zoom or Panopto?

One approach that instructors have tried is to have students take the exam during a live Zoom session. Simply schedule the exam; students turn on their video, and you watch what happens. If you are teaching virtual or hybrid/virtual classes, this can work if the exam occurs during a scheduled class period. For online classes, you’ll have to schedule multiple exam opportunities including evening and weekends, since online classes can’t require a fixed class time.

A similar approach can be taken with Panopto. (Yes, Panopto can be used by students as part of a class too.) This is more effective for online classes, because students can take the exam during a time window that works for them. This works best if you have some other indicators of how a student is doing; you can use Panopto to spot check for student actions instead of watching each of the videos.

For either approach you’ll have to accommodate a number of challenges:

  • Students doing their course work on smartphones won’t be able to share their screen, or do the test and show their video easily.
  • Students with unreliable internet connections will have to re-start/re-connect multiple times during the exam. You’ll need to extend their exam time accordingly.
  • Students may not be able to show their video or have an open audio channel, for instance if they have someone in their household they don’t want shown, or are taking the exam in a noisy public place.