Proofreading Assignment and Exam Problems

Introduction 

One concern students have is assignment and exam clarity. This is not an infrequent concern — writing clear and precise assignment and exam problems is more difficult than it might initially seem, and students in many classes identify it as an area for improvement when filling in student ratings at a course’s end. 

Sometimes TAs are asked to write the problems themselves. More commonly, however, TAs are asked to proofread problems prepared by the teacher, or perhaps by a head TA. This note provides tips on checking problem statements. 

Preliminary Questions 

Before reading farther, think about the following two questions: 

1. In the classes you have taken, when have problem statements been clear, and when they have not? More specifically, what are the characteristics of well-written problem statements? What are some possible characteristics of poorly-written ones? 
    
2. What advice you would give another TA on how to check problem statements effectively? 

Some Tips 

Here is a list of tips for checking problems statements written by the teacher or another TA. (This list is not comprehensive, so think whether there are any additional tips that would be useful.) 

1. Review problem statements, assignment instructions, etc. for precision, accuracy, and clarity. That is, actually do what this note suggests — check problems statements carefully. And, if possible, have someone other than the writer review problem statements. It is all too easy to make mistakes or include ambiguities when writing problems. Rechecking problems statements and/or having someone else proofread the problems is invaluable. If at all possible, do this in advance of when the problems are given to students. However, even if it is not possible to proofread in advance, or even if the teacher did not explicitly ask you to proofread, still make sure you do it, if for no other reason than to ensure you understand the problems. Moreover, do it soon after the material is available rather than waiting, say, until soon before an assignment is due. 

2. Remember the audience. In particular, what might be “obvious” to you as a TA or to a teacher might not be obvious to students in the class. So read from the perspective of a student in the class. 

3. If possible, have multiple TAs proofread high stakes problems. Proofreading is hard, and one TA might not catch all the ambiguities, etc. So on high stakes work such as exam questions it is useful to have two or more TAs proofread problems. 

4. Make (at least) two passes through the work. A common proofreading strategy is to make one pass for “higher-order” issues such as whether a problem statement contains all the needed information, and another for copy-editing, i.e., checking for typos, misspellings, etc. 

5. If possible, actually do the problems yourself. This will give you more in-depth familiarity with the problems than if you just read through them. 

6. Gauge whether the problem is at the correct level of difficulty and duration. Is the problem doable? Is it too hard or too easy? About how long will it take, and is the expected amount of time reasonable? (This is particularly important for exam problems.) 

7. Be particularly attentive to ambiguities. Especially in a large class, if there are multiple ways a problem can be interpreted, then there are multiple ways the problem will be interpreted. Learn to identify possible points of confusion. 

8. When reviewing the questions, think about how you would grade them. In fact, an excellent way of checking a problem statement is to develop a rubric for grading answers — what characteristics should a good answer have, or what criteria should it fulfill? Doing this is a good way to find possible ambiguities or to identify unstated expectations. (“Do we need to specify that the code be written in a certain style?” or “There is more than one way to solve this problem — is any approach OK, or do we need to specify that students use a particular approach?”) It might also be helpful in making grading more manageable. As an example, if a problem asks for a direction given as a 3D vector, then it might be useful to specify that the vector be normalized (i.e., have length 1). This will limit the number of possible correct answers — and therefore make the grading easier — and address the possible ambiguity of whether the answer needs to be normalized. 

9. Make sure any special instructions are explicitly stated. For instance, if a program solution will be graded electronically based on its output, is there any particular output format? If a problem can be done multiple ways, is there any approach that is required, or any that are disallowed? 

10. When copyediting, use effective copyediting strategies. As one possibility, some people find that reading the problem statement aloud helps them find grammatical mistakes or misspellings. There are a number of excellent online resources for copyediting, such as the relevant tips at the editing and proofreading strategies page on the University’s Center for Writing site. 

A Problem Statement Proofreading Checklist 

Here is a checklist, based on the items above, for checking problem statements. This list is just intended as an example. Not all the items here will always be relevant, and some types of problems will require additional items. 

• Is the problem statement written for the correct audience, i.e., is it written so as to be understandable by students in the class rather than a different audience?

• Is the problem at an appropriate level of difficulty, and will it take a reasonable amount of time?

• Is the terminology and notation used consistent with what is used elsewhere in the class?   Are there any ambiguities in the problem statement?

• Is the problem statement free from distracting misspellings, etc.?

• Does the problem statement explain the grading criteria as needed? For example, does a code solution need to include useful comments in order to get full points?

• Are there any special instructions (e.g., format for the answer, submission instructions), and if so, are they explicitly stated? 

Feedback Questions 

Often teachers will ask you to check a problem statement (or entire exam or assignment), and then get back to them. They might ask for specific feedback such as whether there are typos, grammatical errors, etc. However, they might not specify exactly what they are looking for. If they do not, here is a list of items that teachers often find useful: 

1. Difficulty Level: How difficult do you think students will find the problem? Will it be too difficult, difficult but OK, moderate, easy, or too easy? If you are reading a set of problems, for example in a multi-problem assignment or an exam, how difficult is each problem, and how difficult is the entire assignment or exam? And is there a good mix of easy, moderate, and difficult problems? 

2. Duration: How long will the problem (or set of problems) take an average student? An overly long amount of time, a long amount of time, a moderate amount, a short amount? This is one place where doing the problem yourself will likely help. And remember that it will take students longer to solve a problem than it takes you. As an example, in an introductory class, if a problem takes you five minutes to solve, it will likely take a student between two to four times as long, so 10 to 20 minutes. Also remember that easy problems might take a long amount of time if they involve a large amount of writing. A code solution, for instance, might involve a straightforward set of steps, but take a long time due to requiring many input and output statements. 

3. Completeness, Clarity, Precision, etc.: Are there any places where more details or instructions are needed, where the problem statement is ambiguous, where it uses non-standard terminology or notation, or, more generally, where the problem statement should be improved? 

If You Need to Construct Problems 

If you need to construct problems for a lab, assignment, or exam, many tips here are still relevant — write for the appropriate audience, make sure to explicitly state any special instructions, etc. And, if the course teacher does not proofread the problems for you, then if possible get another TA to review the problems. 

Practice 

Suppose the following problem was on an exam for one of our CS 1 classes. Read through it, and then either evaluate it based on the checklist above, or answer the three questions in the “Feedback Questions” section above. 

Problem Statement: Write code [assume the language used is C++, Python, or Java — you can decide which] that, given a sequence of numbers, computes and gives the number of zero crossings in the sequence. A zero crossing occurs each time a positive number is followed by a negative one, or vice versa.