Determining the Quality of Assessments

In this section, we will discuss characteristics of assessments that are used to measure their quality — validity, reliability, practicality, and fairness. These characteristics apply to all the categories of assessments we discussed in the previous section.

Validity may be the easiest quality of an assessment to evaluate. An assessment is valid if it measures what you expect it to measure. For example, if you want to determine if your students have mastered the information from a history unit, that assessment should only include questions about the material in that unit, and it should assess all the important knowledge you expected them to have gained from your instruction of that unit. In that way it is a valid measure of whether students learned the information in that unit.

While it is obvious that an assessment of mastery of a history unit is not valid if it includes questions about material in an unrelated mathematics lesson, there are less obvious ways an assessment might not be valid. For example, if students need to have certain test-taking skills to complete an assessment, the assessment is not valid if they do not have those skills. In that case, the assessment is measuring their ability to complete the test as well as their knowledge of the unit content. Teachers should check that students know how to complete the types of items on a test before giving them an assessment that includes those types of items. For example, if you are going to have the students write an essay for a summative assessment, consider first giving them a formative assessment that is an essay. This will allow you to determine if they are ready to complete an essay for a graded assessment.

In general, a valid assessment would include only information students have been taught and would not require reading skills, writing skills, test-taking skills, or any other skills which students may not yet have.

An assessment is reliable if the results are generally consistent each time it is given to students. There are several ways you can ensure the reliability of an assessment. First, it is important to use consistent instruction across groups of students you teach. When instruction is consistent, each group is equally prepared to complete the assessment and the results should be similar. This does not mean you would never make changes to improve your instruction or your assessments, but you should be able to attribute differences in students’ performance on an assessment to those improvements you made.

You can also ensure the reliability of assessments by providing each group of students who take the assessment with clear and precise directions for completing it. They also should have specific information about how you will evaluate their work on the assessment. For example, you could give them a checklist or a rubric that describes the criteria you will use for grading the assessment.

If an assessment is reliable, you should also find that individual students perform similarly on each section of the assessment. That is, for example, they should show a similar understanding of the material being tested whether they are answering multiple-choice questions or writing an essay.

Finally, the conditions under which the students take the assessment could affect their performance and, therefore, the reliability of the assessment. If your students typically take the assessment in a quiet classroom, but one year the class had to complete it while loud construction was taking place outside the building, the difference in setting may affect the reliability of that assessment that year. In general, if there are any conditions that would affect student ability to complete the assessment successfully, other than their knowledge of the material being assessed, it will affect the reliability of the assessment.

A practical assessment is one that requires a reasonable amount of time for you to write, for students to complete, and for you to grade. It is an assessment that is not expensive to administer and one that does not require more adults to administer it than are available in the classroom.

As you design assessments for your students, consider the amount of time that will be available for them to complete the assessment and write an assessment that can be completed in that amount of time.

Use assessments that will take a reasonable amount of time for you to grade and to review with the class after grading. You should give feedback to the students soon after they complete any assessment, so try to administer it when you know you will have time to grade it and return feedback to the students soon after they take it. Do not, for example, give the students a test just before a holiday break. It is unlikely they would benefit from feedback given to them long after they have completed the assessment.

Further, when designing an assessment that requires individual administration, such as a performance assessment, ensure there will be both enough time to do the individual assessments as well as a way to keep the other students occupied during that time. This may require assistance from other teachers in your building, so you will need to check their availability and prepare a plan with them.

The fourth characteristic of a good assessment is fairness. A fair assessment is one that asks students straightforward questions about only the information they have learned in your class and does not use complex wording that may lead them to answer questions incorrectly.

A fair assessment does not include cultural knowledge that some students do not understand.  Even simply mentioning popular musicians or sports teams in an attempt to make the assessment engaging could put some students at an advantage because they are more familiar with these topics in popular culture than other students.

Fair tests are administered in a manner that gives all students an equal chance to complete them successfully. For some students, this means providing the accommodations they are permitted to have such as more time for the test or someone to read the test to them. Teachers must take care that limited test-taking skills or a disability does not put any student at a disadvantage.

A fair assessment is also one that you will have adequate time to grade, giving each student’s work equal attention and providing sufficient feedback to each student.