Content experts' assessment of test questions in an online graduate course

Consuelo Dl Habito and Kunchon Jeotee¹
University of the Philippines Open University
Los Baños, the Philippines
¹ Visiting Professor from Sukhothai Thammathirat Open University
Nonthaburi, Thailnd

Assessment in higher education can mean different things to administrators, programme chairs, faculty and students in an educational institution. It can provide information about student progress and learning, teaching quality and effectiveness, and a programme's quality and accountability. Assessment in an online graduate course can be approached in two ways. One approach is to have developed test items in a question bank undergo a form of qualitative and quantitative assessment on their validity using content experts. The other approach evaluates how students have answered the test questions using facility and discriminative efficiency indices, which can identify specific test items that can be improved or modified.

This study aims to assess whether the set of test items in the quiz bank satisfy the learning objectives stated in the modules of a graduate course. Six content specialists evaluated the validity of each of the 145 test questions in a question bank against the learning objectives of the course modules, using the index of item-objective congruence (IOC) (Rovinelli and Hambleton 1976). The content experts' assessment of 140 questions showed a high level of item-objective congruence (96.6%) (IOC >= 0.5) of the test questions against the learning objectives of the course modules. However, five questions showed a low level of item-objective congruence (IOC <= 0.5) and should be deleted from the test bank. Statistics or measures of assessment are important for improving the development of test items and should be further pursued to enhance the quality of learning and delivery of course content. While examinations can be a tool of assessment for knowledge content about the course, it is also important to know whether the analytical and other technical skills of the students finishing the course have also been developed.