Keyword-Based Processing for Assessing Short Answers in the Educational Field

Abstract

When grading open-ended engineering exam responses, it is assessed to what extent its content and quality suit the requirements and accomplish the objectives of the test. This is a time consuming and subjective task. The support of a software tool that identifies the correctness of the response and provides useful feedback to both student and teacher may alleviate its complexity. In this work, a semi-automatic evaluation method based on augmented Spanish keyword recognition is presented. This assessment is based on the occurrence of a set of keyterms that the teacher expects to appear in a good response. The evaluation is based on an augmented catalogue of terms, automatically created from the teacher selected keyterms, resulting in an ad hoc thesauri. The method uses state-of-the-art techniques, but also ad hoc procedures developed from the Spanish corpus Wikicorpus, for pre-processing the texts. The results, tested using real anonymized data from engineering exam topics such as database techniques and bigdata, indicate good performance in the thesauri creation and keyword matching. Besides, the keyterms strategy allows simple individualized feedback. However, the relationship found between automatic and human grading indicates that further research is required.