Short Essay Grading Using Automated Essay Scoring Techniques

Abstract

Assessment of the knowledge is considered as one of the most important aspect of the learning process. Conventional class room learning has been shifted and developed towards m-learning and e-learning concepts. Despite its development the underlying problem of ‘How to assess learning?’ still remains. Essay type questions are considered as the most appropriate question types compared to closed ended questions to evaluate the knowledge of the students. However the evaluation of those answers generally consumes huge time, effort and unavoidable human errors such as biasness, inconsistency, etc. Therefore development of an automated essay assessment is encouraged due to these reasons. The primary strength of automated scoring compared to human scoring lies in its efficiency, absolute consistency in applying the same evaluation criteria across essay submissions and over time and ability to provide instantaneous feedback. Computers are neither influenced by external factors (e.g., deadlines) nor emotionally attached to an essay. Computers are not biased by their stereotypes or preconceptions of a group of examinees. The main objective of this research is to present a novel approach towards developing an automated essay scoring (AES) system to evaluate short essays with the combination of natural language processing (NLP) techniques and Vector Space Models (VSM) to reduce time, effort and eliminate unavoidable human errors such as biasness and inconsistency in evaluation of students' short essay answers. This incorporates NLP techniques such as lemmatization, tokenization, handling of spelling mistakes, relation of objects, negation, sentence cases and short term resolution. Besides Vector Space Models are Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA), and Probabilistic Latent Semantic Analysis (PLSA). VSMs is used to provide a score for student answers by means of providing a score by computing similarities of the model answer across a collection of student answers in a vector space model and finally predict an accurate score by the help of the NLP techniques. Importance of this approach is that we do not utilize any domain specific corpus; thus, training the system for each prompt is not necessary. Instead a semantic space is built using students’ answers. A model answer is used to measure the coverage of the answer. Further we have discussed some important implications when implementing the system as well. We obtained a correlation of 0.813 and 0.773 for the two data sets (vision & hearing data sets) with average value of human raters’ score. Finally the results conclude that there’s a significant and strong relationship between average value of human raters’ score and the system score.