SAMA: Short Answer Marking Agent
We have developed a cloud based engine that will grade free-text answers to short answer questions, those of between one word and one paragraph, normally worth between one and six marks. The system can assess any answer provided that the mark scheme clearly indicates how each mark is achieved and is fact based.
Currently the system has been trained for Computer Science questions and we are currently training it for other subjects. It marks with 94% accuracy which is the same or better than competent human graders.
Examples of questions that can be marked are such as:
Compare the usage of RAM and ROM .
We are integrating the cloud engine with an existing front end system but we can also provide an API to integrate it with your front end system.
SAMA will not mark all answers, currently it will refer approximately 50% of answers to be human-graded but then these answers will be fed back into the system to improve it's performance.
How it works
Free-Text short answer marking; as accurate as humans