e-Assessment

Knowledge assessment

What is e-Assessment?

e-Assessment or electronic assessment is one of the domains of e-Learning,1) Dube, Tendai, and Minhua Ma. A Flexible E-Assessment System Inspired by Design Methodology Management to Accommodate Diverse Learning Styles. International Journal of Digital Society 1, no. 1, March 2010. and usually refers to an electronically delivered assessment.1)2) Ridgway, J., S. McCusker, and D. Pead. Literature Review of e-Assessment. Bristol, UK: Nesta Future Lab, 2004.

The Joint Information Systems Committee (JISC) defines e-Assessment as

“the end-to-end electronic assessment processes where ICT is used for the presentation of assessment activity and the recording of responses. This includes the end-to-end assessment process from the perspective of learners, tutors, learning establishments, awarding bodies and regulators, and the general public.”3) The Joint Information Systems Committee (JISC). Effective Practice with e-Assessment. HEFCE, 2007. , or

“... a range of activities in which digital technologies are used in assessment. Such activities include the designing and delivery of assessments, marking – by computers, or humans assisted by scanners and online tools – and all processes of reporting, storing and transferring of data associated with public and internal assessments.”

e-Assessment types

e-Assessment is usually divided into divided into:

computer-assisted assessment or computer-aided assessment (CAA) - “practice that relies in part on computers – for example, use of online discussion forums for peer-assessment”

computer-mediated assessment (CMA)

computer-based assessment (CBA) - “assessments delivered and marked by computer”

online assessment.

Advantages of e-assessment

As learning is nowadays more and more facilitated by electronic technologies, it is expected that they will prove to be central in the assessment process too. Technology is expected to facilitate testing and enable authentic assessment.3)

“The variety of applications of e-assessment reported and their innovation and general effectiveness indicate the potential of e-assessment to significantly enhance the learning environment and the outcomes for students in a wide range of disciplines and applications.”4) Roadmap for e-Assessment Report for JISC, Open University, 2006.

The advantages of e-Assessment can roughly be divided into following categories:5) Noyes, Jan M, and Kate J Garland. Computer- vs. paper-based tasks: are they equivalent? Ergonomics 51, no. 9: 1352-1375, September 2008.

Interface richness. Interface richness enables for example dynamic presentation of the test content, adaptive level (tailored testing) and speed of the assessment or innovative item formats.6) Computer-Based & Paper-Pencil Test Comparability Studies. Test, Measurement and Research Services Bulletin. Pearson Education, Inc., 2009.

Accessibility. An e-assessment can be accessed using just a personal computer, and Internet connection. In case of a survey, this also allows a much more diverse sample of population to take the test.7) For example: Carlbring, Per, Gunnarsdottir, Magdalena, Hedensjo, Linda, Andersson, Gerhard, Ekselius, Lisa and Tomas Furmark. Treatment of social phobia: randomised trial of internet-delivered cognitive–behavioural therapy with telephone support. The British Journal of Psychiatry 190, no. 2: 123 -128, February 2007.

Standardization. The test is presented at the same time (if desired) in the same format to everyone.

Results processing. Even if automatic correction and grading is implemented just partly, it saves time and human resources, is cheaper and enables faster and richer feedback. It also enables easier results processing to get feedback on the test items and overall result of all attendants. Also, test responses do not contain handwriting, an element prone to subjective judgements of the evaluator.

Measuring additional parameters. Computers enable measuring parameters like time required for making a response8) Kröhne, Ulf, and Thomas Martens. 11 Computer-based competence tests in the national educational panel study: The challenge of mode effects. Zeitschrift für Erziehungswissenschaft 14: 169-186, May 2011. , or record when an answer previously answered question was changed.

Disadvantages of e-assessment

Disadvantages of e-assessment include:

Computer proficiency and typing skills requirement. Participants with less computer proficiency or typing skills will require more time or help.9) Zandvliet, David, and Pierce Farragher. A Comparison of Computer-Administered and Written Tests. Journal of Research on Computing in Education 29, no. 4: 423-38, 1997. 10) Russell, Michael. Testing on Computers: A Follow-up Study Comparing Performance on Computer and on Paper. Education Policy Analysis Archives 7, no. 20, 1999.

Computer screen properties and limitations. Although the characteristics of display devices have improved drastically during the last years, “the amount of information comfortably presented in a computer display is only about one third of that presented by a standard piece of paper”11) Clariana, Roy, and Patricia Wallace. Paper–based versus computer–based assessment: key factors associated with the test mode effect. British Journal of Educational Technology 33, no. 5: 593-602, November 2002. , reading from a computer screen is still slower and more tiring than reading from a sheet of paper12) Dillon, A. Designing usable electronic text: Ergonomic aspects of human information usage. London: Taylor & Francis, 1994. , and some studies suggest students who read texts from a computer screen find those texts “significantly more difficult to understand, less interesting, and the authors less credible.”13) Murphy, P. K., J. F. Long, T. A. Holleran, and E. Esterly. Persuasion online or on paper: a new take on an old issue. Learning and Instruction 13, no. 5: 511-532, October 2003.

Computer hardware and software characteristics. A malfunction, need for a restart or software failure can all affect the course of an assessment.

Costs related to the start-up and maintenance. These costs include hardware, software, networking, and wiring costs, and the development and maintenance of item pools.14) Patelis, T. An Overview of Computer-Based Testing. The College Board, RN-09, Office of Research and Development, April 2000.

e-Assessment system architecture

Features of an e-assessment system