Blended-learning system for automated paper questionnaires

Blended-learning tool to extend the automatic correction feature of Moodle questionnaires to the non-computerized classroom.

This tool allows teachers to create paper copies of the questionnaires designed and created in Moodle so that students can fill them in class without the need for computers. Once the questionnaires have been completed by the students, the system can correct them automatically, thanks to a system of optical recognition of marks based on conventional low cost scanners, and the results are then transferred to Moodle. The goal is to provide teachers with flexible automated tools that enable frequent assessments with teaching or assessment purposes, as recommended by the European Higher Education Area (EHEA).

I.       Introduction

The European Higher Education Area (EHEA) has brought along new ways of teaching and learning and it has lead to changes in the teaching-learning process. Within this new framework, assessment activities take a new importance as they must run as a continuous process and be teaching and training oriented[1] [2]. Assessment should be twofold. On the one hand, it should allow teachers to obtain information on the development of students as well as the possible causes that could hinder such development in order to improve and adapt their strategies and learning objectives. On the other hand, it should allow students to obtain an indication of their progress so as to increase their motivation and satisfaction with the process [3]. In this context, it is necessary to rethink the design and development of methods and assessment tools because it is important to check the learning results not only at the end of the educational process but for the duration of the same[4].

Assessment, however summative, must be part of the learning process and thus should be included within the classroom routines, and it should be directed primarily towards improving learning rather than simply providing a means for learning accreditation [1]. This is known as learning-oriented assessment [2] [5].

Therefore, a greater effort in assessment procedures  must be made, so as to allow further testing and subsequent feedback to students so that they can benefit from it[5]. ICTs facilitate this task by automatizing the tasks of objective evaluation of tests and by allowing the management of databases of questions that can be included in the questionnaires. However, a significant proportion of teaching takes place in "traditional" classrooms with no computerized equipment.

This project responds to the need for more frequent monitoring of the learning process in a way that would not increase the workload of teachers and would provide immediate feedback to students. The manual evaluation of tests by teachers is hardly a productive process, and it slows the process of feedback down. There are specialized tools that avoid this however they require the availability of computer rooms.

The current virtual campus system, based on learning management systems or LMS (Learning Management System) sucha as Moodle, allows for the creation of efficient assessment questionnaires, so the aim of this project is to create and validate a blended-learning tool that allows complete questionnaire management cycles. This system particularly aims at generating sets of questionnaires (randomized) in PDF (ie, printable) from Moodle questionnaires, then process them and correct them automatically (once answered by the students) through a system of optical recognition of marks, and finally to upload the resultas to Moodle so that they can be reviewed and evaluated as if they had been made on-line (see Figure 1). Therefore, this system will provide teachers with continuous assessment questionnaires without depending on the availability of a computer room, and will increase the efficiency of the correction process. Furthermore, as it takes place in Moodle, the whole process will be integrated with the rest of the online activities of the given module.

 

 

Fig, 1. General structure of the blended-learning system.

 

II. Evaluation System Blended-Learning

Many of the systems of OMR (Optical Mark Recognition) currently used specialized equipment used to collect the questionnaire responses and often require specialized recognizers. This forces the process to go through a bottleneck (OMR) which makes the practical application of these techniques in everyday teaching.

The evaluation system proposed blended-learning approach for the electronic questionnaire the student and teacher, so that its design is taken into account the following functional objectives:

· The questionnaires must be submitted in a readable format and naturally, without using coding manuals.

· Do not use specific templates separate questionnaires to simplify the use of questionnaires by the students.

· No action is required in a specialized OMR to allow teachers to use their own equipment to drive the process.

The system consists of a series of interrelated modules that can generate paper questionnaires from the database of questions and activity evaluation questionnaire generation Moodle Quiz. The questionnaires were transcribed including spaces for marks for multiple choice answers so that students can fill in the classroom without traumatic changes involve the planning of ongoing activities. Once the questionnaires have been completed by the students, the system allows them to be corrected automatically, thanks to an image processing module that performs the OCR or OMR marks from images obtained with conventional low-cost scanners . The results are transferred to Moodle so that there is no practical difference as to whether the student had answered on-line form.

Moreover, since the whole production process of generation and correction Moodle questionnaires, the results of the process will be available for review through Moodle. Both teachers and students can see the results of evaluation of the questionnaires at any time and from anywhere that has Internet access.

For development, the system is divided into two major subsystems:

· Module blended to integrate the evaluation process in blended learning Moodle. This module allows the generation of PDF questionnaires from collections of questions stored in the database and the integration of the results on the platform.

· Module integrated into Moodle for the processing of paper questionnaires through brand recognition and barcodes. This module is responsible for the recognition of marks on the tests and collecting results in a few files that can be incorporated into Moodle for processing.

A. Blended-learning module

The module allows the generation of blended questionnaires in PDF format from collections of questions stored in the database of the Moodle platform. Questionnaires are currently used consist of multiple type questions, true-false questions in text format. You can get shuffled randomly [17].

The module allows the teacher to select a questionnaire from a Moodle course and indicate how many prints you want to generate. The module generates the following files:

· OMR configuration file containing the coordinates of all elements. This file is necessary for the OMR detector can locate the answers marked by the students.

· PDF file with many pages as instances of questionnaires indicated by the teacher. Each sheet of this file has an identifier that allows us to relate the results of the evaluation phase with the questionnaires. Plus there's a space for students to paste a sticker with bar code, since students are offered a link to generate labels with bar codes.

Figure 2 shows the format of the questionnaires. Has a header that appears in the logo of the university, course name, the bar code with the ID number plus the number of questionnaire sheet. Figure 3 shows part of a real survey. More details on the operation and implementation of this module can be found in [17].

This module also allows, once scanned the results of the questionnaires, pass values ​​to Moodle, so that results can be accessed by students and teacher through the platform. Thus, from Moodle, teachers can go to a specific working directory current file with the images obtained from scanning of all paper questionnaires completed by students and request the automatic processing system. To store results automatically in Moodle, scan the resulting images must be processed by the brand recognition module, described below.

Figure 2. Format of the questionnaires. Source: [17].

Figure 3. Sample questionnaire generated.

B. OMR Module

OMR module for Moodle is a Java program can read files from multiple page PDF and TIF, as well as images of various resolutions. Process configuration files to identify instances of forms and detect the positions of the marks, making some adjustments to resolve defects and displacements and rotations digitization of the image.

The module processes batch files identifying the image, adjusting the resolution and color characteristics and applying a series of algorithms to perform the following actions on the elements of a document:

Detection of the alignment marks by elresalte edge and Hough transform to eliminate rotation or displacement of the scan.

· Identification of the degree of filling of the marks of response options, by cross correlation with a brand model.

· Obtaining numeric codes printed copy ID and the user has responded by recognizing linear barcodes and two-dimensional or 1D QR (Quick Response) or 2D.

With the results of processing generates a results file. From preliminary tests, it may indicate that the degree of system reliability is high, while ensuring that the displacement produced by the scanner to pick up the leaves is small. Still, if there is a failure of recognition, it can be corrected manually by the teacher. In addition the student will always show the results along with an image of the original questionnaire, so you can check their results have not altered in any way during the process of scanning and recognition.

More details on the operation and implementation of this module can be found in [18].

 

This project is being developed with the cooperation and funding of Vice President for Teaching at the University of Valladolid.

The authors wish to express special thanks to Paul Gallant, Natalie Haro, Jesus Rodilana and David Fernandez, who implemented as part of his Master Thesis some parts of the modules designed by the authors and described in this article.

References

[1]   L.A. Shepard, The Role of Classroom Assessment in Teaching and Learning, Los Angeles, CA: CRESST – CSE Technical Report 517, 2000. Disponible en: http://datause.cse.ucla.edu/DOCS/las_rol_2000.pdf (Última consulta: 25/04/2011).

[2] G. Rodríguez Gómez and M.S. Ibarra Sáiz (Eds.), e-Evaluación orientada al e-Aprendizaje estratégico en la universidad, Madrid: Narcea, 2011 (en prensa).

[3]   J.C. Fortes Garrido, R. Arribas de Paz, and F. Gomez Bravo, “Adaptación y Planificación Eficaz de la Evaluación en el EEES,” Comunicaciones del XVI Congreso Universitario de Innovación Educativa en las Enseñanzas Técnicas, no. 16, 2008.

[4]   A.M. Delgado, and R. Oliver, “La evaluación continua en un nuevo escenario docente”, Revista de Universidad y Sociedad del Conocimiento, vol. 3, no.1, 2006. Disponible en: http://www.uoc.edu/rusc/3/1/dt/esp/delgado_oliver.pdf(Última consulta: 25/04/2011).

[5]   E. Dorrego, “Educación a Distancia y Evaluación del Aprendizaje,” Revista de Educación a Distancia, no. IV, 2006. Disponible en: http://www.um.es/ead/red/M6 (Última consulta: 25/04/2011).

[6]   A. Bartolomé, “Blended Learning. Conceptos básicos,”Revista de Medios y Educación, no. 23, pp. 7-20, 2004.

[7]   C. Morgan, and M. O’Reilly, Assessing Open and Distance Learners. London: Kogan Page, 2002.

[8]   JISC, Effective Practice with e-Assessment. An overview of technologies, policies and practice in further and higher education, 2007. Disponible en: http://www.jisc.ac.uk/media/documents/themes/elearning/effpraceassess.pdf (Última consulta: 25/04/2011).

[9]   Y. Miao, C. Tattersall, J. Schoonenboom, S. Stevanov, and A. Aleksieva-Petrova, “Using open technical e-learning standards and service orientation to support new forms of e-assessment”, In D. Griffiths, R. Koper and O. Liber (Eds), Service Oriented Approaches and Lifelong Competence Development Infrastructures. Proceedings of the 2nd TENCompetence Open Workshop Manchester, UK, pp. 183-191, Bolton: The Institute for Educational Cybernetics, University of Bolton, 2007.

[10]V. González-Barbone, and M. Llamas-Nistal, “eAssessment: Trends in Content Reuse and Standardization,” Proceedings of the 37th ASEE/IEEE Frontiers in Education Conference, Milwaukee, USA, 2007.

[11]G.H. Smith, L.N. Wood, M. Coupland, B. Stephenson, K. Crawford, and G. Ball, “Constructing mathematical examinations to assess a range of knowledge and skills,” International Journal for Mathematical Education in Science and Technology, vol. 27, no. 1, pp. 65-77, 1996.

[12]A. García-Beltrán, R. Martínez, J.A. Jaén, and S. Tapia, “La autoevaluación como actividad docente en entornos virtuales de aprendizaje/enseñanza,” Revista de Educación a Distancia, no. IV, 2006. Disponible en http://www.um.es/ead/red/M6 (Última consulta: 26/04/2011).

[13]M. Blanco, and M. Ginovart, “Análisis de la eficiencia de cuestionarios Moodle como herramienta de evaluación de asignaturas de matemáticas correspondientes a primeros cursos de titulaciones de ingeniería,” Congreso Internacional Docencia Universitaria e Innovación, 2010.

[14]E. Barberà, “Aportaciones de la tecnología a la e-Evaluación”, Revista de Educación a Distancia, no. VI, 2006. Disponible en: http://www.um.es/ead/red/M6 (Última consulta: 25/04/2011).

[15]M. Alacid, M.R. Castellar, and J.M. Obón, “Exámenes de autoevaluación en el aula virtual: Generación y resultados,” Comunicaciones de las I Jornadas sobre la enseñanza de las ciencias y las ingenierías, Murcia, 2008. Disponible en: http://www.murciencia.com/UPLOAD/COMUNICACIONES/examen_autoevaluacion_aula_virtual.pdf(Última consulta: 25/04/2011).

[16]M. Blanco, and M. Ginovart, “Creating Moodle quizzes for the subjects of mathematics and statistics corresponding to the first years in engineering studies,” Proceedings of Edulearn09 - International Conference on Education and New Learning Technologies, 2009. Disponible en http://upcommons.upc.edu/e-prints/bitstream/2117/3064/1/blanco_ginovart_... (Última consulta: 25/04/2011).

[17]N. Haro, Integración de un Sistema de Reconocimiento Óptico de Marcas para la Aplicación de Blended e-Learning, Proyecto Fin de Carrera, Valladolid: ETS Ingenieros de Telecomunicación, 2010.

[18]J. Rodilana, Desarrollo de un sistema de reconocimiento óptico de marcas para integración en un sistema de e-learning, Proyecto Fin de Carrera, Valladolid: ETS Ingenieros de Telecomunicación, 2010.