Conception, design and evaluation of the Interactive tutorial: How to prepare your academic paper. A case study

Authors: Pere Freixa i Font (Universitat Pompeu Fabra), Carles Sora  (Universitat Pompeu Fabra), Joan Soler-Adillon (Universitat Pompeu Fabra) i  J. Ignasi Ribas (Universitat Pompeu Fabra)

Citation: Freixa i Font, Pere et al. (2013). "Conception, design and evaluation of the Interactive tutorial: How to prepare your academic paper. A case study". Hipertext.net, 11, http://www.upf.edu/hipertextnet/en/numero-11/conception_interactive_tutorial.html

Pere FreixaCarles SoraJoan Soler-AdilllonJ. Ignasi Ribas

Abstract: How to prepare your academic paper is the title of the tool developed by the Laboratorio de Comunicación Interactiva (Interactive Communication Laboratory) of the DigiDoc Group and the Library/CRAI at Universidad Pompeu Fabra to facilitate the preparation of scientific student papers according to standardized quality guidelines. The tutorial, which is available at http://stpr.upf.edu/tutorial/, responds to a double need: allowing self-learning by students, and serving as a pedagogic resource for the Curso de Introducción a la Universidad (Introductory course to the University, CIU) the library staff is giving. This article presents the guidelines used to create the tool, its pedagogic and discursive principles as well as the evaluation of the results obtained when first used during the 2012-13 academic year.

Keywords: Self-learning, informational competences, Ci2, conceptual maps, interactive design, university tutorials 


Table of contents:

1. Introduction
2. Goals
3. Evaluation methodology
4. Results and conclusions
References

1. Introduction

The confluence of the needs for literacy and learning of computer and information competences (ALFIN), due to the changes the implementation of the European Higher Education Area (EHEA) has brought about in universities, promotes the creation of programmes and actions which formally started in Spain with the Toledo Declaration on information literacy (2006). Within the university framework, the Red de Bibliotecas Universitarias españolas (Spanish university libraries network, REBIUN) registers the mandate from the Toledo Declaration, and so included the goal of making ALFIN a compulsory and transversal competence in every Spanish degree in its 2nd Strategic Plan for 2007-10. This process must help orientate the transition of REBIUN libraries towards a model encompassing the user’s education on information literacy (Somoza; Abadal, 2007, Somoza, 2009).

  1. During the 2007-08 academic year, Universitat Pompeu Fabra (UPF) conducted a first pilot test by creating a Curso de Introducción a la Universidad (Introductory Course to the University, CIU) for new students (Hernández; Solà; Baños; Amat, 2012). Staff from the Library/CRAI participated in this study, and they used the pedagogic resources available at that time: subject guides (Jaques; Losada, 2005), resources directories, links and access to online catalogues and a first version of the tutorial Cómo realizar un trabajo académico (How to prepare your academic paper), http://www.upf.edu/bibtic/es/recursos/treaca/, from 2005.
  2. The present article explains the process of ideation, design, execution and first evaluation of the renewed educational tool Cómo realizar un trabajo académico, first implemented at UPF during the 2012-13 academic year. This tool stems from the collaboration between the Laboratorio de Comunicación Interactiva (Interactive Communication Laboratory) from the research group DigiDoc and the Library/CRAI from UPF.
  3.  Three aspects have determined the need to renovate the older tutorial. In the first place, the need to create a versatile tool that allows a double use: autonomous self-learning through Internet and pedagogic support for the formative sessions offered by the Library/CRAI staff in the classrooms. In the second place, to devise functionalities answering to specific demands from students with various profiles, formative levels and language levels. And in the third place, to think of innovative discursive solutions for the use of multimedia resources and the interaction possibilities.
  4. Although it has been online for only some months now, several Spanish universities such as UIB, UPC and UAB have recommended the tutorial and linked it to their websites and virtual versions. Universitat Jaume I has included it as teaching material in its formative sessions. 

This article presents the guidelines of the tutorial as well as the evaluation results obtained after we first presented it in the classroom with first and third year UPF students of Audiovisual Communication.

2. Goals

From the 2008-09 academic year on, Universitat Pompeu Fabra has been offering all new students education on Information Competences (CI2) with an academic value of 1 ECTS. The course is part of a generic CIU course, which is offered in all the degrees of the university (Cantos, Trench, 2012). For subject number 5 of the CI2, “How to find the information resources students need to prepare an academic work”, the tutorial How to prepare your academic paper has been completely reformulated.

2.1. Pedagogic support

Following some of the most widely accepted recommendations (Somoza, 2009), the content has been redistributed into five sections: subject of the work, contextualize the subject, find resources, evaluate the resources found and write the work. A conceptual map has been developed for each section, where the items and actions linking them are presented. Each one of these items gives access to specific content, such as textual explanations, hypertextual links or examples.

Mapa conceptual del apartado Contextualizar el tema.

Figure 1. Conceptual map of the section Contextualize the subject

 

 

 Ventana con el contenido específico del ítem Diccionarios y enciclopedias.

Figure 2. Window with the specific content of the item Dictionaries and encyclopedias

2.2. Focus on the student

Focusing on the student means contemplating and responding to several diversity aspects such as formative levels, specialties, idiomatic knowledge, different learning rates and punctual resolution of questions or doubts from each particular student. For that matter, a second interface of the tool has been developed allowing direct consultations, the content offered has been expanded and specialized and the tutorial is available in Spanish, Catalan and English versions.

In the direct consultation interface, the student chooses the search parameters according to the subject matter, section and subsection through which the student wishes to filter the question. The academic areas considered are those wherein UPF offers teaching (Experimental and Health Sciences, Political and Social Sciences, Communication, Law, Economics and Business, Humanities, TIC and Translation and Language Sciences).


Interface of direct consultation

 

Figure 3. Interface of direct consultation



2.3. Self-learning

To facilitate self-learning, two strategies have been considered: data narrativity (Segel; Heer, 2010) and the inclusion of characters acting as pedagogic agents (Moreno, 2007).

Unlike linear narratives such as textual and film narratives, the presentation of data in Interactive environments, the so-called data narrativity, allows the deployment of alternative strategies such as simultaneity or parallel narrative developments. It involves establishing communicative strategies that connect information visualization with the dialogue possibilities of interaction (Freixa; Sora, 2010). Thus, the user is presented with different functionalities and content and is allowed to choose which one is useful or needed to follow, or which one(s) she/he decides to discard.

2.4. Interactive audiovisual discourse

To illustrate the different phases and sections, visual elements such as animated graphics, a voice-over with the most important ideas and a series of animated characters have been introduced; they work both for dramatization purposes and as pedagogic agents. The characters and the voice-over narration both encourage and contribute to the understanding of the pedagogic experience (Moreno, 2007).

Presentación animada del mapa conceptual

Figure 4. Animated presentation of the conceptual map. Animated text, graphics and voice-over lead to the audiovisual discourse.

Thus, the interface of the tool combines two discourses which are traditionally presented separately: the totality and schematization of the conceptual map and the didactic linearity of the audiovisual image. The user who promptly chooses to access the map can turn the narrated dramatization off and consult the different sections of the map. Nevertheless, for the user who visits the tutorial for the first time or the user who prefers a traditional audiovisual reception, the image and sound resources provide with a time and space continuity which favors understanding the interactive tutorial. Thus, an audiovisual and interactive discourse is promoted, one which is able to combine the properties of the different media converging in the digital audiovisual tutorial with the acknowledgement of the role of the user in developing the interactive dialogue (Ribas, 2009).

3. Evaluation methodology

The evaluation aims at considering the efficacy of the tool based on two parameters: the acceptance by the students and its pedagogic usefulness in the classroom. To accomplish those purposes, two questionnaires have been presented: a quantitative one, with numerical answers for the students, and a qualitative one, with open textual answers for the Library/CRAI staff who has given the CIU/CIU2 courses. The first questionnaire has been presented to 55 third-year students who had no previous knowledge of the tutorial and 46 first-year students who had been previously introduced to the tutorial during the CIU. The qualitative survey has been presented to the 6 people working at the Library/CRAI who taught the CIU in UPF’s Campus de la Comunicación.

This first evaluation must work as a base to consider if it is appropriate to conduct a wider evaluation including the rest of faculties where the tutorial has been used and the statistical study of the results obtained.

1. Qualitative survey. Acceptance among the students

1

Evaluate how much the tutorial has helped you understand the process of preparing an academic paper.

0

1

2

3

4

5

2

Evaluate how much the dramatized setting of conceptual maps favors understanding. Evaluate how much has each of these elements helped you:

2.a

Animated characters

0

1

2

3

4

5

2.b

Voice-over

0

1

2

3

4

5

2.c

Tag animation

0

1

2

3

4

5

3

How often are you going to use each of these tools when you use the tutorial again?

3.a

Narrated conceptual map

0

1

2

3

4

5

3.b

Deployed conceptual map

0

1

2

3

4

5

3.c

Direct consultation

0

1

2

3

4

5

4

Evaluate how much each one of these tools has adapted to your learning rate

4.a

Narrated conceptual map

0

1

2

3

4

5

4.b

Deployed conceptual map

0

1

2

3

4

5

4.c

Direct consultation

0

1

2

3

4

5

5

Evaluate how much each of these tools adapts to your knowledge level

5.a

Narrated conceptual map

0

1

2

3

4

5

5.b

Deployed conceptual map

0

1

2

3

4

5

5.c

Direct consultation

0

1

2

3

4

5

Table 1. Survey model for the analysis of the tutorial’s acceptance among 1st and 3rd year students

2. Qualitative survey. Acceptance as a support tool in the classroom

1

Have you used the tutorial in the CIU / CI2 subjects

Yes

No

If the answer is YES, go to question 2. If it’s NO, go to question 9

2

Which competences did you want to explain?

3

Which parts of the tutorial have you shown or used?

4

Does the tool seem pedagogically effective or useful to you?

5

Have you shown the various paths and functionalities to access information?

6

Have students been able to use the tool?

7

Would you use the tool in the next academic year?

8

Free comment

9

Have you used the Prezi presentation or other materials? Why?

10

Why have you decided not to use the tutorial?

11

Do you think it might be useful in the future?

12

Free comment

Table 2. Survey model for the analysis of the tutorial’s acceptance among the staff at the Library/CRAI.

While waiting for the Library/CRAI to undertake specific CI2 actions with graduate students (expected for 2013), so that the acceptance of the tutorial among all its potential users can be appreciated, it has been considered that the data offered by the surveys allow for some assessments and future expectations.

This evaluation also assumed the impossibility of considering the autonomous Internet users of this tool. This is one of the basic user profiles the tutorial has been designed for, but because it is difficult to quantify possible biases, the assessment of the application through an online survey has been discarded. The possibility of conducting an indirect evaluation through web navigation parameters and indicators such as reading time, links clicked, external windows activated and number of sessions started is being considered.

4. Results and conclusions

Quantitative survey. First and third year students

 

Questions

 

1

2.a

2.b

2.c

3.a

3.b

3.c

4.a

4.b

4.c

5.a

5.b

5.c

1st year

3.98

2.17

3.63

3.59

2.45

4.24

3.86

2.99

4.01

3.82

3.06

3.94

3.91

3rd  year

3.56

1.88

3.40

3.35

1.52

4.05

4.01

2.29

3.95

3.73

2.56

3.57

3.63

Table 3. Quantitative results. The table expresses the medians obtained
First-year students consider very positively (3.98) the usefulness of the tutorial to understand the process of preparing a paper. This assessment is also satisfactory for third-year students (3.56, both figures belong to question 1).

The previous positive assessment matches the unanimous appreciation by the Library/CRAI staff when answering questions 4 and 7. They agree that the tool has a pedagogic usefulness, and they are also willing to use the tutorial in future formative actions.

The best-valued functionality, both by students and teachers, is the deployed conceptual map (question 3.b: 4.24, 1st-year students; 4.05; 3rd-year students). It is the only function used in the classroom by all teachers, and it is the one the students consider better fitting in with their learning rates (question 4.b: 4.01,1st-year students; 3.95, 3rd-year students).

The option of direct consultation is well-accepted by part of the students, (question 3c: 3.86, 1st-year students; 4.01, 3rd-year students), despite only one teacher showed it in the classroom.

The animated dramatizations, which were not shown in the classroom by the teachers, are considered irrelevant at a formative level (question 2a: 2.17, 1st-year students; 1.88, 3rd-year students). They are perceived as mere complementary visual information without pedagogic value. A wider study lacking, it seems reasonable to deduct that narrations might turn out to be a useful tool to tackle the adaptation of content according to age criteria.

No teacher has shown the tutorial in its entirety. The lack of time and familiarity with other tools are the most common reasons for that. Nor guided actions have been conducted in the classroom, such as exercises or practical approaches to check if the tutorial is understood by part of the students.

The possibility of improving the formation of the Library/CRAI staff is suggested in all the tutorial’s functionalities, so that their presentations provide with a wider and thorough explanation. Several teachers also stress the need to expand the terminal content of the tool, both for covering all specialties and to keep the application updated.

References

Note: International Congress on University Teaching and Research, CIDUI, 2012, a communication was presented in poster format that included a previous version of this work while still not been implemented or evaluated (Freixa; Ribas; Soler-Adillon; Sora; Badell; Magre; Martínez; Vicioso, 2012).

Alonso-Arévalo, Julio; Gallego-Lorenzo, Josefa (2009). «Desarrollo de tutoriales en bibliotecas universitarias en el contexto del aprendizaje y la investigación: adaptación al Espacio Europeo de Enseñanza Superior». IV Encontro Ibérico EDIBCIC, 2009, Coimbra, pp. 301-317, <http://hdl.handle.net/10760/13660> [Consultado: 15/12/2012].

Cantos, Carme; Trench, Joan (2012). «La experiencia de la Universitat Pompeu Fabra con las competencias informacionales e informáticas (CI2)». ALFARED. Monográfico Experiencias ALFIN en bibliotecas universitarias, pp. 1-3,
<http://www.alfared.org/sites/www.alfared.org/files/u49/25-pompeu%20fabra.pdf> [Consultado: 15/12/2012].

Declaración de Toledo sobre la alfabetización informacional (ALFIN). Toledo, 2006. http://www.webcitation.org/5NrAiGhSS

Freixa, Pere; Ribas, J.Ignasi; Soler-Adillon, Joan; Sora, Carles; Badell, Joan Isidre; Magre, Anna; Martínez, Mercè; Vicioso, Alexis (2012). «Com elaborar un treball acadèmic: Dissenyar un recurs en línia de suport a l’aprenentatge per al CRAI de la UPF». Congreso Internacional de Docencia Universitaria e Innovación, CIDUI, Barcelona, <http://www.cidui.org/revista-cidui12/index.php/cidui12/article/view/247/237> [Consultado: 15/12/2012].

Freixa, Pere; Sora, Carles (2010). «Universo UPF: un proyecto interactivo y participativo de visualización de datos acerca del entorno universitario». Hipertext.net, n. 8, <http://www.upf.edu/hipertextnet/numero-8/universo_upf.html> [Consultado: 15/12/2012].

Grupo de Trabajo de Alfabetización Informacional (2009). Hacia la Alfabetización Informacional en las bibliotecas públicas españolas. Informe de trabajo del Grupo de Trabajo de Alfabetización Informacional. Madrid: Consejo de Cooperación Bibliotecaria, <http://www.mcu.es/bibliotecas/MC/ConsejoCB/gt_alfin/ALFIN_en_BP_2009.pdf> [Consultado: 15/12/2012].

Hernández, Carme; Solà, Pau; Baños, Josep Eladi; Amat, Oriol (2012). «Impacto del Curso de Introducción a la Universidad en la UPF». Congreso Internacional de Docencia Universitaria e Innovación, CIDUI, <http://www.cidui.org/revista-cidui12/index.php/cidui12/article/view/139/126> [Consultado: 15/12/2012].

Jaques, Cecília; Losada, Marina (2005). «Elaboración de guías temáticas en el web en la biblioteca de la Universitat Pompeu Fabra». El profesional de la Información, v. 14, n. 1, pp. 67-69.

Moreno, Roxana (2007). «Animated Pedagogical agents: How Do They Help Students Construct Knowledge from Interactive Multimedia Games?». Richard Lowe y Wolfgang Schnotz (eds). Learning with Animation. Cambridge: Cambridge University Press, pp. 183-207.

Moreno, Verónica; Frangi, Alejandro; Piella, Gemma; Infante, Jorge; Bellalta, Boris (2009). «El curso de introducción a la universidad como estrategia organizativa para acercar al estudiantado de nueva incorporación a la Escuela Superior Politécnica»,
X Congrés Interuniversitari d’Organització de les Institucions Educatives, <http://hdl.handle.net/10230/16113> [Consultado: 15/12/2012].

REBIUN (2007). II Plan Estratégico 2007-2010, <http://www.rebiun.org/doc/plan.pdf> [Consultado: 15/12/2012].

Ribas, J.Ignasi (2009). «Integració de mitjans en el discurs interactiu: el cas de la difusió cultural». Quaderns del CAC. Convergència tecnològica i audiovisual, n. 31-32, pp.43- 53,
<http://www.cac.cat/pfw_files/cma/recerca/quaderns_cac/Q31-32_Ribas.pdf> [Consultado: 15/12/2012].

Segel, Edward; Heer, Jeffrey (2010). «Narrative Visualization: Telling stories with Data». IEEE Transactions on Visualization and Computer Graphics, v. 16, n. 6, pp.1139-1148,
<http://dx.doi.org/10.1109/TVCG.2010.179> [Consultado: 15/12/2012].

Somoza, Marta (2009). Análisis de los tutoriales web creados por bibliotecas universitarias. Barcelona: Universidad de Barcelona, Tesis Doctoral, 2009,
<http://hdl.handle.net/10803/769> [Consultado: 15/12/2012].

Somoza-Fernández, Marta; Abadal, Ernest (2007). «La formación de usuarios en las bibliotecas universitarias españolas». El profesional de la Información, v. 16, n. 4, pp. 287-293.

Last updated 21-05-2013
© Universitat Pompeu Fabra, Barcelona