Short Biography 

Mélanie Revilla Melanie Revilla is a postdoctoral researcher at the Research and Expertise Centre for Survey Methodology (RECSM) and an adjunct professor at Universitat Pompeu Fabra (UPF, Barcelona, Spain). She received her PhD from UPF in 2012, in the areas of statistics and survey methodology, under the supervision of professors Willem Saris (UPF) and Peter Lynn (Essex University). She graduated in statistics and economics at the 'Ecole nationale de la statistique et de l'administration économique' (ENSAEParitech, Paris, France) and holds a Master of Science in Economics from the Barcelona Graduate School of Economics (Spain). She has published around 30 papers or book chapters related to the following topics: measurement errors, data quality, questionnaire design, web surveys, mobile devices, experiments, passive measurement.

https://www.upf.edu/web/survey/entry/-/-/67233/adscripcion/melanie-audreyrevilla

Description of the course:

We hope to show with this course how important is the design of a questionnaire for the results, and how difficult it can be. The main goal is to highlight the very high number of decisions to be made, and the impact that each of them can have on the answers, but also how all these decisions interact with each other, and thus the necessity to use a tool that takes into account all these interactions to help us designing good questionnaires.

The design of a questionnaire does not start with questions. It starts with the definition of the concepts of interest. From the concepts, assertions can be made, and then requests for an answer. This is the so called 3-step procedure proposed by Saris and Gallhofer (2007). This course will begin by a detailed presentation of this procedure. Then, we will introduce many other decisions that need to be taken when designing a questionnaire: decisions about the scale format, the use of a “don't know” answer, the grouping and ordering of the items, the mode of data collection, etc. All these decisions can affect the answers of the respondents and therefore the results of the study. However, how can we know which formulation is the most accurate? Because all these decisions can interact with each other, it is difficult to decide what the best formulation for a question is, i.e. the formulation which will give highest quality data. One tool available to help us in this decision is the Survey Quality Predictor software. We will show how the program works and how it can be used during the questionnaire design stage. We will finish by some practical applications.

Course prerequisites: Familiarity with the use of survey data is preferable.

Software used: We will introduce the software SQP. 

Main Reading:

Saris, W.E., and Gallhofer, I.N. (2014) Design, Evaluation and Analysis of Questionnaires for Survey Research. New York: Wiley. Second Edition.

Revilla, M., Zavala Rojas, D., and W.E. Saris (2016). “Creating a good question: How to use cumulative experience”. In Christof Wolf, Dominique Joye, Tom W. Smith and Yang‐Chih Fu (editors), The SAGE‐Handbook of Survey Methodology. SAGE. Chapter 17, pp.236-254.

Suggestions for further reading:

Belson, W. (1981), The Design and Understanding of Survey Questions, London: Gower.

Billiet, J. (2016). “What does measurement mean in a survey context?” In Christof Wolf, Dominique Joye, Tom W. Smith and Yang‐Chih Fu (editors), The SAGE‐Handbook of Survey Methodology. SAGE.

Converse, J. M., and S. Presser (1986), Survey Questions: Handcrafting the Standardized Questionnaire, Beverly Hills: Sage.

Revilla, M. (2015). “Effect of using different labels for the scales in a web survey”. International Journal of Market Research, 2015, 57(2):225-238. First published online on June 1, 2014. DOI: 10.2501/IJMR-2014-028

Revilla, M. (2015). “Comparison of the quality estimates in a mixed-mode and a unimode design: an experiment from the European Social Survey”, Quality and Quantity. 2015, 49(3): 1219-1238. Published online first June 2014. DOI: 10.1007/s11135-014-0044-5

Revilla, M., Toninelli, D., Ochoa, C., and G. Loewe (2016). “Do online access panels really need to allow and adapt surveys to mobile devices?” Internet Research 26(5): 1209 - 1227. Available at: http://www.emeraldinsight.com/doi/abs/10.1108/IntR-02- 2015-0032

Saris W. E., D. Oberski, M. Revilla, D. Zavalla, L. Lilleoja, I. Gallhofer, and T. Grüner (2011), The Development of the Program SQP 2.0 for the Prediction of the Quality of Survey Questions, RECSM Working Paper 24: www.upf.edu/survey/_pdf/RECSM_wp024.pdf

Scherpenzeel, A. (1995), A Question of Quality: Evaluating Survey Questions by Multitrait-Multimethod Studies, Amsterdam: Nimmo

Schuman, H., and S. Presser (1981), Questions and Answers in Attitude Survey: Experiments on Question Form, Wording and Context, New York: Academic Press.

Sudman, S., and N. M. Bradburn (1983), Asking Questions: A Practical Guide to Questionnaire Design, San Francisco: Jossey Bass.

Tourangeau, R., L. J. Rips, and K. Rasinski (2000), The Psychology of Survey Response, Cambridge, MA: Cambridge University Press.