January 19, 2021, at 11h (CET) via Zoom.

Open question formats: Comparing the suitability of requests for text and voice answers in smartphone surveys [Abstract]

Presenter: Jan Karem Höhne (University of Mannheim, RECSM-Universitat Pompeu Fabra)
Annelies Blom (University of Mannheim), Konstantin Gavras (University of Mannheim), Melanie Revilla (RECSM-Universitat Pompeu Fabra), Leonie Rettig (University of Mannheim)

Abstract:

While surveys provide important standardized data about the population with large samples, they are limited regarding the depth of the data provided. Although surveys can offer open answer formats, the completeness of and detail provided in these formats is often limited, particularly in self-administered web surveys, for several reasons: On the one hand, respondents find it difficult to express their attitudes in open answer formats by keying in the answers. Respondents also keep their answers short or skip such questions altogether. On the other hand, survey designers seldom encourage respondents to elaborate on their open answers, because the ensuing coding and analysis have long been conducted manually. This makes the process time-consuming and expensive, reducing the attractiveness of such formats. However, technological developments for surveys on mobile devices, particularly smartphones, offer the collection of voice instead of text answers, which may facilitate answering questions with open answer formats and provide richer data. Additionally, new developments in automated speech-to-text transcription and text coding and analysis allow the proper handling of open answers from large-scale surveys. Given these new research opportunities, we address the following research question: How do requests for voice answers, compared to requests for text answers, affect response behavior and survey evaluations in smartphone surveys? We conducted an experiment in a smartphone survey (N = 2,400) using the opt-in Omninet Panel (Forsa) in Germany in December 2019 and January 2020. From their panel, Forsa drew a quota sample based on age, education, gender, and region (East and West Germany) to match the German population on these demographic characteristics. To collect respondents’ voice answers, we developed the JavaScriptand PHP-based “SurveyVoice (SVoice)” tool that records voice answers via the microphone of smartphones. We randomly assign respondents to answer format conditions (i.e., text or voice) and ask them six questions dealing with the perception of the most important problem in Germany as well as attitudes towards the current German Chancellor and several German political parties. In this study, we compare requests for text and voice answers in smartphone surveys with respect to several aspects: First, we investigate item nonresponse (i.e., item missing data) as an indicator of primarily low data quality. Second, we investigate response times (i.e., the time elapsing between question presentation on the screen and the time until the survey page was submitted) as an indicator of respondent burden. Finally, we investigate respondents’ survey evaluations (i.e., level of interest and level of difficulty stated by respondents) as an indicator of survey satisfaction. This experiment aims to test the feasibility of collecting voice answers for open-ended questions as an alternative data source in contemporary smartphone surveys. In addition, it explores whether and to what extent voice answers collected through the built-in microphones, compared to open answers entered via the keyboard of smartphones, represent a sound methodological substitute.

February 23, 2021, at 11h via Zoom.

​Are you paying for or with quality? Survey participation due to monetary incentives and measurement quality – Evidence from the GESIS Panel.

Presenter: Hannah Schwarz (RECSM-Universitat Pompeu Fabra)

Abstract:
In times of decreasing response rates, monetary incentives are increasingly used to motivate individuals to participate in surveys. Receiving an incentive can affect respondents’ motivation to take a survey and, consequently, their survey taking behaviour. On the one hand, the resulting extrinsic motivation might undermine intrinsic motivation thus leading respondents to invest less effort into answering a survey. On the other hand, monetary incentives could make respondents more eager to invest effort into answering a survey, as they feel they are compensated for doing so. This study aims to assess whether there are differences in measurement quality between respondents who are motivated to take surveys due to the received incentive and respondents for who this is not a reason for participation. We implemented two Multitrait-Multimethod (MTMM) experiments in the probability-based GESIS Panel in Germany (2019) to be able to estimate the measurement quality of 18 questions asked to panelists. By coding panelists’ open answers to a question about their reasons for participation, we distinguish panelists who state that they are motivated by the incentive from those who do not. We analyse the MTMM experiments for these two groups separately and compare the resulting measurement quality estimates. 

March 16, 2021, at 13h (CET) via Zoom.

Affective polarization: its measurement in multi-party contexts and its relationship with ideology

Presenter: Josep Maria Comellas (RECSM - Universitat Pompeu Fabra)

Mariano Torcal (RECSM Director - Universitat Pompeu Fabra)

Abstract:

Affective polarization broadly refers to the extent that individuals feel sympathy towards in-groups and antagonism towards out-groups. While this topic has been extensively studied in the United States, affective polarization has increasingly received comparative attention in an attempt to study this phenomenon in multi-party settings. In the first part of the presentation, we revise some of the main indices proposed in the literature to measure affective polarization, and we explain the ones that we have implemented using different datasets (CNEP, CSES, E-DEM). Then, in the second part, we present a paper that is focused on the relationship between ideology and affective polarization. Concretely, we test the predominance of identity over issues in explaining affective polarization in a multi-party system, taking advantage of an original panel dataset (E-DEM, 2018-2019) collected in Spain. The main results show that ideological identity and affective polarization strongly reinforce each other over time, polarizing society in identity terms but no so much due to conflicts emerging for issue positioning and sorting. Issue-based ideology exerts more modest affective polarizing effects, and only among those individuals whose positions in concrete issues are quite in line with their ideological identity.

May 18, 2021, at 11h (CET) via Zoom.

Mobile optimization strategies: effects on web survey participation

Presenter: Marc Asensio (RECSM-Universitat Pompeu Fabra)

Abstract: 
In recent years, smartphone penetration has continuously increased around the world and a growing part of the population relies exclusively on smartphones to access the internet. Coverage rates for web surveys are better than ever, but to maximize response rates, efforts must be made to adapt survey designs to accommodate ‘smartphone dependent’ participants.  Furthermore, to capitalize on new data collection opportunities offered by mobile devices, there is an interest in actively encouraging and normalizing responding to surveys on mobiles in the general population. A range of mobile optimization strategies are available for this purpose, but not much is known about their relative effectiveness and impact on survey costs and errors. We address this question in the present study, through a comparison of three strategies to optimize mobile device experience in web surveys, which were implemented in a probability-based, three-wave election study conducted in Switzerland in 2019: (1) the standard approach of providing a URL to a browser-based survey and optimizing the display of the questionnaire on smartphones (N= 8000); (2) adapting the invitation to promote mobile response and providing a QR code to access the survey (N= 1088); and (3) providing a QR code to download and participate via a smartphone application (N=1087). We compare the three mobile optimization strategies to draw conclusions about their relative impact on a) response rates and sample composition – overall and on mobile devices; b) estimates for target survey variables; and c) the progression of fieldwork, to draw conclusions about which strategy provides the best balance in terms of cost efficiency and representation of the target population.

November 23, 2021, at 12h (CET) via Zoom.

Willingness to participate in in-the-moment surveys triggered by online behaviours

Presenter: Carlos Ochoa (RECSM-Universitat Pompeu Fabra)

Abstract: 
Surveys are a fundamental tool of empirical research, but they suffer from errors: in particular, respondents can have difficulties recalling information of interest for researchers. Latest technological developments offer new opportunities to collect data passively (i.e., without participant’s intervention), avoiding errors of recall. Registering online behaviours (e.g., visited URLs) by means of a ‘meter’ software voluntarily installed by a sample of individuals on their browsing devices, is one of these opportunities. However, metered data is also affected by errors and cannot cover all the information of interest. Asking participants about such missing information by means of web surveys conducted in the precise moment an event of interest is detected by the meter has the potential to fill the gap. However, this method requires participants to be willing to participate.

In this webinar, the results of recent research on the willingness to participate in in-the-moment web surveys triggered by metered data will be presented. A conjoint experiment implemented in an opt-in metered panel in Spain (N=804) revealed overall high levels of willingness to participate, ranging from 69% to 95%, depending on the conditions offered to participants. The main aspects affecting this willingness are related to the incentives offered. Differences across participants were observed for household size, education, and personality traits. Answers to open questions also confirmed that the incentive is the key driver to decide to participate, while other potential problematic aspects such as the limited time to participate, privacy concerns, and discomfort caused by being interrupted play a limited role.

Finally, participants were also asked about their preferences in the method used to be invited to participate in in-the-moment surveys. The results showed that panelists are willing to accept several invitation methods, being those using smartphones the ones obtaining higher levels of acceptance and coverage, as well as the ones that most panelists consider they would see first.

December 14, 2021, at 12h (CET) via Zoom.

Adjusting to the survey: How interviewer experience relates to interview duration

Presenter: André Pirralha (RECSM-Universitat Pompeu Fabra)

Abstract: 

Interviewers are important actors in telephone surveys. Several studies have shown interviewers' importance in determining the interview pace and managing the effort respondents dedicate to answering. On the other hand, we also know that interviewers are very heterogeneous regarding the duration of the interviews and that the time dedicated to each interview tends to shorten over the course of fieldwork. While several hypotheses have been discussed in the literature, it is often argued that interviewers show a learning effect and optimize survey administration as they gain within-survey experience.

This paper examines the relationship between general survey experience, within-survey experience and interview duration using data from wave 1 of the parents Computer-Assisted Telephone Interviews from the National Educational Panel Study (NEPS), Starting Cohort Grade 9. We employ multilevel models that show considerable influence of the interviewers on the interview duration and find that interview duration decreases as the within-survey experience increases. This effect is robust even after controlling for various interviewer, respondent, and interview characteristics.

October 13, 2020 at 11h.
New Experiment On The Use Of Images To Answer Web Survey Questions [Abstract]
Presenter: Oriol Bosch. (RECSM-Universitat Pompeu Fabra)

November 17, 2020 at 11h.
Estimating the size of measurement errors of the “Satisfaction With Democracy” Survey Indicator for different scales, countries and languages [Abstract]
Presenter: Carlos Poses. (RECSM-Universitat Pompeu Fabra)

December 15, 2020 at 11h.
Linguistic complexity of survey questions [Abstract]
Presenter: Diana Zavala-Rojas. (RECSM-Universitat Pompeu Fabra)
Article co-authored with Toni Badia and Carme Colominas (GliCom - UPF) 

Internet use is linked to slower decline in cognitive functioning: evidence from the health and retirement study

Organized by  RECSM  and Demosc. January 29, 2017. 12h, Room 24.S05 (Mercè Rodoreda building). Presented by Valeria Bordone.

Abstract

Over the last two decades, the digital revolution brought changes in the everyday lives of individuals across the world. While the use of digital technologies has been picked up fast by younger cohorts, also older adults have increasingly adopted it. In this study, we ask whether Internet use is linked to cognitive functioning among older adults. Using panel data from the U.S. Health and Retirement Study and individual fixed-effect models controlling for activity status and self-reported health, we show that using the Internet is associated with slower declines in measures of cognitive functioning for both women and men. Additionally, we investigate the “digital divide”, by accounting for sub-groups of the population, defined by age and educational attainment. 

RonR! Second Seminar Series. An Introduction to the Tidyverse

Organized by RonR! working group (Jorge Cimentada, Robert T. Lange and Bruno Arpino). Program. (February-March, 2017). Presentations:

• Seminar 1: "Data Exploration". Presented by: Jorge Cimentada. (February 16, 2017)
• Seminar 2: "Data cleaning made easy". Presented by: Jorge Cimentada. (February 23, 2017)
• Seminar 3: "ggplot2: The grammar of graphics". Presented by: Robert T. Lange. (March 2, 2017)
• Seminar 4: "Automating everything: we should always strive for it!". Presented by: Jorge Cimentada. (March 9, 2017)
• Seminar 5: "Making your analysis reproducible". Presented by: Jorge Cimentada. (March 16, 2017) 

Pooling time series based on slightly different questions about life satisfaction. Application of the reference distribution method to time series on life satisfaction

Presented by Tineke de Jonge (Senior Volunteer of the World Database of Happiness at the Erasmus University Rotterdamn). (December 21, 2016)

Abstract: Subjective well-being has traditionally been measured using verbal response scales, yet, these verbal scales have not kept up with the present trend to use numerical response scales. A switch from a verbal scale to a numerical scale, however, causes a severe problem for trend analyses, due to the incomparability of the old and new measurements. The Reference Distribution Method has been developed recently to deal with this comparison problem. The idea that underlies this method is that the estimated means for equivalent questions about the same topic asked in different representative surveys in one year should be approximately the same when compared irrespective of the primary response scales used. In the Reference Distribution Method use is made of a reference distribution based on the responses to a numerical 10- or 11-point scale which is used to decide at which points response options from a verbal scale transit from one state to an adjacent state, for example from ‘fairly satisfied’ to ‘very satisfied’. Next, for each wave of the time series in which the verbal scale is used, a population mean is estimated using all these transition points and the responses in this wave. These estimates are on a level that is comparable to that of the mean of the reference distribution and are appropriate for use in an extended time series based on the response frequencies measured using a verbal and a numerical scale. We applied the Reference Distribution Method to pool time series on life satisfaction in the USA, Japan, The Netherlands and Spain, using results from the World Values Survey to derive reference distributions from. This resulted in consistent time series spanning a time period of almost 60 years for Japan, 40 years for The Netherlands and 35 years for the USA and Spain. Life satisfaction in Japan and The Netherlands was almost equal in the eighties, but at present differs more than one point in favor of The Netherlands. Life satisfaction in Spain reached the lowest value of 6.0 in 2012, but has increased since then to 6.3 in 2015.

Why should we and how could we study migrants

Ineke Stoop (Senior methodologist at The Netherlands Institute for Social Research/SCP and Deputy Director Methodological of the European Social Survey). (December 21, 2016)

Abstract: To be able to fully understand migration, information from migrants is required. This information should cover their socio-demographic and socio-economic characteristics and cultural backgrounds, but also their reasons for migration, their aspirations and preferences, and their experiences in the country they migrate to. This information can partly be drawn from registers, but in many cases surveys (and sometimes qualitative studies) will be required to fully grasp why people migrate, how they are treated, what they expect and whether these expectations have been met. Studying migration is far from easy. Firstly, the target population may be hard to define in advance. When are temporary labour migrants migrants? Secondly, not all migrants are registered in the country where they move to, for several reasons. In addition, migrants living in hostels and other non-residential living conditions are often excluded from general social surveys. Thirdly, migrants may be harder to contact, may refuse to participate in a survey, and may not be able to participate for specific reasons, including language problems. Deploying native speaking interviewers and translated questionnaire may solve some of these problems, but will almost inevitably result in measurement problems and may possibly hamper comparability across groups. The presentation will outline the differing aims when surveying migrants, efforts that can be made to increase representativeness, and the challenges of cross-national and cross-group comparability. Guidelines for comparative survey design and implementation will be presented, and examples from Dutch surveys among migrants.

The Roles of Media Choice and Media Effects in Political Knowledge Gaps

Presented by Thomas J. Leeper (Assistant professor in Political Behaviour in the Depatment of Goverment at the London School of Economics). (July 5, 2016)

Abstract: Mass media are frequently cited as having the potential to reduce political knowledge gaps between citizens, but are also seen as a force for segmentation, disengagement, and widening gaps. If media have no effect on political knowledge, gaps between the engaged and disengaged persist regardless of who is exposed to news. But gaps can also persist if those who are inclined to seek out news learn just as much or as little as those who are disinclined to attend to the news but happen across it incidentally because uniform gains in knowledge do not reduce knowledge inequalities. Indeed, gaps can only be closed by media if the effects of news exposure are larger for those who tend to avoid media exposure than for those who are inclined to attend to the news. Yet past research on political communication has not sufficiently acknowledged the connection between knowledge gaps, media choice, and the heterogeneity of media effects. The present study contributes a novel and large-scale choice-based experiment on knowledge of the ongoing crisis in Syria that finds media effects are relatively homogeneous across those with different media preferences, suggesting that under most conditions knowledge gaps between the politically engaged and disengaged are widened or at least sustained even when everyone learns from the news. Under no realistic conditions are gaps between these groups overcome by media use.

Can Citizens Think Coherently about Politics? The Problems of Preference Reversal and Political Ideology

Presented by Paul M. Sniderman (Fairleigh S. Dickinson, Jr. Professor in Public Policy). (3 May, 2016)

Abstract: It is the judgment of some of the most distinguished public opinion researchers that citizens are not capable of forming genuine preferences about major political issues.  It is the consensus of most public opinion specialists that ordinary citizens are not capable of organizing their political ideas along ideological lines. The obstacle is the minimal amount of attention that most citizens pay to politics and therefore the minimal amount of information that most have about it.  I shall show that changes in political preferences that have appeared to be changes for no good reason often are changes for good reason and  propose a solution to the inadequate information problem.

RonR! First Seminar Series

Organized by RonR! working group (Jorge Cimentada and Bruno Arpino). Program. (April-May, 2016). Presentations:

• Seminar 1: "An overview of the strengths of R". Presented by: Jorge Cimentada. (April 13, 2016)
• Seminar 2: "Getting familiar with R objects". Presented by: Basilio Moreno and Jorge Cimentada. (April 20, 2016)
• Seminar 3: "Manipulating and processing data". Presented by: Daniel Ciganda. (April 27, 2016)
• Seminar 4: "An introduction to R programming". Presented by: Bruno Arpino. (May 11, 2016)
• Seminar 5: "An introduction to sequence analysis". Presented by: Albert Julià. (May 17, 2016) 

Cross-National Surveys: the example of the European Social Survey

Presented by Ineke Stoop (Senior methodologist at The Netherlands Institute for Social Research/SCP Deputy Director Methodological of the European Social Survey). (October 7, 2015)

Abstract: Cross-national surveys play an important role in our societies. They are at the base of comparative social research and provide information on key issues in our societies. To fulfil this role they must pursue two potentially conflicting aims: high quality and optimal comparability across countries and over time. The presentation will focus on the European Social Survey, an academically driven face-to-face survey, started in 2002 and conducted biennially in more than 30 European countries. It presents the aims of the ESS, the history and the organisation, and then focuses on the content and the methodology. The main theme will be the pursuit of quality and comparability, given time, budgetary and practical constraints. The presentation will also comprise a short overview of other cross-national surveys.

Multilevel Models for Comparative Longitudinal Survey Data: Techniques, Problems, and Advice

Presented by Malcolm Fairbrother (Senior Lecturer in Global Policy and Politics University of Bristol). (May 5, 2015)

Abstract: Many surveys of respondents from multiple countries or sub-national regions have now been fielded on multiple occasions. This presentation will first describe two useful techniques for analysing such datasets, using multilevel models. These techniques treat y as a function of some time-varying x and/or of a time-invariant x, thereby illuminating the correlates of social or political change. The presentation will then address some potential problems in multilevel models fitted to comparative longitudinal survey data, showing that in published research such models have often been specified erroneously. They have often omitted one or more relevant random effects, thereby ignoring important clustering in the data, leading to downward (anticonservative) biases in the standard errors. Monte Carlo simulations and empirical examples illustrate both the techniques and the problems, and point to some simple recommendations for making reliable inferences from multilevel models.

Demography, Destiny, and American Politics:  How Latinos, Immigration, and the Changing Face of America Will Define the Next US Election

Presented by Gary Segura (Professor​, Department of Political Science​, Stanford University and​ Principal and Co-Founder ​Latino Decisions). (April 21, 2015)

Abstract: The future shape of American politics will be determined, in large measure, by how Latinos are incorporated into the political system.  The combination of population growth and growth in the electorate, combined with Latino electoral history as a population with significant inter-election movement over time, suggests that both political parties face opportunity in their approach to current and future generations of Latino voters. Latino population growth is coupled with a broader-scale diversification of the US on racial and ethnic lines.  It is true that Latinos are a complex group and that complexity makes for a politics more nuanced and less lock-step than the media and casual observers might conclude.  Nevertheless, over the last several elections, there can be little question that Latinos have become a political force-perhaps still weaker than their potential but a force nonetheless.  It is in this context that the current policy debate over immigration and immigrants--current and yet ongoing for most of the last 20 years--has become the critical litmus test by which Latinos evaluate candidates as "friendly," a test that (with few exceptions) the Republican Party and its candidates have repeatedly failed.  In this presentation, I will illustrate the demographic and issue bases of Latino political power and what they might portend for the next period of American elections.​​

Highlights in Survey Research

ESS  Seminar. (October 10, 2014). Presentations:

• "One survey in 30 countries: the challenges of achieving equivalence in the European Social Survey now and in the future". Presented by Rory Fitzgerald (Director European Social Survey ERIC).
• "The Future of Survey Research". Presented by Jon A. Krosnick  (Stanford University, USA).
• "Measurement Quality in the Social Sciences - Standards for Quality Assessment". Presented by Beatrice Rammstedt (GESIS, Germany).
• "Not All Methods are Created Equal". Presented by Charles E. Lance (University of Georgia, USA)

The future of social change models in social science

Presented by Manuel Voelkle (Max Planck Institute for Human Development, Germany), Marc Delsing (Radboud University Nijmegen, The Netherlands) and Han Oud (Radboud University Nijmegen, The Netherlands). (March 8, 2013) 

Treatment of attrition in panel designs and other panel issues"

Presented by Peter Lynn (University of Essex, U.K.), Annette Scherpenzeel (CentERdata / Tilburg University, The Netherlands) and William van der Veld (Radboud University, The Netherlands). (September 27, 2012)

​Causal inference for policy evaluation: case studies and statistical complications

Presented by Leonardo Grilli (University of Florence), Michela Bia (CEPS/INSTEAD, Luxembourg) and Bruno Arpino (RECSM - UPF). (March 30, 2012)  

Probability sampling vs. non-probability sampling

Presented by Matthias Ganninger (GESIS-Leibniz Institute for the Social Sciences, Mannheim, Germany), Jaak Billiet (Catholic University of Leuven, Belgium) and José Luis Melero (TNSGlobal - Spain). (June 1, 2011)

"Comparative research: Advantages and disadvantages of the different methodological approaches

Presented by Bart Meuleman (University of Leuven), Elmar Schlüter (WZB Berlin) and Marco Steenbergen (University of Bern). (February 11, 2011)

The Quality of Measures frequently used in Longitudinal Studies

Presented by William van der Veld (Radboud University Nijmegen, the Netherlands) and Toni Toharudin (Padjadjaran University, Indonesia). (December 17, 2010)

New Directions In Survey Research

Program. (March-May, 2010). Presentations:

• Seminar 1: "Embedding Experiments in Surveys". Presented by: Alexander Kuo (Juan March Institute, Stanford University), Marieke van Londen (Radboud University Nijmegen), and Marcel Coenders (Utrecht University). (March 1, 2010)
• Seminar 2: "Combining Genetic and Public Opinion Data". Presented by: Chris Dawes (University of California, San Diego), and Peter John Loewen (University of Toronto). (March 22, 2010)
• Seminar 3: "Potential and Limits of Online Surveys". Presented by: Willem Saris (Universitat Pompeu Fabra), Annette Scherpenzeel (CentERdata, Tilburg University), and Melanie Revilla (Universitat Pompeu Fabra) (May 4, 2010)