RECSM frequently organizes seminars about methodological issues and substantive social science research here at University Pompeu Fabra.
January 19, 2021, at 11h (CET) via Zoom.
Open question formats: Comparing the suitability of requests for text and voice answers in smartphone surveys [Abstract]
Presenter: Jan Karem Höhne (University of Mannheim, RECSM-Universitat Pompeu Fabra)
Annelies Blom (University of Mannheim), Konstantin Gavras (University of Mannheim), Melanie Revilla (RECSM-Universitat Pompeu Fabra), Leonie Rettig (University of Mannheim)
February 23, 2021, at 11h via Zoom.
Are you paying for or with quality? Survey participation due to monetary incentives and measurement quality – Evidence from the GESIS Panel.
Presenter: Hannah Schwarz (RECSM-Universitat Pompeu Fabra)
In times of decreasing response rates, monetary incentives are increasingly used to motivate individuals to participate in surveys. Receiving an incentive can affect respondents’ motivation to take a survey and, consequently, their survey taking behaviour. On the one hand, the resulting extrinsic motivation might undermine intrinsic motivation thus leading respondents to invest less effort into answering a survey. On the other hand, monetary incentives could make respondents more eager to invest effort into answering a survey, as they feel they are compensated for doing so. This study aims to assess whether there are differences in measurement quality between respondents who are motivated to take surveys due to the received incentive and respondents for who this is not a reason for participation. We implemented two Multitrait-Multimethod (MTMM) experiments in the probability-based GESIS Panel in Germany (2019) to be able to estimate the measurement quality of 18 questions asked to panelists. By coding panelists’ open answers to a question about their reasons for participation, we distinguish panelists who state that they are motivated by the incentive from those who do not. We analyse the MTMM experiments for these two groups separately and compare the resulting measurement quality estimates.
March 16, 2021, at 13h (CET) via Zoom.
Affective polarization: its measurement in multi-party contexts and its relationship with ideology
Presenter: Josep Maria Comellas (RECSM - Universitat Pompeu Fabra)
Mariano Torcal (RECSM Director - Universitat Pompeu Fabra)
Affective polarization broadly refers to the extent that individuals feel sympathy towards in-groups and antagonism towards out-groups. While this topic has been extensively studied in the United States, affective polarization has increasingly received comparative attention in an attempt to study this phenomenon in multi-party settings. In the first part of the presentation, we revise some of the main indices proposed in the literature to measure affective polarization, and we explain the ones that we have implemented using different datasets (CNEP, CSES, E-DEM). Then, in the second part, we present a paper that is focused on the relationship between ideology and affective polarization. Concretely, we test the predominance of identity over issues in explaining affective polarization in a multi-party system, taking advantage of an original panel dataset (E-DEM, 2018-2019) collected in Spain. The main results show that ideological identity and affective polarization strongly reinforce each other over time, polarizing society in identity terms but no so much due to conflicts emerging for issue positioning and sorting. Issue-based ideology exerts more modest affective polarizing effects, and only among those individuals whose positions in concrete issues are quite in line with their ideological identity.
April 6, 2021, at 15h (CET) via Zoom.
[MCSQ]: The Multilingual Corpus of Survey Questionnaires.
Presenter: Danielly Sorato (RECSM-Universitat Pompeu Fabra)
The Multilingual Corpus of Survey Questionnaires (MCSQ) is the first publicly available corpus of international survey questionnaires, comprising survey items from the European Social Survey (ESS), European Values Study (EVS), and the Survey of Health Ageing and Retirement in Europe (SHARE). The recently released Version 2.0 (entitled Mileva Marić-Einstein) is composed of questionnaires of the aforementioned studies in the (British) English source language and their translations into eight languages, namely Catalan, Czech, French, German, Norwegian, Portuguese, Spanish, and Russian, as well as 29 language varieties (e.g. Swiss-French). The MCSQ is a relevant digital artefact, that allows researchers in the fields of social sciences and linguistics to quickly search and compare survey items in a myriad of languages.
The MCSQ was developed in the SSHOC (Social Sciences & Humanities Open Cloud) Project. It forms part of the EU Horizon 2020 Research and Innovation Programme (2014-2020) and is conducted under Grant Agreement No. 823782.
The digitalized survey items are an interesting resource for survey research, translation studies, lexicology, among others. In this seminar, we present the corpus characteristics and showcase applications of the MCSQ.
May 18, 2021, at 11h (CET) via Zoom.
Mobile optimization strategies: effects on web survey participation
Presenter: Marc Asensio (RECSM-Universitat Pompeu Fabra)
In recent years, smartphone penetration has continuously increased around the world and a growing part of the population relies exclusively on smartphones to access the internet. Coverage rates for web surveys are better than ever, but to maximize response rates, efforts must be made to adapt survey designs to accommodate ‘smartphone dependent’ participants. Furthermore, to capitalize on new data collection opportunities offered by mobile devices, there is an interest in actively encouraging and normalizing responding to surveys on mobiles in the general population. A range of mobile optimization strategies are available for this purpose, but not much is known about their relative effectiveness and impact on survey costs and errors. We address this question in the present study, through a comparison of three strategies to optimize mobile device experience in web surveys, which were implemented in a probability-based, three-wave election study conducted in Switzerland in 2019: (1) the standard approach of providing a URL to a browser-based survey and optimizing the display of the questionnaire on smartphones (N= 8000); (2) adapting the invitation to promote mobile response and providing a QR code to access the survey (N= 1088); and (3) providing a QR code to download and participate via a smartphone application (N=1087). We compare the three mobile optimization strategies to draw conclusions about their relative impact on a) response rates and sample composition – overall and on mobile devices; b) estimates for target survey variables; and c) the progression of fieldwork, to draw conclusions about which strategy provides the best balance in terms of cost efficiency and representation of the target population.
November 23, 2021, at 12h (CET) via Zoom.
Willingness to participate in in-the-moment surveys triggered by online behaviours
Presenter: Carlos Ochoa (RECSM-Universitat Pompeu Fabra)
Surveys are a fundamental tool of empirical research, but they suffer from errors: in particular, respondents can have difficulties recalling information of interest for researchers. Latest technological developments offer new opportunities to collect data passively (i.e., without participant’s intervention), avoiding errors of recall. Registering online behaviours (e.g., visited URLs) by means of a ‘meter’ software voluntarily installed by a sample of individuals on their browsing devices, is one of these opportunities. However, metered data is also affected by errors and cannot cover all the information of interest. Asking participants about such missing information by means of web surveys conducted in the precise moment an event of interest is detected by the meter has the potential to fill the gap. However, this method requires participants to be willing to participate.
In this webinar, the results of recent research on the willingness to participate in in-the-moment web surveys triggered by metered data will be presented. A conjoint experiment implemented in an opt-in metered panel in Spain (N=804) revealed overall high levels of willingness to participate, ranging from 69% to 95%, depending on the conditions offered to participants. The main aspects affecting this willingness are related to the incentives offered. Differences across participants were observed for household size, education, and personality traits. Answers to open questions also confirmed that the incentive is the key driver to decide to participate, while other potential problematic aspects such as the limited time to participate, privacy concerns, and discomfort caused by being interrupted play a limited role.
Finally, participants were also asked about their preferences in the method used to be invited to participate in in-the-moment surveys. The results showed that panelists are willing to accept several invitation methods, being those using smartphones the ones obtaining higher levels of acceptance and coverage, as well as the ones that most panelists consider they would see first.
December 14, 2021, at 12h (CET) via Zoom.
Adjusting to the survey: How interviewer experience relates to interview duration
Presenter: André Pirralha (RECSM-Universitat Pompeu Fabra)
Interviewers are important actors in telephone surveys. Several studies have shown interviewers' importance in determining the interview pace and managing the eﬀort respondents dedicate to answering. On the other hand, we also know that interviewers are very heterogeneous regarding the duration of the interviews and that the time dedicated to each interview tends to shorten over the course of ﬁeldwork. While several hypotheses have been discussed in the literature, it is often argued that interviewers show a learning eﬀect and optimize survey administration as they gain within-survey experience.
This paper examines the relationship between general survey experience, within-survey experience and interview duration using data from wave 1 of the parents Computer-Assisted Telephone Interviews from the National Educational Panel Study (NEPS), Starting Cohort Grade 9. We employ multilevel models that show considerable inﬂuence of the interviewers on the interview duration and ﬁnd that interview duration decreases as the within-survey experience increases. This eﬀect is robust even after controlling for various interviewer, respondent, and interview characteristics.
November 17, 2020 at 11h.
Estimating the size of measurement errors of the “Satisfaction With Democracy” Survey Indicator for different scales, countries and languages [Abstract]
Presenter: Carlos Poses. (RECSM-Universitat Pompeu Fabra)
December 15, 2020 at 11h.
Linguistic complexity of survey questions [Abstract]
Presenter: Diana Zavala-Rojas. (RECSM-Universitat Pompeu Fabra)
Article co-authored with Toni Badia and Carme Colominas (GliCom - UPF)
Internet use is linked to slower decline in cognitive functioning: evidence from the health and retirement study
Organized by RECSM and Demosc. January 29, 2017. 12h, Room 24.S05 (Mercè Rodoreda building). Presented by Valeria Bordone.
Over the last two decades, the digital revolution brought changes in the everyday lives of individuals across the world. While the use of digital technologies has been picked up fast by younger cohorts, also older adults have increasingly adopted it. In this study, we ask whether Internet use is linked to cognitive functioning among older adults. Using panel data from the U.S. Health and Retirement Study and individual fixed-effect models controlling for activity status and self-reported health, we show that using the Internet is associated with slower declines in measures of cognitive functioning for both women and men. Additionally, we investigate the “digital divide”, by accounting for sub-groups of the population, defined by age and educational attainment.
RonR! Second Seminar Series. An Introduction to the Tidyverse
Organized by RonR! working group (Jorge Cimentada, Robert T. Lange and Bruno Arpino). Program. (February-March, 2017). Presentations:
- Seminar 1: "Data Exploration". Presented by: Jorge Cimentada. (February 16, 2017)
- Seminar 2: "Data cleaning made easy". Presented by: Jorge Cimentada. (February 23, 2017)
- Seminar 3: "ggplot2: The grammar of graphics". Presented by: Robert T. Lange. (March 2, 2017)
- Seminar 4: "Automating everything: we should always strive for it!". Presented by: Jorge Cimentada. (March 9, 2017)
- Seminar 5: "Making your analysis reproducible". Presented by: Jorge Cimentada. (March 16, 2017)
Pooling time series based on slightly different questions about life satisfaction. Application of the reference distribution method to time series on life satisfaction
Presented by Tineke de Jonge (Senior Volunteer of the World Database of Happiness at the Erasmus University Rotterdamn). (December 21, 2016)
Abstract: Subjective well-being has traditionally been measured using verbal response scales, yet, these verbal scales have not kept up with the present trend to use numerical response scales. A switch from a verbal scale to a numerical scale, however, causes a severe problem for trend analyses, due to the incomparability of the old and new measurements. The Reference Distribution Method has been developed recently to deal with this comparison problem. The idea that underlies this method is that the estimated means for equivalent questions about the same topic asked in different representative surveys in one year should be approximately the same when compared irrespective of the primary response scales used. In the Reference Distribution Method use is made of a reference distribution based on the responses to a numerical 10- or 11-point scale which is used to decide at which points response options from a verbal scale transit from one state to an adjacent state, for example from ‘fairly satisfied’ to ‘very satisfied’. Next, for each wave of the time series in which the verbal scale is used, a population mean is estimated using all these transition points and the responses in this wave. These estimates are on a level that is comparable to that of the mean of the reference distribution and are appropriate for use in an extended time series based on the response frequencies measured using a verbal and a numerical scale. We applied the Reference Distribution Method to pool time series on life satisfaction in the USA, Japan, The Netherlands and Spain, using results from the World Values Survey to derive reference distributions from. This resulted in consistent time series spanning a time period of almost 60 years for Japan, 40 years for The Netherlands and 35 years for the USA and Spain. Life satisfaction in Japan and The Netherlands was almost equal in the eighties, but at present differs more than one point in favor of The Netherlands. Life satisfaction in Spain reached the lowest value of 6.0 in 2012, but has increased since then to 6.3 in 2015.
Why should we and how could we study migrants
Ineke Stoop (Senior methodologist at The Netherlands Institute for Social Research/SCP and Deputy Director Methodological of the European Social Survey). (December 21, 2016)
Abstract: To be able to fully understand migration, information from migrants is required. This information should cover their socio-demographic and socio-economic characteristics and cultural backgrounds, but also their reasons for migration, their aspirations and preferences, and their experiences in the country they migrate to. This information can partly be drawn from registers, but in many cases surveys (and sometimes qualitative studies) will be required to fully grasp why people migrate, how they are treated, what they expect and whether these expectations have been met. Studying migration is far from easy. Firstly, the target population may be hard to define in advance. When are temporary labour migrants migrants? Secondly, not all migrants are registered in the country where they move to, for several reasons. In addition, migrants living in hostels and other non-residential living conditions are often excluded from general social surveys. Thirdly, migrants may be harder to contact, may refuse to participate in a survey, and may not be able to participate for specific reasons, including language problems. Deploying native speaking interviewers and translated questionnaire may solve some of these problems, but will almost inevitably result in measurement problems and may possibly hamper comparability across groups. The presentation will outline the differing aims when surveying migrants, efforts that can be made to increase representativeness, and the challenges of cross-national and cross-group comparability. Guidelines for comparative survey design and implementation will be presented, and examples from Dutch surveys among migrants.
The Roles of Media Choice and Media Effects in Political Knowledge Gaps
Presented by Thomas J. Leeper (Assistant professor in Political Behaviour in the Depatment of Goverment at the London School of Economics). (July 5, 2016)
Abstract: Mass media are frequently cited as having the potential to reduce political knowledge gaps between citizens, but are also seen as a force for segmentation, disengagement, and widening gaps. If media have no effect on political knowledge, gaps between the engaged and disengaged persist regardless of who is exposed to news. But gaps can also persist if those who are inclined to seek out news learn just as much or as little as those who are disinclined to attend to the news but happen across it incidentally because uniform gains in knowledge do not reduce knowledge inequalities. Indeed, gaps can only be closed by media if the effects of news exposure are larger for those who tend to avoid media exposure than for those who are inclined to attend to the news. Yet past research on political communication has not sufficiently acknowledged the connection between knowledge gaps, media choice, and the heterogeneity of media effects. The present study contributes a novel and large-scale choice-based experiment on knowledge of the ongoing crisis in Syria that finds media effects are relatively homogeneous across those with different media preferences, suggesting that under most conditions knowledge gaps between the politically engaged and disengaged are widened or at least sustained even when everyone learns from the news. Under no realistic conditions are gaps between these groups overcome by media use.
Can Citizens Think Coherently about Politics? The Problems of Preference Reversal and Political Ideology
Presented by Paul M. Sniderman (Fairleigh S. Dickinson, Jr. Professor in Public Policy). (3 May, 2016)
Abstract: It is the judgment of some of the most distinguished public opinion researchers that citizens are not capable of forming genuine preferences about major political issues. It is the consensus of most public opinion specialists that ordinary citizens are not capable of organizing their political ideas along ideological lines. The obstacle is the minimal amount of attention that most citizens pay to politics and therefore the minimal amount of information that most have about it. I shall show that changes in political preferences that have appeared to be changes for no good reason often are changes for good reason and propose a solution to the inadequate information problem.
RonR! First Seminar Series
Organized by RonR! working group (Jorge Cimentada and Bruno Arpino). Program. (April-May, 2016). Presentations:
- Seminar 1: "An overview of the strengths of R". Presented by: Jorge Cimentada. (April 13, 2016)
- Seminar 2: "Getting familiar with R objects". Presented by: Basilio Moreno and Jorge Cimentada. (April 20, 2016)
- Seminar 3: "Manipulating and processing data". Presented by: Daniel Ciganda. (April 27, 2016)
- Seminar 4: "An introduction to R programming". Presented by: Bruno Arpino. (May 11, 2016)
- Seminar 5: "An introduction to sequence analysis". Presented by: Albert Julià. (May 17, 2016)
Cross-National Surveys: the example of the European Social Survey
Presented by Ineke Stoop (Senior methodologist at The Netherlands Institute for Social Research/SCP Deputy Director Methodological of the European Social Survey). (October 7, 2015)
Abstract: Cross-national surveys play an important role in our societies. They are at the base of comparative social research and provide information on key issues in our societies. To fulfil this role they must pursue two potentially conflicting aims: high quality and optimal comparability across countries and over time. The presentation will focus on the European Social Survey, an academically driven face-to-face survey, started in 2002 and conducted biennially in more than 30 European countries. It presents the aims of the ESS, the history and the organisation, and then focuses on the content and the methodology. The main theme will be the pursuit of quality and comparability, given time, budgetary and practical constraints. The presentation will also comprise a short overview of other cross-national surveys.
Multilevel Models for Comparative Longitudinal Survey Data: Techniques, Problems, and Advice
Presented by Malcolm Fairbrother (Senior Lecturer in Global Policy and Politics University of Bristol). (May 5, 2015)
Abstract: Many surveys of respondents from multiple countries or sub-national regions have now been fielded on multiple occasions. This presentation will first describe two useful techniques for analysing such datasets, using multilevel models. These techniques treat y as a function of some time-varying x and/or of a time-invariant x, thereby illuminating the correlates of social or political change. The presentation will then address some potential problems in multilevel models fitted to comparative longitudinal survey data, showing that in published research such models have often been specified erroneously. They have often omitted one or more relevant random effects, thereby ignoring important clustering in the data, leading to downward (anticonservative) biases in the standard errors. Monte Carlo simulations and empirical examples illustrate both the techniques and the problems, and point to some simple recommendations for making reliable inferences from multilevel models.
Demography, Destiny, and American Politics: How Latinos, Immigration, and the Changing Face of America Will Define the Next US Election
Presented by Gary Segura (Professor, Department of Political Science, Stanford University and Principal and Co-Founder Latino Decisions). (April 21, 2015)
Abstract: The future shape of American politics will be determined, in large measure, by how Latinos are incorporated into the political system. The combination of population growth and growth in the electorate, combined with Latino electoral history as a population with significant inter-election movement over time, suggests that both political parties face opportunity in their approach to current and future generations of Latino voters. Latino population growth is coupled with a broader-scale diversification of the US on racial and ethnic lines. It is true that Latinos are a complex group and that complexity makes for a politics more nuanced and less lock-step than the media and casual observers might conclude. Nevertheless, over the last several elections, there can be little question that Latinos have become a political force-perhaps still weaker than their potential but a force nonetheless. It is in this context that the current policy debate over immigration and immigrants--current and yet ongoing for most of the last 20 years--has become the critical litmus test by which Latinos evaluate candidates as "friendly," a test that (with few exceptions) the Republican Party and its candidates have repeatedly failed. In this presentation, I will illustrate the demographic and issue bases of Latino political power and what they might portend for the next period of American elections.
Highlights in Survey Research
ESS Seminar. (October 10, 2014). Presentations:
- "One survey in 30 countries: the challenges of achieving equivalence in the European Social Survey now and in the future". Presented by Rory Fitzgerald (Director European Social Survey ERIC).
- "The Future of Survey Research". Presented by Jon A. Krosnick (Stanford University, USA).
- "Measurement Quality in the Social Sciences - Standards for Quality Assessment". Presented by Beatrice Rammstedt (GESIS, Germany).
- "Not All Methods are Created Equal". Presented by Charles E. Lance (University of Georgia, USA)
The future of social change models in social science
Presented by Manuel Voelkle (Max Planck Institute for Human Development, Germany), Marc Delsing (Radboud University Nijmegen, The Netherlands) and Han Oud (Radboud University Nijmegen, The Netherlands). (March 8, 2013)
Treatment of attrition in panel designs and other panel issues"
Presented by Peter Lynn (University of Essex, U.K.), Annette Scherpenzeel (CentERdata / Tilburg University, The Netherlands) and William van der Veld (Radboud University, The Netherlands). (September 27, 2012)
Causal inference for policy evaluation: case studies and statistical complications
Presented by Leonardo Grilli (University of Florence), Michela Bia (CEPS/INSTEAD, Luxembourg) and Bruno Arpino (RECSM - UPF). (March 30, 2012)
Probability sampling vs. non-probability sampling
Presented by Matthias Ganninger (GESIS-Leibniz Institute for the Social Sciences, Mannheim, Germany), Jaak Billiet (Catholic University of Leuven, Belgium) and José Luis Melero (TNSGlobal - Spain). (June 1, 2011)
"Comparative research: Advantages and disadvantages of the different methodological approaches
Presented by Bart Meuleman (University of Leuven), Elmar Schlüter (WZB Berlin) and Marco Steenbergen (University of Bern). (February 11, 2011)
The Quality of Measures frequently used in Longitudinal Studies
Presented by William van der Veld (Radboud University Nijmegen, the Netherlands) and Toni Toharudin (Padjadjaran University, Indonesia). (December 17, 2010)
New Directions In Survey Research
Program. (March-May, 2010). Presentations:
- Seminar 1: "Embedding Experiments in Surveys". Presented by: Alexander Kuo (Juan March Institute, Stanford University), Marieke van Londen (Radboud University Nijmegen), and Marcel Coenders (Utrecht University). (March 1, 2010)
- Seminar 2: "Combining Genetic and Public Opinion Data". Presented by: Chris Dawes (University of California, San Diego), and Peter John Loewen (University of Toronto). (March 22, 2010)
- Seminar 3: "Potential and Limits of Online Surveys". Presented by: Willem Saris (Universitat Pompeu Fabra), Annette Scherpenzeel (CentERdata, Tilburg University), and Melanie Revilla (Universitat Pompeu Fabra) (May 4, 2010)