To involve millennials in surveys and obtain quality answers, researchers need new tools that best capture the interest and attention of this population. Emojis have the potential to improve communication and make participating in a survey via smartphone more attractive: this type of survey is an increasingly common practice among millennials, who are also the population group that uses emojis the most.

Few studies have analysed what the implementation and use of emojis in smartphone surveys might involve. A pioneering investigation in this field by Oriol J.Bosch (London School of Economics and Political Science) and Melanie Revilla (Department of Political and Social sciences at UPF), both researchers linked to the Research and Expertise Centre for Survey Methodology (RECSM-UPF), has shed light on this.

“Our results have shown that millennials’ stated and actual willingness to use emojis to answer open survey questions is high”.

The research, recently published in the journal Quality & Quantity. International Journal of Methodology, has three main goals: to find out if millennials want to use emoijs in surveys and for what kinds of questions; if they really use them in a real survey environment when they are encouraged to do so, and if encouraging this population to use emojis to answer open questions has a positive impact on data quality, survey completion time and survey assessment.

“Our results have shown that millennials’ actual and statedwillingness to use emojis to answer open survey questions is high (they prefer this system to traditional scales with verbal tags or other visual scales) and that the impact of encouraging them to respond with emojis, in general, it is positive and promising”, the researchersaffirm.

Survey on some 1,600 millennials from Spain and Mexico

In an experiment embedded within a general survey, some participants (treatment group) were encouraged to answer six open questions using emojis, compared to others who were not directly encouraged to do so (control group). The respondents of the treatment group were provided with an emoji keyboard placed below the text box to ensure that they saw the same rendering across different devices and operating systems.

In total, 1,614 respondents, young people between 16 and 34 years with regular access to the Internet via a smartphone,completed the survey, 808 in Spain and 806 in Mexico. Data collection was conducted in June 2017, via the online fieldwork company Netquest.

Preference for emojis, which convey more information

Some of the results of the survey were that a high proportion of millennials expressed their willingness to use emojis in different contexts of the surveys in both countries: in general, the percentage was 88.1% in Mexico and 75.5% in Spain. In the responses to the survey, the actual use of emojis in the treatment group was 98.5% in both countries, more than had been expected based on stated willingness.

The impact of encouraging the treatment group to use emojis in open questions was positive, since the average information conveyed was greater for this group in the two countries, compared to the control group. Specifically, an increase of 35.7% in information was shown for Spain and 64.3% for Mexico.

For every additional emoji used, respondents conveyed 0.15 more points of information and devoted an extra 3.5 seconds of time.

Regarding response time to questions, it was higher for the treatment group (increases of 38.6% for Spain and 39.6% for Mexico). This may be due to the fact that searching for the desired emoji or providing more information (for example, the same text plus an emoji) lengthens completion time.

“A greater number of emojis used to answer open questions significantly increases the amount of information conveyed and completion time”, the authors state. Thus, for every additional emoji used, respondents conveyed 0.15 more points of information and devoted an extra 3.5 seconds of time.

Regarding survey assessment, the proportion of respondents who (very much) liked responding to this set of open questions is also bigger for the treatment group, with an increase of 19.7% in Spain and 5.8% in Mexico.

Emojis can be ambiguous and hinder interpreting the results

However, according to the researchers, the fact that emojis help to convey more information does not necessarily mean that the quality of the data is higher, and their potential ambiguity must be taken into account: “Some emojis present a high degree of sentiment and/or semantical misconstrual, and this may make the responses that include them more difficult to interpret”, they say.

This is especially problematic for messages based on emojis alone. “In our study, 37% of the experimental group answers contained only emojis and no text, and this has the potential to harm data quality”, the authors claim, stressing the need for further research to study the real use of emojis and their impact for other types of questions, topics and age cohorts.

Reference work: Oriol J. Bosch, Melaine Revilla (May 2020). “Using emojis in mobile web surveys for Millennials? A study in Spain and Mèxic“. Quality & Quantity