Conference Agenda

For more than 20 years, one of the main aims of the European Social Survey (ESS) has been to provide researchers, policy makers and the wider community high quality data measuring change (and stability) over time in Europe. A significant proportion of the quality measures in contemporary cross-national surveys such as the ESS is implemented at the early stages of the survey life cycle (the design phase), be it related to questionnaire design, sample design, fieldwork plans, translation or more technical aspects of the data specifications. While emphasizing the importance of these contributions to overall data quality, this session aims to highlight the data quality management in the ESS at the data collection and post-data collection stage. This may include well-known survey disciplines such as fieldwork monitoring and data processing, but also user contributions to data quality through what may be dubbed “the life cycle feedback-loop” in which exploration and analyses by data users result in new and improved versions of data and metadata, traditionally an important, if less communicated, part of the validation and quality enhancement of the ESS and surveys in general. Data processing, from interim data quality checks to post-collection data cleaning may often appear as a black box to researchers and other end data users. However, the ESS prides itself on its transparency via thorough documentation and communication. Hence, the proposed session aims to present in detail our data processing and quality control procedures, clarifying which procedures are used, and examine their impact on data quality. The session is primarily focused on interim and post-collection data management at the central level but we would also invite papers dealing with related issues on a decentral (country) level. Furthermore, we would welcome papers highlighting data user contributions to data quality in the ESS. Papers related to all these issues from other surveys would also be relevant to this session.

In cross- cultural survey projects, national questionnaires are usually developed by translating one or more source questionnaires into all relevant target languages. For the comparability of the data gathered by such multilingual survey projects, it is of utmost importance that the quality of all translations is of highest quality, and that all translated questionnaires do ideally “ask the same questions” as the source questionnaire(s). The ESS has been the first cross-cultural survey to rigorously implement the translation scheme “TRAPD” (consisting of the steps Translation, Review, Adjudication, Pretesting and Documentation), based on an interdisciplinary team or committee approach, since its beginning. Currently preparing its 12th round, translating its questionnaires has been a particular focus in all ESS rounds so far. By setting high standards to its translations and often being at the forefront of new questionnaire translation developments, the ESS’ translation scheme has been inspiring others within the community. This does not only refer to the translation process as such – including, for instance, its approaches to translate into shared languages or to systematically assess its translation qualities –, but also to developing its source questionnaire, which is formulated in a way to minimise later translation problems as much as possible by involving translation experts in the source questionnaire development and carrying out “advance translations”. Experimenting with and implementing innovations, e.g., in the field of translation technology, is a core element of the ESS translation approaches. This session invites presentations on various aspects related to questionnaire translation or survey language in a broader sense, whether linked to the ESS or not. Topics may cover different approaches or methods to translate questionnaires, to assess or measure translation quality; technical and other innovations in the field; looking closer at existing ESS translations, e.g., into shared languages, comparing or assessing particular translations or expressions; discussing translatability matters, also related to questionnaire design or pretesting; other linguistic or language-related topics, such as minority languages, choice of interview or questionnaire language, easy or plain language, gender aspects in language, or the influence of translation and/or language on survey results.

In terms of its methodological rigour and data quality, the ESS has been a benchmark international comparative face-to-face survey project for over 20 years. However, the upcoming transition from interviewer-administered to self-completion survey brings new methodological and ethical challenges to the ESS, with respondent engagement being one of them. To this end, new ways of respondent motivation in self-completion surveys need to be developed and evaluated thereby ensuring that the ESS remains the benchmark when it comes to methodological and data quality in international comparative surveys. In face-to-face surveys, trained interviewers can keep the respondent focused on the survey and can even stop the interview when the external context (circumstances) is not appropriate. It is much more challenging to keep the respondent focused (solely) on the survey in self-completion: a paper questionnaire offers no control or information on the respondent engagement or number of sittings, and Web surveys are conducted on devices with multiple applications constantly battling for user attention. While not true for all question types and all respondents, research into cognitive processes shows that respondent focus and response times generally affect the quality of the responses (Tourangeau, 1989; Yan and Tourangeau, 2007). Passive monitoring of the respondent by collecting paradata when they are completing a self-completion survey can provide vast and diverse information on their level of engagement. Still, it cannot increase the quality of responses while the interview is being completed in the way a trained interviewer can. Active monitoring and intervention (e.g. notifications when the respondent is speeding, straight-lining, not responding, etc.) can partly increase the respondent's focus but can also influence the responses or even willingness to complete the survey. Graphical approaches and gamification can make the survey more enjoyable. Still, the literature shows that the researchers should mainly focus on the fundamental components of respondent burden coming from the instrument itself (e.g. Guin, T. D.-L. et al, 2012). The session is focused on both instrument design to keep the respondents engaged and approaches to monitoring and assisting the respondent through the survey in an ethically appropriate manner with the aim to produce data of the highest possible quality.

General population surveys are currently challenged by several societal developments, such as budget constraints and the respondents’ more active lifestyle, which leads to a lower contact success rate and higher costs in interviewer-based survey designs. At the same time, internet penetration rates are increasing fast and steadily. General population surveys are therefore pushed to innovating the design and several experiments on different designs for fielding a general population survey on the web have been fielded for more than a decade now. Survey methodologists study mode effects between interviewer-based and web/paper self-completion for over a decade. For example, Switzerland has fielded a comprehensive mixed-mode experiment using the European Social Survey (ESS) in 2012, a complex experiment on push-to-web designs using the European Values Study has been fielded in six countries in 2017 and during the pandemic, the ESS has been fielded as a self-completion web/paper survey in several countries in 2021. Given the change towards self-completion of the ESS in 2027, several experiments based on the ESS questionnaire have been fielded or are in the field. This session welcomes contributions that show the effects of a mode switch on results of general population surveys with a special focus on changes over time. This includes two types of research questions: effects of a mode-switch on time-series data as well as the changes in effects of a mode-switch over time. The first type of research questions includes how to demonstrate a mode effect in a time-series, how to correct for mode effects, how to visualize time-series with a mode change in-between and many more. The second type of research questions includes changes over time in under- or overrepresentation of specific groups in the population, changes in, or persistency of, mode effects regarding some variables or change in the share of paper vs. web participation, mobile participation etc. We welcome contributions based on ESS data but also based on any other general population survey that provides insights into the effect of switching from an interviewer-based to a self-completion survey relevant to the mode-switch of ESS foreseen in 2027 (e.g., including items and concepts used also in the ESS, such as trust, attitudes towards democracy, immigration, family, or welfare state).

General population surveys are currently challenged by several societal developments, such as budget constraints and the respondents’ more active lifestyle, which leads to a lower contact success rate and higher costs in interviewer-based survey designs. At the same time, internet penetration rates are increasing fast and steadily. General population surveys are therefore pushed to innovating the design and several experiments on different designs for fielding a general population survey on the web have been fielded for more than a decade now. Survey methodologists study mode effects between interviewer-based and web/paper self-completion for over a decade. For example, Switzerland has fielded a comprehensive mixed-mode experiment using the European Social Survey (ESS) in 2012, a complex experiment on push-to-web designs using the European Values Study has been fielded in six countries in 2017 and during the pandemic, the ESS has been fielded as a self-completion web/paper survey in several countries in 2021. Given the change towards self-completion of the ESS in 2027, several experiments based on the ESS questionnaire have been fielded or are in the field. This session welcomes contributions that show the effects of a mode switch on results of general population surveys with a special focus on changes over time. This includes two types of research questions: effects of a mode-switch on time-series data as well as the changes in effects of a mode-switch over time. The first type of research questions includes how to demonstrate a mode effect in a time-series, how to correct for mode effects, how to visualize time-series with a mode change in-between and many more. The second type of research questions includes changes over time in under- or overrepresentation of specific groups in the population, changes in, or persistency of, mode effects regarding some variables or change in the share of paper vs. web participation, mobile participation etc. We welcome contributions based on ESS data but also based on any other general population survey that provides insights into the effect of switching from an interviewer-based to a self-completion survey relevant to the mode-switch of ESS foreseen in 2027 (e.g., including items and concepts used also in the ESS, such as trust, attitudes towards democracy, immigration, family, or welfare state).

In the past few years, large-scale surveys such as the ESS have faced issues with declining response rates and escalating costs associated with conducting surveys (Brick and Williams, 2013; de Leeuw, Hox and Luiten, 2018). In addition, the COVID-19 pandemic pushed researchers further to experiment with other methods. As a result, many of these projects (e.g., the EVS or the GSS) have started introducing self-administered modes and using a mixed-mode design as an alternative to traditional face-to-face data collections. Particularly, push-to-web approaches (Dillman, 2017) have so far been the most promising. While the primary method of the ESS remains face-to-face, the ESS has a clear objective to transition to a mixed-mode setting in the upcoming period. The use of multiple modes may come with benefits (lower costs, increase in response rates, decrease in certain types of sample biases), but can be a source of measurement error at the same time. Mode effects can impact time trends or introduce additional measurement error in county-level comparisons. Thus, understanding the consequences of mode shifts and finding optimal mixed-mode designs are critical to the ESS. In this session, we invite contributions which provide insight into the impact of survey mode on measurement errors or present findings related to mixed-mode design choices and questionnaire design.

The population of Europe and other OECD countries is aging rapidly, which means that exploring various challenges faced by retirees is crucial for developing both robust theoretical models as well as effective policies. Different types of pension systems may expose older individuals to increased risk of poverty. Moreover, the transition to retirement and aging are linked to various stressors, including changes in living conditions, family forms, and access to social networks. The aim of this session is to discuss the research on various facets of life after retirement. We focus on issues related to: 1. Pension systems and old-age poverty; 2. Family forms and financial and psychological well-being of the retirees; 3. Trust and old-age social exclusion; 4. Unionization of retired workers.