Conference Agenda

In recent years, numerous meta-analyses on teaching quality have highlighted inconsistent findings among studies that share similar research questions and constructs (e.g. Praetorius et al., 2018). However, they differ in how the same construct is measured, which has been shown to systematically influence the found effect sizes (Seidel & Shavelson, 2007). The present study investigates how different operationalizations of teaching quality and student outcomes affect observed teaching effects, while controlling for additional sources of variance such as the statistical model, study design and setting.

Unlike meta-analyses, which rely on reported results from previously analysed data, we plan on systematically re-analysing about 20 existing datasets on teaching quality and independently compute effect sizes, thus maintaining a constant statistical model. To disentangle the impact of operationalization from additional sources of variance, we first categorize the empirical data with respect to the setting and study design. Afterwards, we analyse how varying operationalizations of teaching quality and student outcomes affect the variance in standardized effect sizes. The findings of this investigation will provide insights for improving consistency in research by highlighting the relative importance of operationalization for the heterogeneity of effects in teaching quality research. The findings will be presented at the conference.

Educational data mining refers to the application of machine learning methods to educational data sets. Using data from the nationally representative German National Trends in Student Achievement Study 2021 (IQB-Bildungstrend 2021; Stanat et al., 2022), this secondary data analysis examines the characteristics of low-achieving 4^th-graders in reading and mathematics during the 2020/21 school year. The focus is on the questions of (a) which combinations of variables are most important for predicting failure to meet the minimum standards and (b) the extent to which PREs provide a better predictive performance than other tree-based methods and a regression analysis.

Educational Data Mining techniques were applied to analyze two data sets with N_r = 24,500 cases in reading and N_m = 24,511 in math, utilizing 103 predictor variables. Two separate models were developed using Prediction Rule Ensembles (PRE, Fokkema, 2020), an innovative machine learning method combining rules from a random-forest-like approach with linear predictors in a lasso-regularized regression.

The results revealed associations with socio-economic and motivational factors, as well as pandemic-related aspects, such as the proportion of in-person teaching and technical implementation at home. Further exploration indicated that students with limited cultural capital face less risk of underachievement in classes with good participation of all students during distance learning. The accuracy of PRE (measured by mean squared error loss) was found to be comparable to that of a random forest, yet not significantly better than a lasso-regularized GLM. Nevertheless, the potential for interpretation in PRE remains superior to that of both models.

Background: The network approach to mental disorders, which focuses on specific symptoms and their interactions, has gained in popularity over the last decade. Symptom interactions, in particular, may play a key role in the development and maintenance of mental disorders. These symptom interactions might also change over time, for example, through interventions. To account for these changes, network models with time-varying parameters have been developed. However, these models are highly complex and typically assume continuous change. An alternative could be modelling discrete phase transitions in symptom interactions, for instance, using regime-switching models, which capture distinct states (e.g., strong vs. weak symptom interactions) with different parameters.

Objective: This study explores existing methods for modeling phase transitions in longitudinal network models, as well as general approaches of modeling phase transitions in time-series, that could be adapted to the network approach.

Method: This scoping review examines time-series models with phase transitions in the behavioral sciences. The literature was screened for models that capture variable interactions and are suitable for common behavioral science data. The advantages and disadvantages of different models for use in network-theoretical approaches are evaluated. To test the models, data from a randomized controlled psychotherapy study with patients suffering from chronic depression are analyzed.

Results: The study provides an overview of existing methods of analyzing phase transitions in time-series and their applications in the behavioral sciences. Furthermore, it offers initial insights into how phase-transitioning network models can be estimated and used to model the course and treatment of mental disorders.

One key challenge in ambulatory assessment (AA) - the collection of intensive longitudinal data in daily life - is the occurrence of careless responding, which can compromise data quality. Research on the prevalence and predictors of careless responding in AA remains limited, and it is an open question whether clinical subpopulations (e.g., individuals with chronic pain) differ from healthy controls in their tendency to respond carelessly. Using a multilevel latent class analysis (ML-LCA) approach, we modeled momentary careless responding in AA and individual differences therein to examine whether individuals with vs. without chronic pain differ in the type and extent of careless responding. The sample included 233 individuals with chronic pain and 168 individuals without, who completed a 14-day AA phase with five measurement occasions per day. Across both groups, we identified three profiles of momentary careless responding at Level 1 (“careful responding”, “fast & invariable responding”, and “inconsistent responding”) and four Level-2 respondent classes (“careful”, “infrequently careless”, and two types of “frequently careless” respondents). A multigroup extension of the ML-LCA model revealed partial measurement invariance for the Level-1 profiles and full measurement invariance for the Level-2 classes. In additional analyses, we predicted profile and class membership using Level 1 and Level 2 covariates. While momentary pain intensity, momentary stress, and daily sleep quality (Level 1), as well as executive functioning (Level 2), were not significantly associated with careless responding, aggregated stress and aggregated daily sleep quality (both Level 2) significantly predicted class membership.